<p>Must Have Technical/Functional Skill</p> <ul> <li>Design, develop, and maintain cloud native data engineering solutions using Azure Databricks. </li><li>Build and manage PySpark notebooks to process large scale structured and semi structured datasets. </li><li>Design, create, and maintain Delta Lake tables, ensuring data reliability, ACID transactions, and schema enforcement. </li><li>Develop scalable data workflows and pipelines using Databricks notebooks and orchestration patterns. </li><li>Optimize performance of Spark jobs, including tuning partitions, memory usage, caching strategies, and query execution. </li><li>Work extensively with PySpark and Spark SQL, choosing the appropriate approach based on use case and performance needs. </li><li>Support cloud data migration initiatives, migrating data pipelines from on prem or legacy platforms to Azure Databricks. </li><li>Integrate Databricks with upstream and downstream systems (e.g., data sources, storage layers, reporting tools). </li><li>Ensure data pipelines are robust, reusable, and maintainable, following enterprise data engineering best practices. </li><li>Implement error handling, logging, monitoring, and recovery strategies for production grade data pipelines. </li><li>Collaborate with data architects, analysts, and downstream consumers to understand data requirements. </li><li>Perform debugging and root cause analysis for data quality, performance, or pipeline failures. </li><li>Support testing, validation, and reconciliation of data during development, migration, and production phases. </li><li>Follow security, governance, and compliance standards applicable to cloud data platforms. </li><li>Actively participate in Agile/Scrum delivery, owning data engineering stories from development through deployment. </li><li>Maintain documentation for notebooks, workflows, data models, and migration approaches. </li></ul> <p>Roles & Responsibilities</p> <ul> <li>Develop and maintain data engineering solutions using Azure Databricks and PySpark. </li><li>Create, enhance, and optimize Databricks notebooks for data ingestion, transformation, and aggregation. </li><li>Design and manage Delta Lake tables and pipelines supporting analytics and reporting use cases. </li><li>Support cloud data migrations, including data validation and performance benchmarking. </li><li>Optimize Spark jobs for performance, scalability, and cost efficiency. </li><li>Collaborate with platform, DevOps, and data governance teams to ensure environment stability. </li><li>Perform data pipeline testing and validation, ensuring correctness and completeness. </li><li>Troubleshoot and resolve issues related to Spark jobs, Delta tables, and workflow execution. </li><li>Participate in code reviews and enforce data engineering best practices. </li><li>Support production deployments and post deployment stabilization. </li><li>Provide inputs to data architecture and platform improvement initiatives. </li><li>Mentor junior data engineers when required. </li></ul> <p>Salary Range $120,000-$140,000 Per year</p> <p>TCS Employee Benefits Summary:</p> <p>Discretionary Annual Incentive.</p> <p>Comprehensive Medical Coverage: Medical & Health, Dental & Vision, Disability Planning & Insurance, Pet Insurance Plans.</p> <p>Family Support: Maternal & Parental Leaves.</p> <p>Insurance Options: Auto & Home Insurance, Identity Theft Protection.</p> <p>Convenience & Professional Growth: Commuter Benefits & Certification & amp; Training Reimbursement.</p> <p>Time Off: Vacation, Time Off, Sick Leave & Holidays.</p> <p>Legal & Financial Assistance: Legal Assistance, 401K Plan, Performance Bonus, College Fund, Student Loan Refinancing.</p> <p>#LI-SP1</p>