<p>Must Have Technical/Functional Skills</p> <p>ESSENTIAL QUALIFICATIONS</p> <ul> <li>Experience: Minimum of 5+ years of experience as a Data Engineer or ETL Developer in complex, large-scale data environments. </li><li>Databricks Expertise: Strong hands-on experience working with Databricks, including using Apache Spark for data processing and optimization. </li><li>ETL Tools: Proficient with DataBricks ETL tools and frameworks </li><li>Big Data Technologies: In-depth knowledge of big data processing frameworks like Spark, Kafka, etc. </li><li>NoSQL/MongoDB: Expertise in working with NoSQL databases, especially MongoDB, for large-scale data storage, retrieval, and optimization. </li><li>Programming Skills: Proficient in Python or Java, Power shell for building data pipelines. </li><li>NoSQL Proficiency: Experience working with both relational databases and NoSQL databases like MongoDB. </li><li>Data Modeling: Expertise in designing and implementing data models, including OLAP, OLTP, dimensional, and document-based models (NoSQL). </li><li>Data warehousing: Data warehousing is a key part of ETL process, as it stores data from multiple sources in an organized manner and will be needed to build a repeatable ETL workflow to support many different data sources. </li><li>Redesign and refactor legacy custom ETL processes to reusable ETL workflows that can ingest diverse data sources to normalize data to standard JSON / XML output. </li><li>Data Governance: Knowledge of data governance, security standards, and best practices for managing sensitive data. </li><li>Version Control: Experience with Git or other version control systems for code management. </li><li>Certification: Databricks certification, MongoDB certification, or other relevant certifications in data engineering, cloud platforms, or big data technologies. </li><li>Soft Skills: Strong problem-solving skills, excellent communication, and the ability to work in a collaborative team environment. </li><li>Analytical Mindset: Ability to translate business requirements into scalable, efficient, and reliable ETL solutions. </li><li>Solid understanding of legacy communication protocols and migration strategies. </li><li>Experience with cloud platforms like AWS, Azure, or Google Cloud </li></ul> <p>Roles & Responsibilities</p> <ul> <li>ETL Design and Development: Develop, maintain, and optimize ETL processes for data ingestion, transformation, and data warehousing across multiple platforms, including NoSQL databases. </li><li>Data Pipelines: Design, build, and manage scalable data pipelines using technologies like Databricks, Apache Spark, Python, and NoSQL databases. </li><li>NoSQL/MongoDB Expertise: Work with MongoDB to design efficient document schemas, implement query optimization, and handle large-scale unstructured data. </li><li>Data Integration: Collaborate with cross-functional teams to ensure seamless data integration between different sources such as databases (both relational and NoSQL), APIs, and external files. </li><li>Performance Optimization: Implement and monitor performance metrics, optimize data processing performance, and manage ETL job scheduling and dependencies. </li><li>Data Quality: Ensure data quality and integrity across ETL pipelines, implementing processes for data validation, cleansing, and enrichment. </li><li>Automation: Automate repeatable ETL tasks and data processing workflows to improve efficiency and accuracy. </li><li>Collaboration: Work closely with data architects, analysts, and business stakeholders to gather and understand data requirements. </li><li>Cloud Platforms: Leverage cloud services (Azure, GCP) for data storage, processing, and infrastructure management, ensuring scalability and cost efficiency. </li><li>Best Practices: Maintain documentation, adhere to data governance, and best practices in data management, including security and compliance. </li><li>Build Microservices APIS to expose ETL services . </li></ul> <p>Generic Managerial Skills, If any</p> <p>Leading the team and driving the projects for closure</p> <p>Salary Range: $90,000-$120,000 a year</p> <p>TCS Employee Benefits Summary:</p> <p>Discretionary Annual Incentive.</p> <p>Comprehensive Medical Coverage: Medical & Health, Dental & Vision, Disability Planning & Insurance, Pet Insurance Plans.</p> <p>Family Support: Maternal & Parental Leaves.</p> <p>Insurance Options: Auto & Home Insurance, Identity Theft Protection.</p> <p>Convenience & Professional Growth: Commuter Benefits & Certification & Training Reimbursement.</p> <p>Time Off: Vacation, Time Off, Sick Leave & Holidays.</p> <p>Legal & Financial Assistance: Legal Assistance, 401K Plan, Performance Bonus, College Fund, Student Loan Refinancing.</p> <p>#LI-SP1</p>