<p>Databricks Architect</p> <p>Must Have Technical/Functional Skills</p> <p>Experience: 5+ years of hands-on data engineering experience, with at least 3 years focused on the Databricks/Spark</p> <p>Ecosystem</p> <p>Databricks Expertise: Deep, hands-on expertise with the Databricks Lakehouse Platform, including Delta Lake,</p> <p>Structured Streaming, Delta Live Tables, and cluster configuration/optimization.</p> <p>Programming Mastery: Expert-level proficiency in Python and PySpark. Advanced SQL skills are essential.</p> <p>Data Warehousing Concepts: Strong understanding of data modeling principles, including dimensional modeling</p> <p>(Kimball), data warehousing concepts, and ETL/ELT design patterns.</p> <p>Cloud Proficiency: Proven experience working with a major cloud provider (Azure, AWS, or GCP), particularly with</p> <p>data storage S3 and related services.</p> <p>Software Engineering Mindset: Experience with software engineering best practices, including version control (Git),</p> <p>code reviews, testing, and CI/CD.</p> <p>Roles and Responsibilities</p> <p>Data Pipeline Development: Design, code, and deploy robust and scalable batch and streaming data pipelines</p> <p>using PySpark, Spark SQL, and Delta Live Tables to ingest data from sources such as Point-of-Sale (POS), e-commerce</p> <p>platforms, loyalty systems, and marketing clouds.</p> <p>Data Modeling and Transformation: Implement complex data transformations and business logic within the Medallion</p> <p>architecture (Bronze, Silver, Gold layers). Build and optimize the final "Gold" customer-dimension tables that will</p> <p>serve as the single source of truth.</p> <p>Data Quality: Implement data quality frameworks and cleansing routines to ensure the accuracy and trustworthiness</p> <p>of the Customer 360 data.</p> <p>Performance Optimization: Proactively monitor, debug, and tune Databricks jobs and Spark clusters for performance</p> <p>and cost-efficiency. Implement best practices for partitioning, caching, and data layout in Delta Lake.</p> <p>Infrastructure as Code (IaC) & CI/CD: Work with DevOps teams to manage Databricks environments, clusters, and</p> <p>job deployments using tools like Terraform and AWS DevOps/GitHub Actions. Champion and implement CI/CD best</p> <p>practices for data pipelines.</p> <p>Data Governance and Security: Implement data governance features within Databricks Unity Catalog, including</p> <p>data lineage tracking, access controls, and data masking to ensure compliance and security.</p> <p>Collaboration: Partner closely with Functional Consultants, Data Scientists, and Analytics Engineers to understand</p> <p>their data requirements and deliver well-structured, consumption-ready datasets.</p> <p>Education</p> <p>Bachelors</p> <p>Salary Range: $120000 - $150000 a year</p>