<p>Kforce has a client that is seeking a Data Engineer in Maryland Heights, MO.</p> <p>Overview: We are seeking a highly skilled and motivated Data Engineer to join our growing team. This role will focus on leveraging cutting-edge technologies to design and build scalable data solutions. The ideal candidate will have a strong background in cloud-driven environments, big data frameworks, and real-time data streaming technologies.</p> <p>Key Responsibilities:</p> <ul> <li>Data Engineer will design, develop, and deploy Spark pipelines from scratch using Scala </li><li>Migrate and process data to the cloud, predominantly on AWS </li><li>Implement and optimize batch processing and real-time streaming solutions, transitioning to a fully real-time environment in AWS </li><li>Utilize Apache Kafka (or AWS Kinesis) for real-time data streaming </li><li>As a Data Engineer, you will develop and manage databases using DynamoDB and other big data tools such as Hive and HBase </li><li>Write and optimize SQL queries, including CTEs (Common Table Expressions) and window functions </li><li>Collaborate with cross-functional teams to ensure seamless data integration and delivery* Hands-on experience with Spark, specifically in building pipelines from scratch </li><li>Experience with streaming technologies, particularly Kafka (or AWS Kinesis as an alternative) </li><li>Expertise in Scala for Spark development (not API development) </li><li>Solid understanding of DynamoDB in AWS for database management </li><li>Advanced SQL skills, including the use of CTEs and window functions </li></ul> <p>Preferred Skills:</p> <ul> <li>Experience working in a fully real-time data processing environment </li><li>Knowledge of additional big data technologies like Hive or HBase </li><li>Proficiency in AWS, with a strong focus on EMR </li><li>Familiarity with Spark on AWS EMR to optimize performance and scalability </li></ul>