Our growing data teams are seeking a savvy Senior Data Engineer to join them and help build and evolve the next generation of Audience & Identity data platforms that handle data at scale and use state-of-the-art technologies to unlock the data's business value.If you're excited about working with cutting-edge technology in a fast-paced environment, we'd love to have you on board. Don't hesitate — join our team today!CustomerOur client has superior end-to-end technology, a premium marketplace, and best-in-market advisory services that power the world's advertising businesses of the largest media and entertainment companies. For example, Fox, NBC Universal, Viacom in the USA, Sky, Channel 4, RTE, and Mediaset in Europe.ProjectWe are working on a solution to simplify segment ingestion, audience construction, and activation for targeted placements and campaigns. Our platform bridges the gap by combining buyers' consumer insights with sellers' media viewership and geographic data, addressing identity fragmentation and unifying audiences across devices.ResponsibilitiesDesign, implement, and maintain scalable data pipelines to ingest and transform large volumes of data from various sourcesOptimize streaming and batch data processing and storage solutions for performance and scalabilityMonitor system performance, troubleshoot issues, and implement solutions to ensure high availability and reliabilityCollaborate with product managers and other stakeholders to understand data requirements and deliver data solutions that meet business needsCommunicate technical concepts and solutions effectively to non-technical stakeholdersStay abreast of emerging technologies and best practices in data engineering. Identify opportunities to improve data processes, tools, and infrastructure to enhance efficiency and effectivenessDocument data pipelines, processes, and systems to ensure clarity and maintainabilityShare knowledge and best practices with team members to foster a culture of learning and collaborationWork with and mentor Junior Data EngineersRequirements5+ years of hands-on experience in the Software Development field and/or Big DataStrong experience in Scala or JavaProficient in Databricks with a solid foundation in SparkExperience with CI/CD tools (Jenkins)Familiarity with shell, ability to work well with command lines, and a solid knowledge of data structures, algorithms, and system designWork experience with ETL pipelinesStrong skills in SQL and experience with at least one of the major database platformsAt least an Upper-Intermediate level of EnglishWould be a plusExperience with Python, and GolangExperience in the media or advertising industryExperience with graph databases (GraphX, Neo4J) and related analysisExperience with Kubernetes, Terraform, and ELKExperience with SnowflakeAWS certification and previous experience with AWS services such as S3, EMR, EKS, Glue, EC2, IAM, CloudFormation, Athena, Lambda, and others
