Senior Data Engineer

Responsibilities Design, build, and maintain scalable data pipelines on Azure Databricks and Google Cloud Platform (GCP) using PySpark and Python. Work with Azure and GCP platforms, specifically managing services like Azure Databricks, Azure SQL Server, BigQuery, DataProc, and Azure Machine Learning. Develop and maintain CI/CD pipelines using GitHub Actions to automate the deployment of data engineering solutions. Utilize GitHub for version control, ensuring that all code changes are tracked, reviewed, and managed effectively. Work closely with cross-functional teams to define requirements, design solutions, and deliver high-quality data engineering outputs. Skills Must have It is expected that the engineer: Is proficient in Data Engineering concepts (Object Storage, Data Pipelines, ELTs, Data Modeling, etc.) Is proficient in Software Engineering concepts (Continuous Integration, Continuous Deployment, Unit Testing, Coverage, Linting, Version Control System, etc.) Have knowledge or understanding the basic concepts of Software Development lifecycle (Release Engineering, Deployments, Code review, Requirements definition) Platforms - Azure Databricks, SQL Server, Azure Machine Learning, Azure Blob Storage Programming Language - Python, PySpark CI/CD - GitHub Actions, GitHub VCS - GitHub LANGUAGES English: B2 Upper Intermediate