Posted 17 days ago
Description
Location: Remote with occasional site visits in either London or Birmingham
- Build, maintain and optimise Databricks data pipelines using PySpark, SQL and Delta Lake to transform raw data into curated, business-ready datasets.
- Manage production deployments through Azure DevOps (ADO), including code reviews, testing, release management and work tracking.
- Monitor and troubleshoot data pipelines, resolving issues and optimising performance, reliability and cost.
- Implement data quality checks and governance controls using Unity Catalog to ensure data integrity, security and compliance.
- Collaborate with business stakeholders and the Platform team to translate requirements into technical solutions, maintaining clear documentation and support runbooks.
- Proven commercial experience as a Databricks Engineer, Data Engineer or Data Platform Engineer, with hands-on experience supporting production environments.
- Strong technical expertise in Databricks, PySpark, SQL and Delta Lake, including data transformation and pipeline optimisation.
- Experience with Azure DevOps (ADO), including CI/CD pipelines, code reviews, automated testing and production releases.
- Knowledge of Unity Catalog, data governance, data quality validation and production monitoring.
- Strong problem-solving and communication skills, with the ability to work collaboratively with business and technical teams
Interviews: October 2026
Start Date: November 2026
#J-18808-Ljbffr
