About this role
to
Position: Senior Data Engineer (Databricks & Cloud Data Platforms)
Reporting to: Practice Head: Data and Automation
ROLE OVERVIEW
As a Senior Data Engineer, you will play a critical consulting role in designing, building, and modernising enterprise data platforms, with a strong focus on Databricks lakehouse implementations across cloud environments.
You will work closely with solution architects, analysts, data scientists, and client stakeholders to deliver secure, scalable, and high-performing analytics platforms that drive real business impact.
This is a hands-on technical role suited to someone passionate about solving complex data challenges and delivering best-in-class cloud-native solutions.
responsibilities
RESPONSIBILITIES
• Design and implement scalable Databricks-based data platforms
• Build robust ETL / ELT pipelines for batch and streaming workloads
• Lead data migration and modernisation initiatives from legacy platforms to cloud-based lakehouse architectures
• Develop high-performance data processing solutions using PySpark, Python, and SQL
• Implement Delta Lake, Unity Catalog, and modern governance frameworks including Purview
• Design enterprise-grade lakehouse and medallion architectures
• Optimise Spark workloads for performance, scalability, and cost efficiency
• Build orchestration solutions using Databricks Workflows, ADF, Airflow, or similar tools in Azure/AWS
• Collaborate with technical and business stakeholders to define solution requirements
• Mentor junior engineers and contribute to technical best practices within the Data & AI practice
• Produce technical documentation, architecture artefacts, and reusable delivery assets
IDEALLY YOU ARE:
• A strong technical leader who can own complex delivery outcomes
• Comfortable engaging directly with clients and stakeholders
• A proactive problem-solver with excellent analytical skills
• Able to clearly communicate technical concepts to both technical and non-technical audiences
• Passionate about mentoring and uplifting engineering teams
• Delivery-focused, quality-driven, and adaptable in consulting environments
MINIMUM EXPERIENCE
• 8+ years in Data Engineering projects
• 3+ years of hands-on Databricks delivery experience
• Deep expertise in Python and PySpark
• Strong understanding of the Medallion Architecture and Lakehouse Architecture
• Hands-on experience building and maintaining live Databricks AI/BI Governance Dashboards using SQL
• Strong experience with Azure data services including Synapse, ADF, OneLake, Purview
• Practical exposure to AWS data platforms including Glue, S3, Redshift
• Proven track record delivering data migration and modernisation projects
• Experience working in consulting or client-facing environments
• Bachelor’s degree in computer science, Engineering, Information Systems, or equivalent practical experience
TECHNICAL REQUIREMENTS
• Databricks Expertise (Essential)
• Cloud Data Platform Experience (Azure Fabric, AWS) (Essential)
300-450