About this role
Salary: £70,000 - 80,000 per year
Requirements: 5+ years of data engineering experience.Experience delivering cloud-based data platforms.Experience supporting migration of data workloads between environments.Experience working with Infrastructure as Code using Terraform.Experience working within regulated or enterprise-scale environments.PythonPySparkSQLData pipeline developmentETL / ELT designData modelling fundamentalsData quality and validationAWSS3IAMCloud-based data platformsTerraformInfrastructure automationGitCI/CD conceptsAgile deliveryDevOps principlesSC clearance or SC eligibility is mandatory.UK citizenship or the right to work in the UK, with at least 5 years residence in the UK.Desirable: Databricks, Palantir Foundry, Foundry Pipeline Builder, ontology concepts, Delta Lake, healthcare or NHS data experience, Secure Data Environment (SDE) experience, cloud migration programme experience. Responsibilities: Support the migration of data pipelines, notebooks, and data processing workloads between cloud environments.Analyse existing data engineering solutions and identify migration requirements.Develop and maintain data processing pipelines using modern cloud data platform technologies.Support testing, validation, and reconciliation of migrated datasets.Work with structured and semi-structured data stored in cloud environments.Troubleshoot and resolve data processing issues across development, test, and production environments.Support the operation and enhancement of cloud-based data platforms.Assist with onboarding datasets, users, and workloads onto target platforms.Collaborate with platform and architecture teams to implement scalable data solutions.Support platform optimisation, monitoring, and operational readiness activities.Develop and maintain Terraform code supporting data platform deployments.Support deployment of data platform components, jobs, permissions, and environment configurations.Assist with automation of deployment and release processes.Maintain infrastructure consistency across environments.Support cloud migration initiatives and platform transformation programmes.Contribute to platform standards, engineering practices, and reusable deployment patterns.Produce technical documentation and implementation guides. Technologies: AWSCI/CDCloudDatabricksDevOpsETLGitIAMSupportPythonPySparkSQLTerraformSecurity More:
We are driving a major cloud data modernisation programme to migrate legacy Databricks pipelines and workloads to a modern AWS infrastructure. We work closely with architects and stakeholders to build automated, large-scale data solutions that secure and transform critical healthcare data assets. This is a hybrid role based in Leeds or London, with 3 days per week onsite, and the salary is £70,000 to £80,000 plus depending on experience. SC clearance or SC eligibility is required.
last updated 29 week of 2026