About this role
Salary: £? - ? per year
Requirements: Experience as a Lead Engineer, Principal Engineer, Staff Engineer, Software Architect, or similar senior technical role.Strong server-side software engineering experience, with flexibility across programming languages such as Python, Java, Go, Scala, C#, or similar.Experience designing and building distributed data services or large-scale backend platforms.Experience with Apache Spark, Apache DataFusion, Trino, Presto, or similar distributed data processing technologies.Strong architecture and design experience across distributed systems and data services.Experience designing scalable data processing or data streaming workflows.Experience building cloud-native applications on AWS, Azure, or GCP.Experience with Kubernetes, Docker, and containerised environments.Experience building and operating CI/CD pipelines and Infrastructure as Code.Strong understanding of distributed systems, scalability, resilience, and secure-by-design principles.Comfortable making architectural decisions within evolving, fast-paced environments.Excellent communication skills with the ability to work across technical and non-technical teams.Experience building federated data services or federated data platforms.Experience within healthcare, life sciences, research, government, or other highly regulated environments.Knowledge of Trusted Research Environments (TREs), Secure Data Environments (SDEs), or similar secure data platforms.Experience with data governance, metadata services, data catalogues, or secure data access models.Exposure to event-driven architectures and streaming technologies such as Kafka, Flink, or Spark Streaming.Experience evaluating or contributing to open-source technologies Responsibilities: Lead the design and development of scalable, cloud-native data services supporting a federated platform.Define architectural patterns, engineering standards, and best practices that will scale across the wider organisation.Design and build distributed server-side services using modern software engineering practices.Architect and develop federated data services capable of securely querying and processing data across multiple environments.Design high-performance data processing and streaming workflows using technologies such as Apache Spark, Apache DataFusion, or similar distributed query frameworks.Design and implement Kubernetes-based services and containerised workloads.Build secure APIs and backend services supporting authentication, authorisation, governance, and audit requirements.Contribute to the design of distributed systems, data pipelines, and platform architecture.Establish CI/CD pipelines, Infrastructure as Code, automated testing, and deployment practices.Evaluate and integrate open-source technologies and distributed data frameworks.Mentor engineers and promote engineering excellence across the team.Work closely with architecture, product, security, and external stakeholders. Technologies: AWSArchitectAzureBackendC#CI/CDCloudDockerFlinkGCPSupportJavaKafkaKubernetesPythonScalaSecuritySparkDevOps More:
We are hiring a Lead Backend Software Engineer for a high-profile programme building secure, cloud-native data services that will support data-driven research and innovation at national scale. This is a founding engineering role where we will shape the architecture, engineering standards, and core data services from the ground up. You will work closely with our Engineering Manager and a small team of senior engineers in a highly regulated environment. The role is based in London with 2 days per week onsite, offered as an initial 6-month contract with a likely extension, and the rate is £700–£800 per day outside IR35.
last updated 29 week of 2026