About this role
Salary: £75,000 - 80,000 per year
Requirements: 10+ years of dedicated data engineering experience managing massive, complex datasets3+ years of deep, hands-on production experience with Azure Databricks, Spark (PySpark/Scala), and Delta Lake ecosystemsExtensive experience across the Azure data suite, specifically Azure Data Factory, Azure Blob Storage, and Azure SQL DatabaseStrong proficiency in Python, Spark, and SQLPractical experience using Azure Purview for governance and cataloguingWorking knowledge or exposure to R, YAML, and JavaScript within data workflowsExperience with event-driven data such as Kafka or Azure Event Hubs and modern DevOps tooling including Azure DevOps, Git, Docker, and KubernetesSolid understanding of both SQL and NoSQL database design, data warehousing principles, and data modelling techniquesExperience working within financial services, central banking, or an economic data environment is highly advantageousExceptional ability to bridge the gap between technical infrastructure and economic or analytical business logicMicrosoft Certified: Azure Data Engineer Associate or Databricks certifications are preferredSC Clearance required, active or eligible to undergo Responsibilities: Design, develop, and maintain robust, scalable ETL/ELT pipelines ingesting data from APIs, relational databases, streaming services, and financial data providersImplement advanced data processing logic for cleaning, enriching, and aggregating large-scale data using Spark and SQLFine-tune data workloads for maximum performance, throughput, and cloud cost-efficiencyDrive the implementation of Azure Databricks services, leveraging Unity Catalog and Delta Lake architecturesDevelop and maintain data solutions using Python, SQL, R, YAML, and JavaScriptLead hands-on implementation of Azure Purview to manage data quality, governance, metadata, and end-to-end data lineage trackingEstablish automated data validation, quality checks, and real-time alerting processes across all production environmentsPartner with DevOps teams to design, build, and maintain robust CI/CD pipelines for automated environmental deploymentsCollaborate closely with economists, data scientists, and senior analysts to translate complex analytical needs into production-ready data systemsDrive engineering excellence through active participation in code reviews, architectural discussions, and knowledge-sharing sessions Technologies: AzureCI/CDCloudDatabricksDevOpsDockerETLGitJavaScriptKafkaKubernetesNoSQLPythonPySparkSQLScalaSparkUnityGameDevSecurity More:
We are a London-based team working in a hybrid setup of 3 days in the office and 2 days remote. We are building and scaling our Azure Databricks platform to power our Monetary Analysis, Forecasting, and Modelling frameworks, with a focus on secure, reliable, and high-performance data infrastructure. This is a full-time, permanent senior role with a salary of £75,000 to £80,000 plus benefits and bonus, and it offers the opportunity to work closely with economists, data scientists, and senior analysts in a business-critical environment.
last updated 29 week of 2026