About this role
Salary: £? - ? per year
Requirements: Strong commercial experience as a Data Engineer or Senior Data EngineerExpert-level Python development skillsExtensive hands-on experience with Databricks in enterprise environmentsStrong Apache Spark and PySpark expertiseAdvanced SQL development and optimisation skillsExperience designing and building large-scale ETL and data integration solutionsStrong understanding of distributed computing concepts and data processing frameworksExperience with modern cloud platforms, including Azure, AWS or GCPKnowledge of software engineering principles, testing and code quality practicesExperience working in Agile development environmentsExcellent problem-solving and troubleshooting capabilitiesDesirable: experience in investment banking, financial services or other highly regulated environmentsDesirable: exposure to risk, trading, treasury or regulatory data domainsDesirable: experience with Kafka, streaming technologies or real-time data processingDesirable: Databricks certificationsDesirable: experience with Airflow, Databricks Workflows or similar orchestration toolsDesirable: CI/CD pipeline development and DevOps practicesDesirable: experience with Docker and KubernetesDesirable: experience supporting data science or machine learning workloads Responsibilities: Design, develop and maintain scalable data pipelines using Python, Databricks and PySparkBuild and optimise ETL/ELT solutions that process large-scale datasetsDevelop high-quality, reusable code and engineering frameworksDeliver robust, performant and maintainable data solutionsTroubleshoot and resolve complex production and performance issuesEnhance and modernise enterprise data platformsDevelop and support Databricks-based data solutions and Lakehouse architecturesUse Delta Lake and Spark to deliver scalable data processing capabilitiesOptimise distributed data processing workloads for performance and reliabilityImplement data quality controls and monitoring solutionsContribute to platform stability, scalability and operational excellenceWork with business and technology stakeholders to understand requirements and deliver effective solutionsParticipate in Agile delivery activities, including sprint planning, refinement and estimationSupport strategic data initiatives across multiple business areasCollaborate with architects, engineers and product teams to deliver high-quality outcomesIdentify and resolve technical risks and implementation challengesPromote engineering best practices, including testing, code reviews and version controlSupport CI/CD adoption and deployment automationContribute to coding standards and engineering governanceImprove development and support processesMentor junior engineers and share knowledge within the team Technologies: AirflowAWSAzureCI/CDCloudDatabricksDevOpsDockerETLGCPSupportKafkaKubernetesMachine LearningPythonPySparkSQLSpark More:
We are recruiting for a global financial institutions growing data engineering function, which delivers critical data platforms and analytics capabilities across the business. This hands-on contract role offers the opportunity to work on complex data challenges in a modern cloud environment alongside engineers, architects and business stakeholders. The contract is for six months, with a very likely extension, at £850 per day inside IR35. The role is hybrid, based near Liverpool Street in London, with three days per week in the office. GCS is acting as an Employment Business in relation to this vacancy.
last updated 40 week of 2026