About this role
Salary: £35,000 - 35,000 per year
Requirements: Experience developing and managing PySpark workflows within AWS environments.Knowledge of AWS Lambda and event-driven solutions to support automated data processing.Experience provisioning and supporting cloud infrastructure using Amazon EC2 and automation tools.Understanding of Agile ways of working and experience contributing to CI/CD delivery practices.Ability to optimise data processing, storage, and queries to improve performance and data quality.Experience designing and automating ETL pipelines using PySpark and Spark SQL.Strong full stack development background with technologies including .NET 4.8, C#, JavaScript, C++, and Java. Responsibilities: Design, build, and maintain data products that support analytics, business intelligence, and operational decision-making across Debt Solutions.Develop and operate ETL pipelines in cloud-based environments.Integrate data from a variety of source systems and transform it into meaningful, trusted information.Use SQL and modern data engineering practices to create reusable, production-ready solutions.Help develop user-facing dashboards and reports that provide accessible, accurate information to customers and stakeholders.Work as part of a multidisciplinary agile team and contribute to innovative solutions that deliver impact at national scale.Support the modernisation of long-established MI and data services through a new cloud-powered platform. Technologies: AWSLambdaBusiness IntelligenceC#CI/CDCloudEC2ETLSupportJavaJavaScriptPySparkSQLSparkASP.NET More:
We are DWP Digital, the UKs largest government department, using fresh ideas and leading-edge technology to build and maintain digital solutions that support millions of people every day. Our Debt Solutions team helps recover around £3 billion of debt annually while protecting vulnerable citizens, and we are modernising our data services through a new cloud-powered platform. We offer hybrid working with 40% home working and 60% in a hub, a competitive salary of £38,772, a generous civil service pension, paid leave starting at 24 days rising over time, and a broad benefits package including volunteering time, shopping discounts, interest-free loans, and professional development opportunities. We are proud of our award-winning culture and commitment to diversity, inclusion, and career progression.
last updated 34 week of 2026