About this role
Salary: £60,000 - 80,000 per year
Requirements
- Strong commercial experience developing production software in Python.
- Significant experience designing, building and maintaining data ingestion and ETL/ELT pipelines.
- Strong SQL skills and experience with relational databases, particularly PostgreSQL.
- Strong experience with Snowflake and dbt, including data modelling, transformation, testing and optimisation.
- Experience integrating third-party APIs and processing data from multiple formats and sources.
- Strong automated testing, debugging and code review experience.
- Practical experience using AI coding assistants or agents in a professional development workflow.
- Ability to work independently, make sound technical decisions and take ownership of projects.
- Clear communication skills and a collaborative, pragmatic approach to engineering.
- Desirable experience with AWS services such as S3, Lambda, ECS, RDS and OpenSearch; web scraping and browser automation; extracting structured information from PDFs and other complex documents; Docker, CI/CD pipelines and infrastructure automation; or integrating language models into production data workflows.
Responsibilities
- Design, build and maintain production-grade data ingestion and transformation pipelines using Python.
- Integrate data from third-party APIs, web scraping, CSV, Excel, JSON, PDFs and other structured or unstructured sources.
- Develop reliable extraction and processing workflows for complex documents, including financial statements and company information.
- Build reusable components for data validation, transformation and enrichment, and develop integrations with OpenSearch and supporting data services.
- Improve the efficiency, scalability and cost-effectiveness of existing pipelines, working with PostgreSQL and Snowflake as primary data storage and processing technologies.
- Take responsibility for pipeline reliability and data quality by implementing monitoring, logging, alerting, error handling and recovery processes.
- Identify and resolve data quality issues, investigate production incidents, determine root causes and implement lasting fixes.
- Maintain clear technical documentation so systems can be understood and supported by others.
- Use AI coding agents and assistants for implementation, refactoring, debugging, testing and documentation; critically review generated code for production suitability.
- Write and maintain automated tests, and evaluate AI models and AI-assisted approaches for data processing while balancing accuracy, reliability, cost and operational complexity.
- Work with the CTO to agree architecture, priorities and delivery approaches; break down requirements, estimate work, communicate progress and raise risks.
- Participate in technical discussions, architecture reviews and peer code reviews, and deliver agreed work through deployment and ongoing support.
Technologies
- AI
- AWS
- Lambda
- OpenSearch
- CI/CD
- CTO
- Docker
- ETL
- Excel
- Support
- JSON
- PostgreSQL
- Python
- SQL
- Snowflake
- Web
- dbt
- Cloud
- AI Agents
- Security
More
We are mnAi, a data and analytics business providing intelligence on companies, people and markets. Our platform brings together information from diverse sources to support research, analysis and business decision-making, and our small engineering team builds and operates the systems that collect, process, enrich and deliver that information. This is a full-time, permanent Senior Data Engineer role based in Crawley, reporting to the CTO, with a salary of £60,000–£80,000 depending on experience. We offer hybrid working, with a local office that encourages in-person collaboration, 25 days of holiday plus bank holidays, a pension scheme, free and on-site parking, and the option to work from home.
We value engineering fundamentals, responsible AI-assisted development, independent ownership and clear communication over rigid processes. The role offers meaningful influence over technical decisions and our data platform, access to modern AI development tools, and opportunities to develop expertise in a small, collaborative team with minimal bureaucracy. The application asks candidates about the Crawley office arrangement, salary expectations, notice period, and an example of a production system they owned and how they verified a fix.
last updated 47 week of 2026