About this role
Insurance isn’t the first industry most data scientists think of when they imagine cutting-edge Artificial Intelligence (AI) work, but the incredibly rich data and nature of the business make it a great place to put cutting-edge AI to use. CFC's Data & AI team is building production agentic and ML systems that automate and inform complex underwriting decisions that drive real business outcomes - not demos, not proof-of-concepts sitting on a shelf. The team includes ML engineers and software engineers shipping production services, and this role sits alongside them as an analytical counterpart: running experiments, stress-testing assumptions, and generating the evidence that shapes what gets built and how it improves over time. We are looking for a mid-level Data Scientist to join the team that owns business-critical, live solutions utilising Large Language Models (LLMs), such as an email ingestion/extraction solution and underwriting agents. This is not a pure research or offline-modelling role - when research is carried out and potential opportunities identified it is expected that you will work closely with ML engineers and software engineers to build this into a live system, where quality, reliability, and evaluation rigor directly affects the business. We expect that a successful candidate will be able to own the data science side of a production LLM system end-to-end: partnering with stakeholders to build early prototypes, designing evaluation frameworks, measuring agent quality, and turning ambiguous "is this good?" questions into repeatable, defensible metrics - while working closely with engineers to understand what it takes to take that work from prototype to live system. Explore complex, high dimensional, real-world datasets to uncover insights that meaningfully improve underwriting decisions and system performance at scale. Partner directly with underwriting and business stakeholders to scope problems, assess feasibility, and build early prototypes (e.g. PoC agents, rapid evaluation of an LLM approach) before committing engineering investment. Stay involved from prototype through to production, working with ML/software engineers to harden, scale, and maintain what you've built as a key contributor to the codebase. Design and run evaluation frameworks for LLM-powered agent behaviour, including offline (golden datasets, regression suites) and online (production monitoring, A/B testing) evaluation. Build and maintain analytical pipelines — prompt design, calibration against human labels, bias/consistency checks, LLM-as-a-judge, and ongoing validation that the judge stays trustworthy as the underlying models change. Partner with ML engineers to design system nodes/components, translating data science findings into concrete engineering requirements. Define quality metrics for agent outputs (accuracy, hallucination rate, task completion, groundedness, latency/cost trade-offs) and track them over time. Work with software engineers on productionising evaluation and monitoring code: CI/CD integration, release gating, and operational readiness (alerting, dashboards, on-call awareness). Actively explore cutting-edge developments in AI and machine learning — with the space and support to experiment, prototype, and bring new techniques into production where they add value. Investigate how agentic systems behave in production — identifying edge cases, failure modes, and opportunities to make systems more robust and reliable. Prototype and iterate on features for AI/ML pipelines, taking ideas from early exploration through to measurable impact in production services. Document experiments, findings, and methodologies clearly so that insights are reproducible and decisions are traceable.