About this role
AI Quality Analyst at Finom. Location: Amsterdam or Germany or France or Netherlands or Italy or Spain or Estonia or Greece or Poland or Portugal or United Kingdom or Cyprus or Europe. Role: architecting evaluations, curating benchmarks, tracking quality Requirements: Design and implement automated evaluation frameworks for LLMs and agent systems; build ground-truth benchmarks; define quality KPIs; analyze failures and hallucinations; integrate CI/CD evaluation gates; strong Python and software engineering skills. Category: Data and Analytics Seniority: Senior Level Tools: LangSmith, DeepEval, Ragas, Opik, GPT, Claude, LLaMA, Python, CI/CD, MLOps Commitment: Full Time Workplace: Remote Languages: English