About this role
- Management Consulting
- 786.000 werknemers Agentic AI Quality Engineer Baan
- Amsterdam, NL Geplaatst 3 nov. 2026 Delen: Kopiëren Werkervaring 3 tot 25 jaar Full-time / part-time Full-time Functie Opleidingsniveau HBO of WO Taalvereiste Engels ( Vloeiend ) Deadline 15 september 2027 Bouw aan je carrière op Magnet.me Job Description Lead with expertise, drive innovation. At Accenture the Netherlands, we're seeking an Agentic AI Quality Engineer who brings deep expertise, strategic thinking, and a passion for innovation to help solve complex challenges and deliver exceptional results.
What you'll do
Quality & Performance Evaluation - Automate evaluation of multi-turn conversations and long-horizon tasks using metrics for reasoning accuracy, evidence grounding, plan execution, latency, token usage, and recovery.
- Apply schema validation, tracing, and calibrated LLM-as-a-Judge frameworks to detect inaccurate tool calls, loops, and misuse.
Safety & Compliance Guardrails
- Implement filters and golden datasets to test harmful content, policy violations, hallucinations, bias, and brand drift.
- Use human-in-the-loop review for high-risk edge cases.
Adversarial Security & Red Teaming
- Run red-team, prompt-injection, jailbreak, privacy, and PII-leak tests across agent workflows.
- Analyze failures in model logic and API integrations to identify root causes and controls.
MLOps & Production Monitoring
- Embed test harnesses in CI/CD and monitor production telemetry for regressions, behavioral drift, and recovery.
- Maintain evaluation data lineage, versioning, curation, and golden responses.
Consulting & Collaboration
- Translate business risks into measurable quality criteria, practical controls, and release recommendations.
- Align stakeholders on user journeys, edge cases, thresholds, mitigations, testability, observability, governance, and evaluation strategy.
- Communicate findings and resi...