About this role
Functies Vakgebieden Stand 11 november 2026 Agentic AI Quality Engineer - Amsterdam - Geplaatst 15 september - Hybride - AI Engineering Salaris vergeleken € 5.000 € 6.750 € 5.000 € 5.750 € 6.750 Middelste helft AI Engineer € 5.300 - € 6.500 Mediaan € 5.650 Role At Accenture the Netherlands, we're seeking an Agentic AI Quality Engineer who brings deep expertise, strategic thinking, and a passion for innovation to help us solve complex challenges. What you do Quality & Performance Evaluation - Automate evaluation of multi-turn conversations and long-horizon tasks using metrics for reasoning accuracy, evidence grounding, plan execution, latency, token usage, and recovery.
- Apply schema validation, tracing, and calibrated LLM-as-a-Judge frameworks to detect inaccurate tool calls, loops, and misuse.
Safety & Compliance Guardrails
- Implement filters and golden datasets to test harmful content, policy violations, hallucinations, bias, and brand drift.
- Use human-in-the-loop review for high-risk edge cases.
Adversarial Security & Red Teaming
- Run red-team, prompt-injection, jailbreak, privacy, and PII-leak tests across agent workflows.
- Analyze failures in model logic and API integrations to identify root causes and controls.
MLOps & Production Monitoring
- Embed test harnesses in CI/CD and monitor production telemetry for regressions, behavioral drift, and recovery.
- Maintain evaluation data lineage, versioning, curation, and golden responses.
Consulting & Collaboration
- Translate business risks into measurable quality criteria, practical controls, and release recommendations.
- Align stakeholders on user journeys, edge cases, thresholds, mitigations, testability, observability, governance, and evaluation strategy.
- Communicate findings and residual risks for go/no-go decisions, coach teams, and share reusable assets and standards.
What Success Looks Like (KPIs)
- Safety: Zero critical production incidents.
- Aut...