Now hiring

Agentic AI Quality Engineer @ Employer

Amsterdam, NL32B, NLOnsiteContract
Apply with ResuMinder

Opens on the employer's site

About this role

Functies Vakgebieden Stand 11 november 2026 Agentic AI Quality Engineer - Amsterdam - Geplaatst 15 september - Hybride - AI Engineering Salaris vergeleken € 5.000 € 6.750 € 5.000 € 5.750 € 6.750 Middelste helft AI Engineer € 5.300 - € 6.500 Mediaan € 5.650 Role At Accenture the Netherlands, we're seeking an Agentic AI Quality Engineer who brings deep expertise, strategic thinking, and a passion for innovation to help us solve complex challenges. What you do Quality & Performance Evaluation - Automate evaluation of multi-turn conversations and long-horizon tasks using metrics for reasoning accuracy, evidence grounding, plan execution, latency, token usage, and recovery.

  • Apply schema validation, tracing, and calibrated LLM-as-a-Judge frameworks to detect inaccurate tool calls, loops, and misuse.

Safety & Compliance Guardrails

  • Implement filters and golden datasets to test harmful content, policy violations, hallucinations, bias, and brand drift.
  • Use human-in-the-loop review for high-risk edge cases.

Adversarial Security & Red Teaming

  • Run red-team, prompt-injection, jailbreak, privacy, and PII-leak tests across agent workflows.
  • Analyze failures in model logic and API integrations to identify root causes and controls.

MLOps & Production Monitoring

  • Embed test harnesses in CI/CD and monitor production telemetry for regressions, behavioral drift, and recovery.
  • Maintain evaluation data lineage, versioning, curation, and golden responses.

Consulting & Collaboration

  • Translate business risks into measurable quality criteria, practical controls, and release recommendations.
  • Align stakeholders on user journeys, edge cases, thresholds, mitigations, testability, observability, governance, and evaluation strategy.
  • Communicate findings and residual risks for go/no-go decisions, coach teams, and share reusable assets and standards.

What Success Looks Like (KPIs)

  • Safety: Zero critical production incidents.
  • Aut...

Skills

schema validationtracingLLM-as-a-judgegolden datasetshuman-in-the-loop reviewred teamingMLOps

Ready to apply?

Install the ResuMinder extension and we'll auto-fill the application in seconds — no rewriting.

See how your CV scores