<h2 style="text-align:justify;margin:10.0pt 0.0in 0.0in;line-height:115%;font-size:13.0pt;font-family:Calibri, sans-serif;color:#4f81bd">Responsibilities</h2> • Define and execute comprehensive test strategies covering statistical, ML, LLM and agentic AI models. • Perform functional, regression and scenario‑based testing of model behaviours and workflows. • Conduct AI/ML evaluations including accuracy checks, bias/fairness assessment, robustness analysis and drift detection. • Assess end‑to‑end model workflows including data inputs, feature transformations, task completion, tool‑use accuracy and multi‑step reasoning. • Design and maintain automated test and evaluation pipelines, including benchmarking and regression frameworks. • Validate API and tool‑integration behaviour in production‑like environments, identifying dependency or orchestration issues. • Diagnose issues using observability, logging, tracing and debugging tooling, and document findings clearly. • Collaborate with data scientists across departments to understand modelling intent, feature logic and expected behaviours. • Perform data‑management tasks to support AI/ML model testing, including maintaining metadata, documenting key datasets and ensuring clarity of data inputs. • Contribute to AI/ML proof‑of‑concept (POC) initiatives to strengthen evaluation methodologies and support innovation. • Support data‑management/analytics initiatives such as the Analytics Workbench and contribute to AI/ML/data analytics enablement. <h2 style="text-align:justify;margin:10.0pt 0.0in 0.0in;line-height:115%;font-size:13.0pt;font-family:Calibri, sans-serif;color:#4f81bd">Requirements</h2> • Minimum 4 years of relevant experience in model testing, QA/QC, AI/ML evaluation, CI/CD, MLOps, data engineering, or related technical roles. • Proficiency in Python (especially PySpark, MLlib, pytest), R and SQL; knowledge of Scala, Rust, Java, JS or C++ is a plus. • Experience designing and executing test strategies for ML/AI models, including automated pipelines and regression frameworks. • Ability to evaluate statistical, ML and LLM models using performance, bias, robustness and drift metrics. • Strong ability to assess feature engineering logic, dataset integrity, workflow reliability and tool‑integration behaviours. • Experience troubleshooting using logs, traces and debugging tools to identify root‑cause issues. • Strong documentation and communication skills to articulate findings, risks and remediation requirements. • Ability to collaborate effectively with data science, engineering, IT and governance functions. • Understanding of Responsible AI concepts and quality expectations for production‑ready AI/ML systems.

AVP/VP, AI/ML Model Validation Engineer, Data Management Office (Singapore) @ Sumitomo Mitsui Banking Corporation

About this role

Ready to apply?