About this role
Tieto Tech Consulting (MentorMate from 2001 to 2026) provides design-led, data-centric, and AI-powered digital engineering & consulting services to enterprises worldwide. With a strong heritage of Nordic quality, local presence, and global-scale operations, we are backed by 8,500+ IT professionals. Our clients trust us for advanced data and analytics skills, top industry-specific expertise, specialized software R&D, and the capabilities of a multinational team. We are part of Tieto, a leading technology company with an annual revenue of approximately EUR 2bn.
Tieto Tech Consulting’s team in Bulgaria consists of 500+ professionals specializing in the development of tailored digital solutions in diverse industries.
We are looking for a Quality Engineer with experience testing AI-powered products and building automated testing solutions. In this role, you will help ensure the quality, reliability, and performance of modern AI-first applications, including products powered by Large Language Models (LLMs).
You will work closely with engineering, product, and AI teams to define testing strategies, validate complex user scenarios, and build automation frameworks for rapidly evolving products.
This role is based on a hybrid work model, with 3 days per week in our Sofia office.
Key Responsibilities
Validate LLM-based applications and AI-generated outputsEvaluate quality, reliability, and behavior of probabilistic AI systemsCreate test cases from evolving requirementsIdentify gaps, inconsistencies, and risks in product requirementsCollaborate with product managers, engineers, and AI specialists throughout the development lifecycleAnalyze defects, perform root cause analysis, and drive issue resolutionRequired Qualifications
Experience testing AI-powered applications, GenAI solutions, chatbots, copilots, or similar productsUnderstanding of AI/ML concepts and Large Language Models (LLMs)Strong analytical and problem-solving skillsExperience creating test plans, test cases, and quality metricsExcellent spoken and written EnglishA significant advantage would be
Experience with AI testing tools and frameworksExperience evaluating LLM responses for accuracy, relevancy, safety, and consistencyFamiliarity with prompt engineering and AI quality assessment techniquesKnowledge of Python or another programming language used in automationExperience working in Agile development environmentsExposure to cloud platforms such as Azure, AWS, or GCP At Tieto, we believe in the power of diversity, equity, and inclusion. We encourage applicants of all backgrounds, genders (m/f/d), and walks of life to join our team, as we believe that this fosters an inspiring workplace and fuels innovation. Our commitment to openness, trust, and diversity is at the heart of our mission to create digital futures that benefit businesses, societies, and humanity.
Diversity, equity and inclusion | Sustainability | Tieto
Stay alert to recruitment scams:
• Tieto only communicates via @tieto.com email addresses.
• We never ask candidates to pay fees or share financial details during recruitment.
• If you receive a suspicious message claiming to be from Tieto, don't respond — verify it via our official careers site.
• For more information visit Important Advisory Regarding Recruitment Fraud | Tieto Blog