This position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for a Testing Lead-QA based in India.
This is an exciting opportunity for an experienced quality engineering professional to lead testing initiatives for cutting-edge AI products, including Deep Learning, Large Language Models (LLMs), and Vision-Language Models (VLMs). The role combines technical leadership, test strategy development, and documentation ownership in a fast-paced and innovation-driven environment. You will work closely with machine learning engineers, product teams, and software developers to ensure AI systems are reliable, scalable, and production-ready. This position is ideal for someone who enjoys solving complex quality challenges in non-deterministic systems and building structured processes around emerging technologies. You will play a key role in shaping quality standards, improving AI evaluation methodologies, and fostering a quality-first culture across the organization.
This position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for a Testing Lead-QA based in India.
This is an exciting opportunity for an experienced quality engineering professional to lead testing initiatives for cutting-edge AI products, including Deep Learning, Large Language Models (LLMs), and Vision-Language Models (VLMs). The role combines technical leadership, test strategy development, and documentation ownership in a fast-paced and innovation-driven environment. You will work closely with machine learning engineers, product teams, and software developers to ensure AI systems are reliable, scalable, and production-ready. This position is ideal for someone who enjoys solving complex quality challenges in non-deterministic systems and building structured processes around emerging technologies. You will play a key role in shaping quality standards, improving AI evaluation methodologies, and fostering a quality-first culture across the organization.
Accountabilities:
- Define and lead end-to-end testing strategies for Deep Learning, LLM, and VLM products and pipelines.
- Establish testing frameworks covering model evaluation, acceptance criteria, release readiness, and risk assessment.
- Create and maintain comprehensive documentation related to testing methodologies, model assumptions, known limitations, and quality sign-offs.
- Design and execute testing strategies for prompt engineering, RAG pipelines, hallucination control, multi-turn conversations, and long-context model behavior.
- Develop and manage golden datasets, regression testing suites, and benchmarking processes.
- Evaluate multimodal and vision-language systems, including image-text alignment, OCR, captioning, and reasoning capabilities.
- Build Python-based automation frameworks for model evaluation, validation, and regression testing.
- Integrate testing processes into CI/CD and MLOps pipelines to support continuous delivery and model monitoring.
- Generate quality reports, dashboards, and actionable insights for engineering and leadership teams.
- Monitor production performance, identify model drift or degradation, and document behavioral changes across model versions.
- Establish scalable QA standards and mentor teams on testing best practices, documentation, and quality processes.
- Serve as the primary reference point for AI quality standards, testing governance, and risk management.
- 3–4 years of experience in software testing, with significant ownership or leadership experience in AI, Deep Learning, LLM, or Generative AI testing environments.
- Strong hands-on experience testing non-deterministic AI systems, machine learning models, and advanced language models.
- Excellent Python programming skills with experience in test automation, data validation, and quality engineering frameworks.
- Strong understanding of transformer architectures, deep learning workflows, and model evaluation methodologies.
- Proven ability to create clear, structured, and maintainable technical documentation.
- Experience developing testing strategies for prompt engineering, conversational AI systems, and retrieval-augmented generation (RAG) architectures.
- Familiarity with CI/CD pipelines, MLOps practices, and production monitoring for AI systems.
- Strong analytical and problem-solving skills with the ability to work effectively in fast-paced and ambiguous startup environments.
- Excellent communication and stakeholder management skills, with the ability to collaborate across technical and non-technical teams.
- Experience with Vision-Language Models, multimodal AI systems, or computer vision technologies is highly desirable.
- Familiarity with tools such as LangChain, LlamaIndex, MLflow, vector databases, and embedding technologies is considered a strong advantage.
- Exposure to AI governance, compliance requirements, and documentation of model risks and limitations is a plus.
- Flexible working options, including remote, hybrid, or on-site arrangements depending on business needs.
- Opportunity to work on cutting-edge AI technologies, including LLMs, Deep Learning, and multimodal systems.
- High-impact role with significant ownership and visibility across product and engineering teams.
- Collaborative startup environment that encourages innovation, experimentation, and rapid decision-making.
- Exposure to advanced AI architectures, MLOps practices, and next-generation quality engineering methodologies.
- Strong opportunities for professional growth, leadership development, and technical learning.
- Dynamic and cross-functional work environment with direct collaboration with machine learning, engineering, and product experts.
- Opportunity to shape quality standards and processes in an evolving and rapidly growing AI ecosystem.