Testing Manager – Analytics & AI Evaluation Center of Excellence Lead

The Human Resources organization combines strategic workforce expertise, advanced analytics, and innovative technology solutions to deliver exceptional employee experiences and business outcomes. The team partners across the firm to leverage data, AI, and emerging technologies to inform decisions, drive operational efficiency, and support the future of work. As a Testing Manager - Vice President at JPMorganChase within our Human Resources organization, you will lead the newly integrated User Acceptance Testing (UAT), Business Validation, and AI Evaluation function. In this strategic leadership role, you will serve as the critical link between advanced AI and analytics development teams and HR business stakeholders, ensuring that enterprise HR solutions are accurate, scalable, compliant, and aligned with business objectives. 

You will oversee the end-to-end validation lifecycle for AI-powered products, predictive analytics, and HR technology solutions while driving the transformation of evaluation processes through automation. This role requires a strong systems-thinking mindset, exceptional stakeholder management capabilities, and the ability to translate complex technical concepts into actionable business insights.

Job Responsibilities

• Lead the end-to-end User Acceptance Testing (UAT), business validation, and evaluation processes for HR analytics products and enterprise AI solutions.
• Define and execute validation strategies that ensure AI models, predictive analytics frameworks, and data-driven solutions meet business, regulatory, and operational requirements.
• Drive the modernization and automation of validation capabilities by implementing scalable testing frameworks, automated regression suites, and AI evaluation methodologies.
• Establish robust approaches for evaluating Large Language Models (LLMs) and AI-driven solutions, including assessments of model quality, business relevance, fairness, bias mitigation, and data integrity.
• Partner closely with HR leaders, product managers, data scientists, and technology teams to ensure seamless alignment between business requirements and technical solutions.
• Translate complex model performance metrics and validation outcomes into clear, risk-based business recommendations for executive stakeholders.
• Build, develop, and lead a high-performing team focused on validation excellence, innovation, automation, and continuous improvement.
• Promote a culture of operational rigor, technical curiosity, accountability, and collaboration across the organization.
• Ensure adherence to data privacy, governance, compliance, and ethical AI standards when evaluating HR-related solutions and sensitive workforce data.
• Identify opportunities to improve efficiency, scalability, and effectiveness of validation processes through innovative technologies and automation.

Required Qualifications, Capabilities, and Skills

• Proven experience leading validation, testing, analytics, risk, operations, or related functions within a complex, matrixed organization.
• Strong understanding of enterprise AI capabilities, Large Language Models (LLMs), machine learning concepts, and emerging AI technologies.
• Experience establishing, leading, or overseeing automated testing frameworks, validation programs, or business evaluation processes.
• Demonstrated knowledge of software development lifecycles, data analytics ecosystems, and enterprise technology delivery methodologies.
• Strong understanding of structured and unstructured data processing, analytics pipelines, and data quality principles.
• Experience evaluating business outcomes, model performance, and operational effectiveness through data-driven methodologies.
• Exceptional stakeholder management and influencing skills, with the ability to engage effectively across technical and non-technical audiences.
• Demonstrated success leading teams, managing talent, and scaling operational capabilities through automation and process transformation.
• Strong verbal and written communication skills with the ability to present complex information to executive stakeholders.
• Experience managing data privacy, compliance, governance, and ethical considerations associated with sensitive workforce or enterprise data.

Preferred Qualifications, Capabilities, and Skills

• Advanced business degree (MBA or equivalent) or graduate-level qualification in analytics, data science, artificial intelligence, or a related discipline.
• Bachelor's degree in Engineering, Computer Science, Information Technology, Analytics, or a related technical field.
• Experience within financial services, banking, or other highly regulated industries.
• Familiarity with AI evaluation frameworks, prompt testing methodologies, synthetic data generation techniques, and model benchmarking approaches.
• Working knowledge of Python, SQL, automation testing tools, or technologies used to validate analytics platforms, data pipelines, and AI models.
• Experience driving enterprise-scale transformation initiatives focused on automation, digital modernization, or AI adoption.