Medical Expert AI Evaluation Specialist
This role is for one of our clients
Compensation: $130-$180 per hour
Location requirements
Canada, United States
Join a leading AI research initiative focused on advancing healthcare-focused artificial intelligence. We are seeking experienced medical professionals to contribute their clinical expertise toward training and evaluating next-generation AI models capable of sophisticated medical reasoning.
In this role, you will help create realistic clinical scenarios, evaluate AI-generated responses using evidence-based medical standards, and provide expert insights that improve how AI systems reason about diagnosis, treatment, and patient care.
This opportunity is ideal for physicians across a broad range of medical specialties who are passionate about applying their expertise to the future of healthcare technology.
Requirements
Key Responsibilities
Develop Clinical Scenarios
- Create realistic, specialty-specific clinical cases that reflect real-world medical practice.
- Design prompts covering areas such as:
- Diagnostic reasoning
- Treatment planning
- Risk stratification
- Differential diagnosis
- Guideline-based clinical decision-making
Create Reference-Quality Solutions
- Produce high-quality clinical responses that reflect attending physician-level reasoning.
- Ensure recommendations align with current evidence-based practices and accepted clinical guidelines.
Evaluate AI Performance
- Assess AI-generated medical responses using structured evaluation frameworks and scoring rubrics.
- Identify strengths, weaknesses, omissions, and reasoning errors.
- Provide detailed written feedback to improve model performance and clinical accuracy.
Collaborate with Clinical Experts
- Participate in onboarding sessions and specialty calibration meetings.
- Work with multidisciplinary research teams to maintain consistency and quality across medical evaluations.
Required Qualifications
Applicants should meet one of the following criteria:
Attending Physicians
- Board-certified physician with a current, active, and unrestricted medical license.
Resident Physicians
- Currently in the final year of an accredited residency program.
- Recent residency graduates must be board-eligible.
Fellows
- Board-certified or board-eligible in their primary specialty.
- Current, active, and unrestricted medical license.
Physicians from a wide range of specialties are encouraged to apply, including but not limited to:
- Family Medicine
- Pediatrics
- Anesthesiology
- General Surgery
- ENT (Otolaryngology)
- Physical Medicine & Rehabilitation (PM&R)
- Occupational Medicine
- Other medical and surgical specialties
Preferred Qualifications
- Strong understanding of evidence-based medicine and clinical practice guidelines.
- Excellent written communication skills.
- Experience with medical education, clinical research, peer review, or healthcare quality initiatives.
- Interest in artificial intelligence and healthcare innovation.
Engagement Details
- Fully remote with 100% asynchronous work.
- Typical commitment of approximately 20 hours per week, with opportunities for additional hours based on project needs.
- Flexible schedule.
- Weekly payments.
- Rolling onboarding with ongoing project opportunities.
- Engagement duration is reviewed periodically based on project requirements and performance.
Why Join
- Help shape the future of AI-powered healthcare.
- Apply your clinical expertise to improve advanced medical reasoning systems.
- Collaborate with leading AI researchers and healthcare professionals.
- Contribute to high-impact projects that advance patient-centered artificial intelligence.
Equal Opportunity Statement
All qualified applicants will be considered without regard to legally protected characteristics. Reasonable accommodations are available upon request.
Contract Information
- Independent contractor engagement.
- Fully remote work completed on your own schedule.
- Project duration may be extended, modified, or concluded based on business needs and individual performance.
- Work does not involve access to confidential or proprietary information from any employer, client, or institution.
- Weekly payments are processed through supported payment platforms based on approved work completed.
- Please note that visa sponsorship is not available for this opportunity.