Experienced Software Engineers: AI Coding Task Creation And Validation
No longer listedWhat We're Researching
We're hiring experienced software engineers to help us create and validate advanced coding tasks for AI systems. This ongoing work feeds directly into the development of robust evaluation environments for AI agents. Building these benchmarks ensures that emerging tools can handle complex software engineering challenges.
How It Works
You will design realistic software engineering problems that an AI must solve within a terminal environment. This involves setting up the development workspace, working with Docker, and building complete reference solutions. You will also write clean automated tests to verify that each task can be graded reliably. All work is conducted remotely using standard command-line tools and GitHub.
Who This Is For
We welcome backend developers, full-stack engineers, and systems programmers with strong architectural fundamentals. This opportunity is best suited for professionals who are highly comfortable navigating unfamiliar codebases independently. You must have hands-on experience with containerization, automated testing frameworks, and advanced version control workflows.
What You'll Do
Design self-contained software engineering problems for AI evaluation.
Configure development environments and containerize them using Docker.
Write deterministic automated tests to ensure reliable grading.
Develop complete reference solutions for the challenges you create.
Who Should Apply
Experienced software engineer comfortable working in unfamiliar codebases.
Strong proficiency with GitHub, command-line tools, and Docker.
Demonstrated ability to write comprehensive automated test suites.
Willingness to complete a one-time software engineering skills assessment.
Compensation
$82 per task
Ready to participate?
About Terac
Terac is building the world's largest pool of vetted human experts for AI. Researchers, AI labs, and product teams use Terac to recruit, screen, and pay study participants across industries, languages, and skill sets.
Learn more at terac.com or on YouTube at @jointerac.