AI Evaluation Project Engineer

About

Join Mindrift as an AI Evaluation Project Engineer and contribute to innovative projects focused on improving AI coding capabilities. This part-time role requires deep software engineering knowledge, especially in Python and testing.

As part of the evaluation process, you will develop tasks that accurately measure AI coding agents' abilities. Candidates should have at least 5 years in software development, including skills in Docker, React for building interfaces, and a strong grasp of test writing techniques. You'll work collaboratively with AI to refine task quality and evaluate code outputs.

Key Responsibilities

• Create realistic virtual company scenarios
• Develop and calibrate evaluation tasks for AI agents
• Design tasks in simulated developer environments
• Write comprehensive tests for agent-generated outputs
• Collaborate with AI for iterative task assessments

Requirements

• Bachelor's degree in Computer Science or similar
• Over 5 years of software engineering expertise
• Proficient in Python and creating functional tests
• Familiar with Docker, Postgres, and infrastructure tools
• B2 proficiency in English required

Shape the future of AI coding assessment with your engineering experience at Mindrift.