Software Engineer for evaluation datasets

Make a difference in AI development as a Software Engineer for evaluation datasets. Contribute to high-impact projects by assessing AI's reasoning in coding interactions.

In this remote and flexible part-time role, you will develop realistic prompts that reflect coding challenges for AI evaluations. Your responsibilities will include sourcing content from actual PRs and crafting clear, technically accurate responses that highlight reasoning quality. Ideal candidates will combine technical proficiency with a flair for technical writing.

Key Responsibilities

• Create prompts simulating real developer coding scenarios
• Curate and adapt content from existing PRs for datasets
• Compose accurate model responses showcasing strong reasoning

Requirements

• 2+ years experience in software engineering or related fields
• Degree in Software Engineering, Computer Science, or equivalent
• Proficiency in Python, Java, JavaScript, or C++
• Experience with debugging coding issues
• Strong technical writing capability with attention to detail

Apply your expertise to shape the future of intelligent systems with a flexible remote work arrangement.