Backend Engineer (Python / JavaScript) | $15/hr Remote

Position

SwarmBench Task Engineer SWE / Code

Type

Short-Term Contract (4 weeks)

Compensation

$15 per hour

Location

Remote

Commitment

20-40 hours per week with 4 hours overlap with PST

Role Responsibilities

• Build multi-agent benchmark tasks based on real-world open-source code changes such as bug fixes, migrations, and refactors
• Work with the Harbor evaluation framework to run and validate tasks inside Docker environments
• Write clear and precise task instructions specifying file paths, function signatures, expected behavior, and constraints
• Design and implement Python-based verification scripts to validate correctness of agent-generated code changes
• Create decomposition strategies that split complex code changes across multiple independent sub-agents
• Run, debug, and refine tasks within containerized environments to ensure reproducibility and determinism
• Evaluate task performance signals and improve task quality, clarity, and difficulty
• Contribute to benchmark development for advanced AI coding agents

Requirements

• Strong years of experience in Python and JavaScript development
• Experience with AI coding benchmarks (e.g., SWE-bench, Terminal-Bench)
• Strong experience reading and navigating large open-source codebases (e.g., Django, Flask, FastAPI, Node.js, or similar)
• Familiarity with Git workflows including pull requests, diffs, cherry-picking, and working with specific commits
• Comfortable with Docker including writing Dockerfiles, building images, and debugging container issues
• Experience writing test scripts using pytest, unittest, or custom assertion-based testing
• Ability to write clear, precise, and unambiguous technical specifications
• Ability to work independently in a remote environment

Application Process

• Apply/Easy Apply and check email for application form
• Fill Google form
• Assessment Link (After shortlisting to be completed within 24 hours)