Golang backend developer

About

We are looking for an experienced Golang backend developer to design and build a scalable, production‑grade backend system that:

• Manages render jobs end‑to‑end
• Integrates with rendering infrastructure (Deadline + VMs)
• Handles large file workflows (200 GB uploads)
• Provides MCP‑ready APIs (AI‑friendly, structured, deterministic)

Important:
You are NOT building chatbot logic or MCP server.
You ARE building everything MCP will call.

Core Responsibilities

1. Authentication & Access Layer

   • JWT‑based user authentication
   • API key system for MCP server access
   • Middleware for secure request validation

2. User & Wallet System

   • User profiles
   • Wallet balance management
   • Credit deduction per render job

3. Large File Upload System (200 GB+)

   • Chunked upload system
   • Upload session tracking
   • Pre‑signed URL handling

4. Render Job Engine (Core System)

   • Design and implement the central job system:
     • Create, update, and cancel jobs
     • Track job lifecycle
     • Maintain job logs, steps, and statuses
   • Key APIs:
     • POST /jobs
     • GET /jobs/id
     • GET /jobs
     • POST /jobs/id/test-render
     • POST /jobs/id/cancel

5. Scene Processing Orchestration

   • Trigger Python workers (scene checker, repathing, settings)
   • Track step‑level progress
   • Manage queue via Redis

6. Test Render System (10‑minute SLA)

   • Priority job handling
   • Limited frame rendering
   • Trigger VM → Submit to Deadline → Track status

7. Deadline Integration

   • Submit jobs to Deadline
   • Monitor job progress
   • Retry failed jobs

8. SkyPilot Auto Scaling

   • Spin up/down VMs dynamically
   • Queue‑based scaling logic
   • Optimize cost vs performance

9. Cost Engine (Critical)

   • Pre‑render cost estimation
   • Live cost tracking
   • APIs:
     • POST /cost/estimate
     • GET /jobs/id/cost

10. Payment Integration (Razorpay)

    • Order creation
    • Payment verification
    • Transaction logging

11. Output Delivery

    • Secure storage (up to 100 GB output)
    • Signed download URLs
    • Expiry‑based access

12. Notifications System

    • Event‑driven notifications:
      • Test render ready
      • Render completed
      • Payment status

13. Admin APIs

    • Monitor jobs
    • Approve users
    • Kill jobs
    • Track VM usage

MCP (AI Integration) Requirements

• APIs must be AI‑agent friendly: deterministic outputs, no vague responses. Always return:

  • Job status
  • IDs
  • ETA
  • Cost info

• MCP‑Critical Endpoints

  • POST /jobs
  • GET /jobs/id
  • GET /jobs
  • POST /jobs/id/test-render
  • POST /cost/estimate

• MCP Auth Support

  • API key system for the MCP server
  • User mapping via API keys

• Action Logging

  • Log every API‑triggered action
  • Ensure full traceability

Safety Layer

• Rate limiting
• Job limits per user
• Cost protection checks

Tech Stack

• Golang (Gin / Fibre preferred)
• PostgreSQL
• Redis (queue + caching)
• Object storage (S3‑compatible)
• Deadline (render manager)
• SkyPilot (infra orchestration)
• Razorpay (payments)

Ideal Candidate

• Strong experience in Golang backend systems
• Experience with distributed systems / job queues
• Familiar with cloud infra (AWS/GCP, VM orchestration)
• Experience with large file handling / storage systems
• Bonus: Experience with render pipelines / VFX / 3D workflows

Deliverables (V1)

By the end of this project, the backend should:

• Create and manage render jobs
• Run test render within 10 minutes
• Auto‑scale infrastructure
• Track cost in real‑time
• Support 200 GB uploads
• Deliver outputs securely
• Provide clean MCP‑ready APIs