Job Description
Job Title: DevOps Engineer (Infrastructure-as-Code Task Author)
Location: Alexandria, IN
Duration: 1-month paid trial, then extension to 6+ months
Job Description
Hiring experienced DevOps engineers to author and validate Infrastructure-as-Code (IaC) tasks. Design realistic infrastructure scenarios, build ground-truth solutions, and define automated checks for AI agent evaluation. Requires hands-on Terraform, Pulumi, CloudFormation, or cloud CLI (AWS/GCP) skills.
Responsibilities
- Author IaC tasks grounded in real-world AWS and GCP scenarios
- Build correct, idempotent IaC solutions for each task
- Design verifiable automated graders that confirm AI agent reaches correct end state
- Review and QA tasks authored by other engineers (correctness, difficulty, robustness)
- Harden tasks against reward hacking
- Document task intent, assumptions, edge cases, and scoring rationale
Must-Have Qualifications
- 4+ years in DevOps / cloud engineering
- Deep, hands-on Infrastructure-as-Code expertise: Terraform (required); Pulumi, CloudFormation, or CDK a plus
- Strong AWS and GCP depth
- Comfortable scripting in Python and/or Bash for automation and validation
- Clear sense of correctness and verification (articulate in code what makes an infrastructure outcome right and safe)
Preferred Qualifications
- Prior work in AI evaluation, benchmarking, or expert data annotation
- Prior experience with LocalStack
