Software Engineering & DevOps professional evaluating AI models for multilingual accuracy and quality. Contribute expert judgment to human-in-the-loop AI evaluation workflows used by leading enterprises.
Responsibilities
Evaluate AI outputs related to software engineering, DevOps, and infrastructure topics
Perform structured scoring, comparison, classification, and judgment tasks
Assess technical correctness, completeness, security implications, and best-practice alignment
Identify hallucinations, incorrect code, unsafe recommendations, or misleading system guidance
Apply domain-specific engineering and DevOps guidelines consistently across tasks
Requirements
Software engineers, site reliability engineers, DevOps engineers, or platform engineers
Experience with production systems, CI/CD pipelines, cloud infrastructure, or distributed systems
Strong attention to detail and comfort working with structured evaluation criteria
Native or professional fluency in one or more supported languages is required
English fluency is required for guidelines, feedback, and collaboration
Benefits
Contract-based, flexible participation
Project-based work with clear expectations and timelines
Opportunities for recurring work based on performance and demand
Compensation communicated upfront per project or task type
DevOps Intern supporting CI/CD, cloud infrastructure, and automation for Ludia’s mobile game studio. Improving reliability and developer tools in production game environments.
Manager leading global SRE teams for Akamai's distributed Cloud IAM services. Improving reliability, scalability, security, and usability through cloud - native tooling and software.
Cloud Engineer supporting Kinaxis’s AI - powered supply chain orchestration platform reliability. Automating cloud infrastructure, deployments, and production operations across Canadian locations.
Senior DevOps Engineer owning CI/CD, Kubernetes, and cloud infrastructure for Bounteous, a global AI services firm. Automating secure, reliable platforms across the DevOps lifecycle.
DevOps Engineer owning CI/CD and app releases for a gamified sports training platform. Maintaining React Native, Expo/EAS, Supabase, Next.js, and React delivery workflows.
Site Reliability Engineer managing AWS and Kubernetes reliability for Rentsync’s rental - property software products. Leading incident response, observability, automation, and infrastructure hardening.
Senior DevOps Engineer securing Boeing Canada’s Azure, Kubernetes, and on - premise platforms. Leading CI/CD, infrastructure automation, reliability, compliance, and technical mentorship.
Site Reliability Engineer automating enterprise release orchestration and delivery operations for Sun Life. Supporting platform reliability, Kubernetes automation, and transition to a future release management solution.