Staff Site Reliability Engineer strengthening AWS and Kubernetes resilience for Caseware, a fintech company building audit and accounting software. Driving secure delivery, observability, and incident management.
About the role
- Software Engineering & DevOps professional evaluating AI models for multilingual accuracy and quality. Contribute expert judgment to human-in-the-loop AI evaluation workflows used by leading enterprises.
Responsibilities
- Evaluate AI outputs related to software engineering, DevOps, and infrastructure topics
- Perform structured scoring, comparison, classification, and judgment tasks
- Assess technical correctness, completeness, security implications, and best-practice alignment
- Identify hallucinations, incorrect code, unsafe recommendations, or misleading system guidance
- Apply domain-specific engineering and DevOps guidelines consistently across tasks
Requirements
- Software engineers, site reliability engineers, DevOps engineers, or platform engineers
- Experience with production systems, CI/CD pipelines, cloud infrastructure, or distributed systems
- Strong attention to detail and comfort working with structured evaluation criteria
- Native or professional fluency in one or more supported languages is required
- English fluency is required for guidelines, feedback, and collaboration
Benefits
- Contract-based, flexible participation
- Project-based work with clear expectations and timelines
- Opportunities for recurring work based on performance and demand
- Compensation communicated upfront per project or task type
Job title
Job type
Experience level
Salary
Not specified
Degree requirement
Tech skills
Location requirements
Report this job
Found something wrong with the page? Please let us know by submitting a report below.
LP
Senior DevOps Specialist
LinkedIn Recruiter Post
Senior DevOps Specialist contract role in GTA. Hybrid, 3 days onsite. Requires cloud, CI/CD, automation, and troubleshooting skills.
Application Reliability Engineer supporting Innodata’s Google Cloud enterprise applications. Restoring production services, managing deployments, and enhancing microservices for a global AI data engineering company.
Release engineer optimizing build and deployment pipelines for Mozilla’s open - source Firefox browser. Scaling software delivery, improving developer experience, and supporting reliable releases across platforms.
Mozilla release engineer optimizing Firefox build, test, and deployment pipelines at global scale. Improving developer experience, maintaining automation, and responding to critical service outages without an on - call rotation.
LP
DevOps/Cloud Engineer – Senior
LinkedIn Recruiter Post
Senior DevOps/Cloud Engineer contract role in Toronto, ON. Must have strong AWS, Terraform, Ansible, and CI/CD experience.
LP
DevOps Engineer – Azure
LinkedIn Recruiter Post
Hiring DevOps Engineer (Azure) for hybrid role in Toronto, ON. Requires 5+ years experience, Azure DevOps, Docker, Kubernetes, and IaC.
DevOps Developer securing Azure infrastructure for Euna’s cloud - based procurement, budgeting, payments, and grants software. Automating deployments, databases, file transfers, and AI - assisted operations.
DevOps Manager overseeing releases, enterprise tooling, and incident response for Delta Controls, a building - automation solutions manufacturer. Establishing standards across global product teams and offices.
Senior Site Reliability Engineer managing GCP infrastructure for an AI/ML geospatial intelligence company. Improving observability, incident response, reliability metrics, and cloud cost governance.