Data Annotation Specialist, Software Engineering

Posted 2 weeks ago

Apply Now

Resume Score

Check how well your resume matches this job before you apply.

Sign in to check score

About the role

  • Data Annotation Specialist focusing on evaluating coding tasks for AI model development at Cohere. Conducting assessments, labeling outputs, and reporting performance trends.

Responsibilities

  • Evaluate the model's ability to respond to coding requests, workflows, and code base-related questions using available tools.
  • Assess agent trajectories and model capabilities for code generation and debugging requests.
  • Prompt models to complete complex coding tasks and review the accuracy of generated responses.
  • Label, proofread, and improve machine-written and human-written software engineering-related outputs.
  • Report quality and performance trends related to model/agent behaviour and project assignments.

Requirements

  • Subject matter expertise in software engineering - you have both studied in this field and have 3-5 years of relevant industry experience.
  • Proficient knowledge and understanding of Python and the following programming languages: Java, JavaScript, Go, and SQL (any dialect).
  • Prior experience designing, building, and deploying APIs
  • Prior experience working with code agents (OpenCode, Claude Code, Codex, Cursor) or evaluating agent trajectories is a big plus.

Benefits

  • Performance incentives included!

Job type

Contract

Experience level

Mid levelSenior

Salary

CA$40 per hour

Degree requirement

Bachelor's Degree

Tech skills

JavaJavaScriptPythonSQLGo

Location requirements

RemoteCanada

Report this job

Found something wrong with the page? Please let us know by submitting a report below.