Professional - services practitioners auditing AI work products for Mercor’s frontier - model training. Evaluating task realism, domain accuracy, grader judgments, and practical usability.
About the role
- Professional services practitioners auditing AI-generated work products for Mercor, which trains frontier AI models. Reviewing realism, domain accuracy, grading, and practitioner usability.
Responsibilities
- Audit AI work products in the practitioner's own professional field
- Review the AI agent's task, actions, produced files, and grader scoresheet
- Assess whether tasks are realistic and applicable to the relevant profession
- Verify domain content, including standards, figures, regulations, and constraints
- Review eleven yes/no grading questions and explain disagreements
- Independently band the work and assess whether a practitioner could use or accept it
- Compare the independent assessment with the reported score
- Complete approximately one item per hour without performing or redoing the underlying task
- Write a short reason for every judgment
- Work on projects training and enhancing AI systems
Requirements
- Professional experience in one of the listed fields: Project Management Specialists, Lawyers, Accountants and Auditors, Software Developers, or Computer and Information Systems Managers
- Project Management Specialists: 5+ years running real project plans, schedules and budgets; PMP a plus
- Lawyers: US-qualified and in active practice, with 4+ years post-admission and experience drafting own documents
- Accountants and Auditors: 4+ years owning a close, an audit file, or a set of financial statements; CPA or CA strongly preferred
- Software Developers: 3+ years shipping production code maintained by other engineers
- Computer and Information Systems Managers: 4+ years owning an IT function, its budget, and its vendors
- Practical experience producing or reviewing professional work products
- Near-native business English
- Comfort with spreadsheets or PDFs sufficient to check whether numbers reconcile
- Must be able to work as an independent contractor
- H1-B and STEM OPT candidates are not supported
Benefits
- Payments are weekly on Stripe or Wise based on services rendered
- Fully remote role that can be completed on your own schedule
- Competitive pay
- Opportunity to collaborate with leading researchers
- Opportunity to help shape the next generation of AI systems
- Up to $350 for each successful referral
Job title
Job type
Experience level
Salary
Degree requirement
No Education Requirement
Tech skills
Location requirements
Report this job
Found something wrong with the page? Please let us know by submitting a report below.
Professional services practitioners auditing AI - generated work products for Mercor, which trains frontier AI models with human expertise. Reviewing realism, accuracy, grading, and practitioner usability.
Disbursements Reviewer verifying settlement statements, lien searches, and reconciliations for RB Global’s global equipment marketplace. Supporting accounting teams on an 18 - month contract.
Quality Reviewer auditing personalization features in maps/search experience and ensuring compliance with formatting and guideline requirements. Reviewing rater work for logical alignment and accuracy in outputs.
Disbursement Reviewer reviewing and approving settlement statements at Ritchie Bros. Collaborating with finance staff and supporting accounting departments worldwide.