Professional services practitioners auditing AI - generated work products for Mercor, which trains frontier AI models. Reviewing realism, domain accuracy, grading, and practitioner usability.
About the role
- Professional services practitioners auditing AI-generated work products for Mercor, which trains frontier AI models with human expertise. Reviewing realism, accuracy, grading, and practitioner usability.
Responsibilities
- Audit AI agents’ work products in the reviewer’s professional field
- Evaluate whether tasks are realistic and relevant to the job
- Check domain content, standards, figures, regulations and constraints for accuracy and consistency
- Review graders’ yes/no assessments of finished work and explain disagreements
- Independently band the work and assess whether a practitioner could use or accept it
- Compare the reviewer’s band with the reported score
- Review approximately one item per hour without performing or redoing the underlying task
- Provide written reasoning for each judgment
Requirements
- 5+ years running real project plans, schedules and budgets (Project Management Specialists)
- PMP a plus
- US-qualified and actively practicing lawyer with 4+ years post-admission, drafting own documents
- 4+ years owning a close, audit file or set of financial statements (Accountants and Auditors)
- CPA or CA strongly preferred
- 3+ years shipping production code maintained by other engineers (Software Developers)
- 4+ years owning an IT function, its budget and vendors (Computer and Information Systems Managers)
- Professional experience producing and reviewing relevant work artifacts
- Near-native business English
- Comfort using spreadsheets or PDFs to verify numerical reconciliation
- Independent contractor engagement
- Must not require H-1B or STEM OPT support
Benefits
- Fully remote role
- Flexible own schedule
- Weekly payments via Stripe or Wise
- Opportunity to collaborate with leading researchers
- Work on projects training and enhancing AI systems
- Referral earnings of up to $350 per successful referral
Job title
Job type
Experience level
Salary
Degree requirement
No Education Requirement
Tech skills
Location requirements
Report this job
Found something wrong with the page? Please let us know by submitting a report below.
Professional - services practitioners auditing AI work products for Mercor’s frontier - model training. Evaluating task realism, domain accuracy, grader judgments, and practical usability.
Disbursements Reviewer verifying settlement statements, lien searches, and reconciliations for RB Global’s global equipment marketplace. Supporting accounting teams on an 18 - month contract.
Quality Reviewer auditing personalization features in maps/search experience and ensuring compliance with formatting and guideline requirements. Reviewing rater work for logical alignment and accuracy in outputs.
Disbursement Reviewer reviewing and approving settlement statements at Ritchie Bros. Collaborating with finance staff and supporting accounting departments worldwide.