Professional services practitioners auditing AI - generated work products for Mercor, which trains frontier AI models. Reviewing realism, domain accuracy, grading, and practitioner usability.
About the role
- Professional-services practitioners auditing AI work products for Mercor’s frontier-model training. Evaluating task realism, domain accuracy, grader judgments, and practical usability.
Responsibilities
- Audit AI agents’ work product in the practitioner’s own professional-services field
- Review the task as written, the AI agent’s work, produced files, and AI grader’s scoresheet
- Assess whether tasks are real, job-relevant, realistic, borderline, or contrived
- Verify domain content, including standards, figures, regulations, and constraints
- Review eleven grader yes/no judgments and explain disagreements
- Independently band the work and assess whether the score is defensible
- Determine whether a practitioner could use or accept the work
- Complete approximately one hour per item
- Review work only; do not perform the underlying task or redo the agent’s work
- Work on projects training and enhancing AI systems for Mercor’s AI lab and enterprise partners
Requirements
- Working professional practitioner in one of: Project Management Specialist, Lawyer, Accountant or Auditor, Software Developer, or Computer and Information Systems Manager
- Project Management Specialists: 5+ years running real project plans, schedules, and budgets that other people worked to
- Lawyers: US-qualified and in active practice, with 4+ years post-admission, drafting own documents rather than only reviewing
- Accountants and Auditors: 4+ years owning a close, an audit file, or a set of financial statements
- Software Developers: 3+ years shipping production code that other engineers maintained afterward
- Computer and Information Systems Managers: 4+ years owning an IT function, its budget, and its vendors
- Near-native business English
- Comfort with spreadsheets or PDFs sufficient to check whether numbers reconcile
- Field expertise demonstrated through reviewing junior work, signing an audit opinion, conducting consequential code reviews, or participating in project post-mortems
- Delivery history; certification-only candidates without delivery history are a poor fit
- Independent contractor engagement
- Applicants must be eligible to work without H1-B or STEM OPT support
Benefits
- Fully remote work
- Flexible schedule; work can be completed on your own schedule
- Weekly payments via Stripe or Wise based on services rendered
- Projects may be extended, shortened, or concluded early depending on needs and performance
- Competitive pay
- Collaboration with leading researchers
- Opportunity to help shape the next generation of AI systems
- Up to $350 referral payment for each successful referral, with no referral limit (restrictions may apply)
- Reasonable accommodations upon request
Job title
Job type
Experience level
Salary
Degree requirement
No Education Requirement
Location requirements
Report this job
Found something wrong with the page? Please let us know by submitting a report below.
Professional services practitioners auditing AI - generated work products for Mercor, which trains frontier AI models with human expertise. Reviewing realism, accuracy, grading, and practitioner usability.
Disbursements Reviewer verifying settlement statements, lien searches, and reconciliations for RB Global’s global equipment marketplace. Supporting accounting teams on an 18 - month contract.
Quality Reviewer auditing personalization features in maps/search experience and ensuring compliance with formatting and guideline requirements. Reviewing rater work for logical alignment and accuracy in outputs.
Disbursement Reviewer reviewing and approving settlement statements at Ritchie Bros. Collaborating with finance staff and supporting accounting departments worldwide.