Professional evaluator reviewing AI-generated work products across project management, law, accounting, software, or IT. Assessing realism, accuracy, grading, and practitioner-level quality.
Responsibilities
Review complete AI-generated work products representing realistic tasks from the consultant’s professional field
Examine task instructions, completed deliverables, supporting files, and evaluation results
Assess whether work reflects realistic professional practice
Identify substantive issues apparent to an experienced practitioner
Evaluate whether outputs are usable or acceptable in a real professional setting
Assess task descriptions, constraints, expected deliverables, and alignment with normal industry practice
Provide concise explanations supporting judgements
Review technical, legal, financial, operational, or professional content for accuracy
Identify inaccurate standards, figures, regulations, assumptions, terminology, or constraints
Detect subtle domain-specific problems
Review consistency between instructions, supporting information, and completed outputs
Apply current professional judgement when assessing quality and correctness
Review existing yes/no grading decisions against completed work
Identify unsupported or inconsistent grading decisions and explain disagreements
Apply consistent standards across reviewed work products
Independently evaluate overall work-product quality and assign an appropriate quality band
Compare assessments with reported evaluation scores and identify inaccurate scores
Provide short, clear rationales for review decisions in precise business English
Distinguish material problems from minor presentation issues
Support judgements with relevant professional standards or practical experience
Complete approximately one hour of review work per item
Requirements
Substantial hands-on experience in one of the following areas: project management, law, accounting and audit, software development, or IT and information systems management
Project Management: 5+ years of experience running real project plans, schedules, and budgets
Project Management: Experience managing deliverables that teams actively worked against
Project Management: Strong understanding of professional project-management documentation and controls
PMP certification is advantageous but not required
Law: US-qualified lawyer in active practice
Law: 4+ years of post-admission experience
Law: Hands-on experience drafting legal documents
Law: Ability to evaluate legal accuracy, terminology, and professional work product
Accounting & Audit: 4+ years of professional experience
Accounting & Audit: Hands-on responsibility for financial close processes, audit files, or financial statements
Accounting & Audit: Strong knowledge of accounting and audit documentation
CPA or CA qualification is strongly preferred
Software Development: 3+ years of professional software development experience
Software Development: Experience shipping production code subsequently maintained by other engineers
Software Development: Ability to assess code quality, implementation decisions, and realistic development practices
Workers compensation consultant managing Canadian claims, benefits, return - to - work plans, and settlements. Supporting Sedgwick’s claims administration and client service operations.
Managing Consultant advising mining organizations on sustainability, strategy, risk, transactions, and transformation. Leading multidisciplinary engagements that create measurable value for ERM clients.
Senior Adaptive Consultant implementing and supporting Workday Adaptive Planning solutions for global clients. Leading projects, advising customers, and mentoring consultants at Kognitiv.
Intermediate Hydrogeologist and Field Lead delivering groundwater investigations for ERM's sustainability consultancy. Leading field teams, client projects, and hydrogeology business growth across Canada.
Intermediate Hydrogeologist and Field Lead delivering groundwater investigations for ERM's Canadian sustainability consultancy. Leading field teams, technical analysis, client relationships, and hydrogeology business growth.
Principal Consultant leading mining and resource capital projects for ERM, a global sustainability consultancy. Guiding permitting, Indigenous engagement, closure planning, and multidisciplinary delivery.
Principal permitting consultant guiding environmental approvals for ERM's sustainability projects across Canada. Shaping regulatory strategies for mining, energy, transportation, and oil and gas clients.
Principal Consultant leading capital project delivery, permitting, stakeholder engagement, and mine closure planning at ERM, a global sustainability consultancy. Advising resource - sector clients across Canada.
Principal permitting consultant guiding complex regulatory approvals for ERM, a global sustainability consultancy. Advising mining, energy, transportation, and oil and gas projects across Canada.