About the role

Independent contractor designing complex evaluation frameworks for AI audio models. Focusing on role-play scenarios, auditing AI models, and generating training datasets.

Responsibilities

Operate autonomously to design complex evaluation frameworks and provide structured training data.
Role-Play Scenario Execution: Creating and executing complex, role-play-based evaluation scenarios that simulate realistic customer service interactions.
Model Performance Auditing: Evaluating AI model performance across standardized qualitative and quantitative metrics.
Technical Metric Evaluation: Assessing the model's basic computer programming literacy.
Representative Dataset Generation: Contributing to the development of diverse, high-quality audio datasets.

Demonstrable professional expertise in complex customer support, technical troubleshooting, or conversational AI evaluation.
Native or bilingual proficiency in the target language, including fluency across all language skills (reading, listening, writing, and speaking).
Strong analytical and verbal communication skills to confidently conduct simulated customer support role-plays.
Basic computer programming literacy, specifically a comfortable understanding of JSON structures, functions, methods, and simple logic.
A meticulous, detail-oriented approach to working with structured prompts, complex evaluation rubrics, and technical guidelines.
Required Equipment: Access to a high-quality microphone to ensure clean, reliable audio input during voice evaluations.