AI conversation evaluator helping Gramian, an IT talent consultancy, improve recommendation quality. Reviewing interpersonal scenarios and identifying missing context through structured qualitative analysis.
Responsibilities
Review AI conversations involving real-world interpersonal situations and everyday decision-making
Identify missing context and recommend relevant follow-up questions to improve AI responses
Evaluate whether additional information would materially change an AI-generated recommendation
Distinguish load-bearing information affecting a decision from non-essential or descriptive details
Conduct qualitative reviews and document findings using structured evaluation guidelines
Collaborate with research teams to ensure consistent, evidence-based assessments
Maintain accuracy, clarity, and consistency while meeting project deadlines
Requirements
At least 3–5 years of relevant direct professional experience in one or more listed domains
Licensed Social Work pathway: Master's degree in Social Work (MSW) or an equivalent graduate-level qualification
Licensed Social Work pathway: LCSW, LICSW, LMSW, or an equivalent applicable social work license or credential
Experience in client assessment, case review, interviewing, counseling, or related social-work practice
Counseling, Therapy, or Psychology pathway: Master's degree in a relevant discipline with an LMFT, LPC, LCPC, or equivalent counseling or therapy license or credential; or PhD or PsyD in Clinical, Counseling, Social, or Industrial-Organizational Psychology
Relevant experience in counseling, therapy, psychological assessment, interviewing, case review, or other client-facing professional work
Qualitative Research, HCI, or Communication pathway: MA, PhD, or equivalent advanced expertise in Human-Computer Interaction, Communication, Anthropology, Sociology, Qualitative Research, or a closely related discipline
Experience conducting qualitative research, interviews, user research, conversation analysis, thematic analysis, or qualitative coding
Ability to analyze conversations systematically and identify missing context, communication patterns, and opportunities for clarification
Mediation, Coaching, Student Services, Organizational Consulting, or Related Advisory Fields pathway: relevant professional qualification, credential, or demonstrated subject-matter expertise
At least 5 years of direct advisory or client-facing experience for candidates qualifying through this pathway
Experience in coaching, mediation, student advising, organizational consulting, or comparable professional work
Demonstrated ability to gather context, understand individual or organizational needs, and provide informed recommendations
Senior AI architect designing generative and agentic AI solutions for GE Vernova’s grid automation business. Leading cloud, edge, MLOps, and production deployment architecture.
Senior AI architect designing generative and agentic AI solutions for GE Vernova’s grid automation business. Leading cloud, edge, and production deployment architecture.
Senior AI Value Architect driving Celonis - powered AI and process transformations for North American enterprises. Orchestrating value realization, executive sponsorship, and complex business workstreams.
Applied AI Value Architect driving Celonis enterprise AI transformations across finance and supply chain processes. Orchestrating customer roadmaps, AI - enabled programs, and ROI realization.
French (Quebec) speech contributor recording spontaneous conversations for RWS AI technology projects. Annotating speech data and metadata to improve speech recognition and conversational AI systems.
Practice Architect leading production agentic AI and full - stack platforms for TEKsystems’ business technology services. Directing architecture, coding, and distributed engineering delivery.
AI Transformation Lead guiding enterprise adoption of Pencil’s Generative AI advertising SaaS. Structuring client transformations, designing AI processes, and feeding product insights into Pencil’s roadmap.
Senior AI Analyst developing and deploying cloud - based AI solutions for CBC/Radio - Canada’s public - service media systems. Integrating LLMs, intelligent agents and AI capabilities into digital platforms.
AI Business Value Advisor helping Keyrus, an AI and data transformation consultancy, identify use cases and build value roadmaps. Connecting business strategy, process redesign, and measurable AI outcomes.
AI Foundation Architect designing secure, scalable enterprise AI foundations for Keyrus, which industrializes AI for business clients. Connecting cloud, data, applications, models, agents, and security.