AI personalization rater evaluating Gemini responses using connected Google applications. Providing structured feedback to improve accuracy, relevance, helpfulness, and personalization.
Responsibilities
Evaluate AI-generated responses using personalized prompts and information retrieved from connected Google applications.
Assess whether responses are relevant, accurate, helpful, and appropriately personalized.
Identify incorrect personalization, unsupported assumptions, irrelevant recommendations, and inconsistencies.
Compare multiple AI-generated responses and assess their overall user experience.
Provide clear, detailed, and well-reasoned evaluation feedback.
Follow project guidelines to ensure consistent and high-quality ratings.
Identify subtle strengths and weaknesses in AI-generated responses.
Maintain data privacy and comply with project confidentiality requirements.
Requirements
Currently based in Canada.
Willing and able to connect relevant Google applications to Gemini, subject to required permissions and consent.
Active use of Google Workspace or consumer Google apps, with sufficient personal data or history to support personalized retrieval evaluations.
Strong analytical thinking and attention to detail.
Excellent written English and ability to provide structured feedback.
Ability to work independently in a remote environment.
Access to a desktop or laptop with a reliable internet connection.
Bachelor’s degree or equivalent practical experience in any field.
Director leading enterprise data governance, catalog, quality, and AI readiness for CBC/Radio - Canada’s public service media organization. Hybrid role guiding governance programs, councils, stakeholders, and data teams.
Director leading AI innovation pipelines, accelerators, governance, and enablement at CBC/Radio - Canada, Canada’s public broadcaster. Driving scalable AI initiatives across media, technology, and corporate teams.
Expert AI Governance Advisor defining responsible AI governance, risk controls, and regulatory practices at Beneva. Supporting insurance and financial services through ethical, compliant AI adoption.
AI risk oversight manager at Manulife, a global financial services provider. Challenging AI model and operational risks across regulated enterprise portfolios.
AI model risk oversight manager at Manulife, an international financial services provider. Challenging governance, validation, and controls for high - risk AI initiatives.
AI model risk governance leader developing enterprise frameworks, standards, and training for responsible AI at global financial services provider Manulife. Hybrid role partnering with regulated - industry stakeholders.
AI Data Center Infrastructure Engineer designing racks, power, cabling, and network layouts for Cerebras’ large - scale AI chip systems. Automating deployment workflows and integrating compute infrastructure.
Enterprise Sales Executive building new enterprise relationships and selling Toptal’s AI solutions. Driving account growth, complex deals, and AI transformation from strategy to production.
Managing Director growing Unit8's Data & AI consulting business in Montreal. Originating DACH engagements, shaping transformations, and ensuring delivery creates measurable client value.
AI character and animation artist creating consistent cyberpunk and solarpunk poker - game visuals for iLogos Game Studios. Producing animations, cinematic loops, and production - ready assets with generative AI tools.