About the role

Senior Applied AI/ML Scientist advancing trustworthy agents, evaluations, and ML capabilities. Sprout Social powers social-media insights for more than 30,000 brands worldwide.

Responsibilities

Develop agents and agentic features with engineering teams
Define agent quality standards, evaluations, judges, guardrails, and required tools
Lead model selection, development, and validation
Design and run A/B tests and other rigorous experiments to measure customer impact
Set and evolve quality standards across agents
Partner with engineers building and operating agents and evaluation infrastructure
Improve agent-platform capabilities through prompt iteration, reusable judges, context-window management, tool use, and adoption of state-of-the-art models
Communicate AI capabilities, limitations, and quality to product, design, engineering, and leadership
Own evaluation and quality for agents, drive evidence-based model-selection and improvement decisions, and contribute to AI quality roadmaps and methodologies

5+ years of applied AI/ML experience with a strong track record evaluating, measuring, and improving shipped AI/ML or LLM systems with measurable impact
Experience designing evaluation frameworks, building and calibrating judges or automated evaluators, and establishing reliability for ML or LLM/agentic systems
Strong judgment in model selection and validation
Rigorous experimentation and statistical skills, including A/B testing and power analysis
Foundational ML and modeling knowledge, including Python, common ML frameworks, transformers, and embeddings
SQL proficiency and comfort with large datasets
Ability to work closely with product, design, and engineering and translate technical quality questions for stakeholders
Preferred: experience evaluating production LLM-driven or agentic products
Preferred: experience defining quality bars or evaluation methodologies
Preferred: rapid prototyping with new AI/ML techniques
Preferred: familiarity with eval tooling, observability (e.g. Datadog, Phoenix), and MLOps
Preferred: experience influencing product roadmaps with evidence about AI quality and feasibility
Preferred: awareness of model security, bias mitigation, and responsible-AI practices
Must be based in Alberta, British Columbia, or Ontario, Canada

100% employer-paid health benefits, including medical, dental, vision, life insurance, and disability for the employee and eligible dependents
25 vacation days annually
5 paid sick days
All public holidays
Company-wide Rest & Recharge days
Free access to Modern Health, including coaching, therapy sessions, and digital wellness resources
$950 CAD annual Lifestyle Spending Account for physical, mental, and financial well-being
One-time $550 USD-equivalent home office setup stipend
Monthly $50 USD-equivalent internet stipend
No-cost confidential access to financial experts through Your Money Line
Subsidized childcare and eldercare options through Care.com
Company match for donations to eligible organizations