Technical Lead governing AI data readiness and API gateways at Coupa, whose platform helps businesses optimize total spend. Defining safeguards for GenAI applications, data quality, compliance, and observability.
About the role
- Data Scientist responsible for ensuring reliability of ML models at Jobber. Collaborating closely with product teams and contributing to AI systems.
Responsibilities
- Design, implement, and maintain ML model validation frameworks, including custom evaluation metrics, loss functions, and statistical tests, to ensure model quality before and after deployment.
- Build and own regression test suites for ML and LLM models, catching performance regressions and unexpected behaviour across model updates and data drift scenarios.
- Develop and execute MCP evaluations, systematically assessing model capabilities, edge cases, and failure modes across relevant business contexts.
- Monitor models in production using statistical process control, drift detection, and alerting pipelines; proactively surface issues before they impact customers.
- Collaborate with senior data scientists to contribute to the design and refinement of ML model architectures, offering feedback grounded in validation results.
- Document evaluation methodologies, test results, and monitoring runbooks clearly enough that stakeholders across technical and business teams can understand model health.
- Stay current with advancements in LLM evaluation techniques, AI safety, and model observability, and apply emerging best practices to our workflows.
- Communicate findings clearly and concisely to stakeholders, translating model performance signals into actionable recommendations.
Requirements
- Industry experience in data science, machine learning, or a closely related quantitative field.
- Proficiency in Python and the core DS stack: Pandas, Scikit-Learn, XGBoost, and at least one deep learning framework (PyTorch or TensorFlow).
- Solid grasp of statistical concepts underpinning model evaluation: bias–variance tradeoff, calibration, confidence intervals, A/B testing, and data drift.
- Experience with LLM evaluation frameworks (e.g. RAGAS, Eleuther AI Eval Harness, or custom LLM eval pipelines).
- Hands-on experience designing custom evaluation metrics; you've gone beyond off-the-shelf metrics when the problem demanded it.
- Strong understanding of ML and LLM model architectures — you can reason about how a model is built and why it behaves the way it does.
- High proficiency in SQL for data exploration, feature validation, and debugging model inputs.
- Exceptional attention to detail — you treat model validation with the same rigour as software QA.
- Strong written and verbal communication skills; comfortable presenting findings to both technical peers and non-technical stakeholders.
Benefits
- equity rewards
- annual stipends for health and wellness
- retirement savings matching
- extended health package with fully paid premiums for body and mind
- access to a dedicated talent development program including career coaching and opportunities for career development
Job title
Job type
Experience level
Salary
Degree requirement
Tech skills
Location requirements
Report this job
Found something wrong with the page? Please let us know by submitting a report below.
Senior Principal Data Scientist building predictive models and evaluation systems for Autodesk’s agentic AI design platform. Defining telemetry, experimentation, and product intelligence for complex user workflows.
Senior Data Scientist evaluating football ML models for SumerSports’ NFL intelligence platform. Building rigorous testing, comparison, and experimentation tooling for production model quality.
Data Scientist developing fraud detection solutions for Desjardins, North America's largest cooperative financial group. Applying data science expertise to protect financial products and services from fraud.
Data Scientist II applying SQL, Python, machine learning, and AI to Life & Health insurance products at TD Bank. Developing strategies and insights for better decisions.
Senior Data Scientist developing AI, analytics, and automation solutions for Desjardins, North America’s largest cooperative financial group. Building Finance prototypes, models, and AI agents for production.
Lead Analytics Consultant shaping Adobe Analytics and CJA strategies for Bounteous, a global AI services firm. Advising clients, designing implementations, and translating data into executive action.
Senior Data Scientist developing machine learning, GenAI agents, and analytics solutions for Sun Life’s Canadian financial - services business units. Leading technical delivery, stakeholder recommendations, and mentoring.
LP
Senior Data Scientist (GCP & Machine Learning)
LinkedIn Recruiter Post
Senior Data Scientist needed for 12 - month contract, remote - first with 1 day/month onsite in North York, ON. Must have Python, SQL, GCP (Vertex AI & BigQuery), and forecasting experience.
LP
Intermediate Data Scientist
LinkedIn Recruiter Post
Intermediate Data Scientist needed for AI company in insurance. 6 - month contract, 3 days/week in Toronto.