Applied AI Engineer

Posted 1 hour ago

Apply Now

Resume Score

Check how well your resume matches this job before you apply.

Sign in to check score

About the role

  • Applied AI Engineer building production LLM runtime, retrieval, guardrails, and evaluation. Enabling reliable, cost-efficient AI features for mission-critical customer workflows.

Responsibilities

  • Build and operate the runtime, tool calling and orchestration for production agent workflows
  • Design model selection and tiering with cost and latency as first-class constraints
  • Implement and tune retrieval (RAG) and grounding where it measurably improves quality
  • Stand up evaluation harnesses and golden sets to make ship or no-ship calls and catch regressions and drift
  • Design and implement evaluation frameworks for model and agent quality
  • Build guardrails and safety controls for retrieved content and tool outputs
  • Own production behaviour, including latency, cost, reliability and failure modes
  • Partner with product to decide when an agent is or is not the right tool
  • Work with product teams and customers to validate results and outcomes
  • Prove value with the smallest useful slice, then scale what works
  • Instrument and measure impact so evidence drives what ships
  • Build automated QA, including tests, quality gates and regression checks, into the delivery pipeline
  • Set evaluation and safety patterns, contribute to interviews and mentor team members
  • Foster technical excellence, customer obsession and pragmatic shipping
  • Partner with product teams across the portfolio and their roadmaps
  • Report to the Head of AI Platform Engineering
  • Help hire the team and establish engineering patterns as a founding AI hire

Requirements

  • Demonstrated experience shipping software backed by LLMs or ML to real production users, beyond demos and notebooks
  • Ability to reason about evaluation, not just prompting
  • Treat cost and latency as design constraints from the start
  • Clear communication with engineers, product partners and customers
  • Focus on outcomes and value, with pragmatic decisions to ship
  • Bonus: Agent or RAG systems in production
  • Bonus: Model routing or measurable cost reduction
  • Bonus: Evaluation harnesses driving decisions; LLM-as-judge calibrated against humans
  • Bonus: Fine-tuning or distillation
  • Bonus: Knowledge of when not to use an agent
  • Bonus: Strength in Python or C#/.NET
  • Bonus: Experience grounding AI in structured, well-modelled business data
  • Bonus: System and platform design, and trade-off judgement
  • Bonus: Experience building or operating enterprise-scale software, including reliability, security and integration demands
  • Bonus: Productive use of AI coding assistants with judgement when supervising output

Job title

Job type

Full Time

Experience level

Mid levelSenior

Salary

Not specified

Degree requirement

No Education Requirement

Tech skills

Python.NET

Location requirements

RemoteCanada

Report this job

Found something wrong with the page? Please let us know by submitting a report below.