About the role

Technical lead building safe, evaluable LLM agents and AI infrastructure for OpenLoop’s telehealth platform. Setting architecture, observability, retrieval, and model operations direction.

Responsibilities

Set the technical direction for agent runtime and orchestration
Build evaluation systems, including regression tests and AI-graded evaluations
Define model access and operations, including provider routing, version upgrades, and outage planning
Own observability and cost attribution for AI systems
Build retrieval and grounding over company data
Translate human workflows into agent workflows and determine which work must remain human-led
Protect patient data and ensure security and privacy in AI systems
Make cloud and data infrastructure decisions on GCP and partner with the Data Platform team
Conduct code and design reviews and mentor engineers new to AI
Explain AI trade-offs to product, operations, and clinical stakeholders
Own AI architecture and technical quality while partnering with the AI Engineering Manager on people and delivery
Carry long-range technical direction for AI across the company until an AI Principal Engineer is hired

12 or more years of software engineering experience, with real technical leadership experience (tech lead, staff, or principal-level scope)
Two or more years building and running LLM-based systems in production, not just demos or personal projects
Deep, hands-on expertise in agent runtime or evaluation, plus working knowledge across the rest of the AI engineering stack
A track record of measuring AI quality with real evaluation methods and data
Experience leading a small team's technical direction from zero, or close to it
Healthcare or other regulated-industry experience (PHI, HIPAA) preferred
GCP experience preferred
Experience building an internal platform used by other engineering teams preferred
Background in classic machine learning as well as LLMs preferred
Experience scaling a team from a single pod into multiple teams preferred
Authorization to work in the location of the job posting