About the role

Technical Product Manager owning AI agents, evaluations, and product quality for Caseware’s audit and financial reporting software. Driving practitioner-validated delivery across applied science and engineering teams.

Responsibilities

Own agents, the agent builder, the evaluation builder, and agentic memory as products, including their roadmaps, sequencing, and business outcomes
Own the applied-science delivery roadmap and cross-team orchestration for the Caseware AI platform
Sequence evaluation, agentic-memory, and platform workstreams into a single delivery plan
Own the product acceptance bar for every AI feature and establish evaluation baselines
Work with domain experts and partner accounting and audit firms to validate AI outputs against professional standards
Serve as the practitioner voice and SME proxy by shadowing audit workflows, testing prototypes, and identifying actionable failure modes
Partner daily with engineering to convert problem statements into requirements and acceptance criteria
Monitor AI model developments, architectures, agent patterns, and competitor activity
Define and own product health metrics, including task completion, correction rates, override frequency, trust signals, and time-to-completion
Collaborate with applied scientists, engineers, domain experts, and external partner firms
Make autonomous product decisions and own product outcomes

6+ years of experience in technology-focused product management or product development
3–4 years working deeply with AI or machine learning–driven products
Software engineers transitioning into product management are welcome
Proven ability to drive delivery across multiple technical teams, coordinating dependencies, shared interfaces, and timelines across applied science and platform engineering
Direct experience building and running evaluation frameworks for production AI products
Experience defining acceptance bars, running human-in-the-loop and practitioner validation, tracking regressions across releases, and using results to decide what ships
Technical fluency in LLMs, RAG, prompt engineering, embeddings, and agentic patterns
High agency and ownership of outcomes
Intellectual curiosity about AI, including following model releases, reading research, and understanding frontier-lab developments
Experience representing the voice of customers or practitioners and working with domain experts or subject matter specialists
Experience defining quality bars in high-stakes, regulated, or professional-grade contexts
Comfort with ambiguity
Genuine curiosity about auditing and appreciation for high-quality work
Willingness to build accounting and audit domain fluency on the job