Lead Product Manager, Agent Runtime, Evals

Posted 18 hours ago

Apply Now

Resume Score

Check how well your resume matches this job before you apply.

Sign in to check score

About the role

  • Lead Product Manager owning Juniper Square’s AI agent runtime, evaluations, and governance platform. Enabling private-markets fund administration teams to deploy reliable automations.

Responsibilities

  • Own the vision, strategy, and roadmap for Foundry’s agent runtime and evaluation layer
  • Define the lifecycle for agents, apps, and scripts: draft, test, publish, version, deploy, run, observe, improve, and retire
  • Define evaluation processes using golden datasets, data labeling, regression gates, automated judges, and human review
  • Shape model eligibility, routing, cost attribution, and pricing for automations
  • Define agent memory, learning, self-improvement, identity, access grants, and run modes
  • Integrate hosted agents with Juniper Square products, fund administration systems, and third-party tools
  • Design oversight tools including dashboards, run history, traces, metrics, logs, kill switches, and auditability
  • Partner with the workflow orchestration PM on agent registries, typed contracts, and workflow bindings
  • Build configuration tools for business SMEs to configure, test, and tune automations
  • Track the managed agent runtime, framework, and automation-platform market and make build, buy, and partner decisions
  • Lead across platform and adjacent teams and guide ambiguous platform tradeoffs
  • Report to the Group Product Manager for Foundry and work with a dedicated engineering team
  • Deliver real fund administration workflows and automations on Foundry, including capital calls and distributions

Requirements

  • 7+ years of product management experience in B2B or Enterprise SaaS, including 3+ years operating in an AI-oriented role
  • Proven experience owning a complex product area end-to-end, from strategy and roadmap through execution and iteration, ideally in a zero-to-one context
  • Hands-on understanding of agent evaluation, including golden datasets, data labeling, automated judges, regression testing, and human-in-the-loop decisions
  • Background building agents and replacing human-driven processes with products and automation, with real evaluation and monitoring discipline in production
  • Strong engineering partnership and understanding of typed contracts between agents and workflows, structured and deterministic outputs, and agent memory
  • User empathy for non-engineering builders and operators overseeing large numbers of automated agents
  • Fluency with LLMs and common AI tools such as Claude and Cursor
  • Ability to run prototypes and use AI tools to accelerate product work
  • Strong written and verbal communication skills
  • Track record of leading through influence, ideally mentoring or managing other product managers, and setting strategy across multiple teams

Benefits

  • Health, dental, and vision care for you and your family
  • Life insurance
  • Mental wellness coverage
  • Fertility and growing family support
  • Flex Time Off in addition to company-paid holidays
  • Paid family leave, medical leave, and bereavement leave policies
  • Retirement saving plans
  • Allowance to customize your work and technology setup at home
  • Annual professional development stipend
  • Equity

Job type

Full Time

Experience level

Senior

Salary

$200,000 - $230,000 per year

Degree requirement

No Education Requirement

Location requirements

RemoteUnited States

Report this job

Found something wrong with the page? Please let us know by submitting a report below.