Forward Deployed AI Lead shaping Anthropic AI strategies and operating models for Caylent cloud - services clients. Directing FDE implementation, executive workshops, adoption roadmaps, and account expansion.
About the role
- Applied AI Engineer improving Velixo Intelligence, an Excel-native ERP reporting and automation platform. Building LLM evaluations, observability, and cost optimizations for governed financial-system actions.
Responsibilities
- Build and maintain regression suites for GIQL query generation, writeback correctness, and tool selection
- Define measurable quality standards for each action type
- Run structured evaluations before model upgrades or prompt changes and recommend whether to ship
- Track quality over time to detect degradation
- Instrument Gateway actions for tokens, cache behavior, model, latency, and tenant
- Establish and maintain cost-per-action baselines
- Reduce costs through prompt compression, prompt caching, context pruning, and routing to smaller models
- Identify expensive customers, query shapes, and failure or retry loops
- Track new model releases and evaluate them against Velixo workloads
- Maintain a view of the price and performance frontier
- Recommend when to migrate, wait, or run models in parallel
- Supply unit-cost data for credit pricing and plan allowances
- Model margin impact of proposed pricing changes with the VP Finance
- Flag product decisions that may change the cost curve
- Deliver full Gateway instrumentation and a cost dashboard in the first 30 days
- Deliver an evaluation suite and documented baseline by days 31–60
- Deliver a prioritized cost-reduction roadmap and at least one measured optimization by days 61–90
Requirements
- Substantial experience building on LLM APIs in production, not in demos or notebooks
- Experience building an evaluation harness
- Hands-on experience with tracing and observability platforms such as Langfuse
- Experience with evaluation frameworks such as promptfoo
- Knowledge of token accounting, context window management, and caching strategies
- Strong analytical skills and fluency with data
- Ability to write clearly about tradeoffs for a non-technical audience
- Python or TypeScript proficiency
- Nice to have: Experience with MCP or other tool-calling frameworks
- Nice to have: Background in ERP, accounting, or financial systems
- Nice to have: Prior exposure to usage-based or credit-based pricing models
- Nice to have: Experience at a small company where you set your own priorities
Benefits
- Fully remote position
- Equal opportunity employer
- Inclusive work environment
Job title
Job type
Experience level
Salary
Not specified
Degree requirement
No Education Requirement
Tech skills
Location requirements
Report this job
Found something wrong with the page? Please let us know by submitting a report below.
AI Engineer operating production AI agents and reliability tooling for an AI marketing platform serving dental practices. Debugging LLM failures, integrations, monitoring, and scheduling systems.
Développeur IA senior concevant des architectures backend et intégrant LLM, RAG et agents chez Mila. Livraison de plateformes GenAI fiables avec Python, FastAPI et systèmes distribués.
Customer Service Representative handling global customer inquiries across phone, email, and chat. Supporting WorkMatePro’s staffing clients with issue resolution, records, sales follow - up, and CRM updates.
Senior Agentic AI Developer building reliable agentic AI systems for Osedea, a Montreal tech development firm. Designing production architectures, integrations, evaluations, and workflows for clients.
Développeur senior en IA agentique chez Osedea, concevant des systèmes intelligents pour des clients. Architecture, déploiement et amélioration de solutions IA prêtes pour la production.
Senior AI/ML software engineer developing AI strategy and scalable machine - learning components. Building FHIR - based healthcare data solutions for global health stakeholders at Smile Digital Health.
CAD Engineer training and evaluating AI agents in real CAD environments. Designing engineering geometry and reference solutions for a global technology platform.
CFD Engineer training and evaluating AI agents on expert - level fluid simulations. Building verified solutions and Python checks for a global technology platform.
AI Engineer developing AI - enabled applications and security automation for American Express, a global financial services company. Building LLM solutions, prototypes, and cybersecurity research capabilities.