Technical lead building safe, evaluable LLM agents and AI infrastructure for OpenLoop’s telehealth platform. Setting architecture, observability, retrieval, and model operations direction.
About the role
- Staff AI Engineer at Robots and Pencils leading the design and delivery of AI/ML systems. Focused on architecture decisions and contributing to technical leadership in a cross-functional team.
Responsibilities
- Lead the design and implementation of complex ML/AI systems end-to-end, owning architecture decisions and driving solutions from research through production at scale
- Build and evolve scalable ML platforms, pipelines, and infrastructure that support reliable, repeatable model development and deployment across teams
- Set the standard for AI-forward engineering, using tools like Claude and Cursor with sophistication and helping the team adopt them effectively
- Partner with product, engineering, and leadership to shape AI strategy and align technical direction with business outcomes
- Translate complex AI tradeoffs, risks, and opportunities into clear narratives that drive decision-making across technical and non-technical stakeholders
- Lead design reviews and technical discussions, raising the bar for engineering rigor and constructive challenge across the team
- Define AI architecture and engineering standards, bringing depth on tradeoffs, long-term implications, and responsible AI practices
- Mentor and grow junior and mid-level engineers, multiplying impact through coaching, code reviews, and pairing on hard problems
- Take ownership of the most ambiguous and highest-stakes pieces of work, driving them through to production with care for reliability, cost, and safety.
Requirements
- 7+ years professional software engineering experience, with 4+ years focused on AI/ML systems in production and deep hands-on experience with generative AI development
- Expert software engineering background (Python or similar) with strong design sensibilities for scalable, maintainable systems
- Deep expertise with cloud platforms, including in-depth understanding of AWS services and AWS GenAI offerings
- Proven track record designing and shipping complex agentic systems in production environments
- Mastery of AI frameworks and orchestration tools
- Strong experience with evaluation frameworks and observability tools for LLM apps, including building these capabilities where they don't yet exist
- Deep understanding of AI safety, responsible AI principles, prompt injection defenses, and PII handling
- Extensive experience building RAG pipelines: chunking strategies, embedding models, vector databases, and advanced retrieval techniques
- API design experience, including architecting and integrating with internal and third-party services at scale
- Advanced cost optimization expertise: token economics, caching strategies, model routing, quantization
- Strong working knowledge of Docker and Kubernetes for containerized deployments
- Demonstrable, day-to-day usage and expert knowledge of AI-forward coding tools such as Claude Code and Cursor.
Job title
Job type
Experience level
Salary
Degree requirement
Tech skills
Location requirements
Report this job
Found something wrong with the page? Please let us know by submitting a report below.
Senior software engineer evaluating AI coding agents such as Codex and Claude Code. Providing rigorous written and video feedback on engineering quality.
Senior software engineer evaluating AI coding agents for G2i. Assessing engineering judgment, explanations, and trustworthiness across Codex, Claude Code, and Cursor.
Senior engineer evaluating AI coding agents such as Codex, Claude Code, and Cursor. Providing rigorous written and video feedback on engineering judgment, reasoning, and interaction quality.
Forward Deployed Engineer building full - stack AI solutions on AWS and Azure for Huron’s consulting clients. Partnering daily with business users to deliver tested features.
Staff AI Security Engineer securing EQ Bank’s enterprise AI, machine learning, and cloud platforms. Building guardrails, controls, automation, and detection capabilities for Canada’s Challenger Bank.
Staff Software Engineer owning OAuth, authorization, and agent delegation systems. Building governed identity infrastructure for Redpanda’s enterprise AI data platform.
Senior software engineer evaluating AI coding agents for G2i’s engineering team. Assessing reasoning, explanations, and engineering judgment in Codex, Claude Code, and Cursor interactions.
Senior software engineer evaluating AI coding agents such as Codex, Claude Code, and Cursor. Providing rigorous written and video feedback on engineering quality.
Senior engineer evaluating Codex, Claude Code, and Cursor interactions for G2i. Providing rigorous written and video feedback on AI - generated engineering work.