Technical lead building safe, evaluable LLM agents and AI infrastructure for OpenLoop’s telehealth platform. Setting architecture, observability, retrieval, and model operations direction.
About the role
- Applied AI Engineer building production LLM, RAG, and agent systems for Newfold Digital’s global web technology brands. Delivering reliable AI products for millions of customers.
Responsibilities
- Build production AI experiences across Network Solutions, including conversational agents, business and website creation, knowledge and FAQ agents, content experiences, domain discovery, and intelligent automation
- Translate product requirements into practical AI solutions using LLMs, RAG, tool calling, agents, deterministic workflows, or traditional software where appropriate
- Build production AI services using Python, FastAPI, asynchronous workers, PostgreSQL, Redis, queues, model APIs, and external tools
- Implement reliable tool-calling agents and multi-step workflows interacting with internal APIs, MCP tools, business systems, and knowledge sources
- Build event-driven workflows with retries, dead-letter handling, idempotency, and failure recovery
- Build and improve RAG systems covering ingestion, chunking, embeddings, hybrid retrieval, reranking, metadata filtering, context construction, and citations
- Integrate models from OpenAI, Anthropic, Google, xAI, and open-weight ecosystems
- Build and run AI evaluations using curated datasets, regression tests, retrieval metrics, LLM-as-judge techniques, groundedness checks, and tool-execution evaluation
- Debug hallucinations, retrieval failures, incorrect tool usage, agent loops, latency issues, provider failures, and unexpected inference costs
- Break product goals into manageable pieces, validate assumptions, implement solutions, and discuss architecture tradeoffs with senior engineers
- Turn product goals into testable technical tasks and prototype alternatives
- Measure quality, latency, reliability, safety, and cost and use those signals to improve solutions
- Own implementation across APIs, workflows, data, queues, model integration, evaluations, observability, and production support for assigned features
- Use AI coding agents to increase engineering velocity while preserving testing discipline, security, maintainability, and code quality
Requirements
- 3 or more years of professional software engineering experience building production backend, distributed, or cloud-based systems
- At least 1 year of hands-on experience building Applied AI, LLM, RAG, NLP, or agent-based applications
- Strong Python skills including FastAPI, asynchronous programming, Pydantic, SQLAlchemy or SQLModel, and production API development
- Good backend and distributed systems fundamentals including REST APIs, concurrency, background processing, caching, reliability, and production debugging
- Hands-on experience with RabbitMQ, Kafka, Azure Service Bus, or equivalent queue and messaging architectures is required
- Hands-on experience integrating LLM APIs and building structured output, function calling, tool calling, or agent execution workflows
- Experience building or contributing to a RAG or knowledge-grounded system, including retrieval, embeddings, indexing, context construction, and evaluation
- Working knowledge of PostgreSQL and data modeling, with exposure to Redis, pgvector, vector databases, or hybrid search technologies
- Demonstrated use of AI coding agents such as Cursor, Claude Code, Codex, or equivalent tools as a regular part of software engineering work
Job title
Job type
Experience level
Salary
Not specified
Degree requirement
No Education Requirement
Tech skills
Location requirements
Report this job
Found something wrong with the page? Please let us know by submitting a report below.
Senior software engineer evaluating AI coding agents such as Codex and Claude Code. Providing rigorous written and video feedback on engineering quality.
Senior software engineer evaluating AI coding agents for G2i. Assessing engineering judgment, explanations, and trustworthiness across Codex, Claude Code, and Cursor.
Senior engineer evaluating AI coding agents such as Codex, Claude Code, and Cursor. Providing rigorous written and video feedback on engineering judgment, reasoning, and interaction quality.
Forward Deployed Engineer building full - stack AI solutions on AWS and Azure for Huron’s consulting clients. Partnering daily with business users to deliver tested features.
Staff AI Security Engineer securing EQ Bank’s enterprise AI, machine learning, and cloud platforms. Building guardrails, controls, automation, and detection capabilities for Canada’s Challenger Bank.
Staff Software Engineer owning OAuth, authorization, and agent delegation systems. Building governed identity infrastructure for Redpanda’s enterprise AI data platform.
Senior software engineer evaluating AI coding agents for G2i’s engineering team. Assessing reasoning, explanations, and engineering judgment in Codex, Claude Code, and Cursor interactions.
Senior software engineer evaluating AI coding agents such as Codex, Claude Code, and Cursor. Providing rigorous written and video feedback on engineering quality.
Senior engineer evaluating Codex, Claude Code, and Cursor interactions for G2i. Providing rigorous written and video feedback on AI - generated engineering work.