About the role

Software Engineer contributing to AI evaluation and quality systems at Lattice. Focused on LLM-powered products and production AI systems.

Responsibilities

Contribute to AI evaluation pipelines, including offline evals, production tracing, and feedback systems;
Implement and maintain performance metrics (e.g., response quality, task success, reliability) using established frameworks;
Help create and maintain evaluation datasets and test cases to identify regressions;
Analyze results and propose incremental improvements to model and agent quality;
Contribute to AI system components such as RAG pipelines, retrieval systems, and multi-step workflows within existing architectures;
Write clean, maintainable Python code that integrates with LLM providers and internal services;
Support improvements to system reliability, observability, and performance in production;
Deliver well-scoped projects with guidance from more senior engineers;
Break down tasks, make steady progress, and be proactive in unblocking yourself by asking for help when needed;
Contribute to team excellence through code reviews, documentation, and knowledge sharing;
Collaborate with cross-functional partners to ship user-facing features.

2–5 years of professional software engineering experience;
Experience contributing to production systems as part of a team;
Exposure to AI/ML systems with a strong interest in LLM-powered products;
Experience debugging systems, working with data, and iterating on performance;
Proficiency in Python or a similar language;
Strong understanding of LLM concepts (prompting, RAG, evaluation);
Familiarity with backend systems, APIs, and cloud environments (e.g., AWS, GCP);
Exposure to logging, monitoring, or debugging tools;
Interest in learning tools like LangGraph, vector databases, and evaluation platforms.