About the role

Staff Quality Engineer defining quality strategy for LLM and agentic products at StackAdapt. Building AI evaluation frameworks and automated testing for its programmatic advertising platform.

Responsibilities

Own and drive the end-to-end quality strategy for the Automation and Intelligence Pillar pod's AI such as our Ivy product, from test architecture through release readiness
Design and build evaluation frameworks for non-deterministic systems, including LLM output quality, agent behaviour, regression detection, and guardrail/safety testing
Develop reliable, scalable automated test suites and tooling covering functional, system, API, integration, UI, and regression testing, and integrate them into the CI/CD pipeline
Establish quality metrics and benchmarks for accuracy, latency, cost, hallucination/failure rates, enabling data-driven release decisions
Partner with Engineering Managers, Product Managers, Designers, ML/Data Science, and other engineering teams to align requirements, priorities, and definitions of done
Mentor across the QE organization, promote QE best practices, drive process improvements, and improve AI testing company-wide
Participate in technical planning, code review, and architectural decision-making, advocating for testability and reliability
Provide timely, actionable test result feedback to the pod, QA team, and stakeholders

7+ years of professional, hands-on software quality and test automation experience, with a track record of setting technical direction on complex systems
Deep, hands-on experience with API & UI test automation frameworks such as Playwright, Cypress, Jest, or Junit
Fluency in at least one of TypeScript, JavaScript, Python, or Java with preference for TypeScript
Experience testing AI/agentic systems (evals, prompt/regression testing, output validation) and using AI to scale automation (test generation, self-healing tests, failure triage) with technologies such as OpenAI, Anthropic
Experience with CI/CD tools like Jenkins, Github Actions, or CircleCI
Strong understanding of scalable system design for web-based APIs and backend processing systems
In-depth understanding of QA methodologies in a software company with fast release cycles
Proven ability to define test case design standards and coverage strategy such as edge/negative, compatibility, performance and concurrency scenarios and to own a test case management practice (e.g. TestRail) that others build on
Knowledge of React, including but not limited to adding data-testids
Ability to explore a codebase and use technical tools to gain an understanding of technical implementation
Experience coaching and influencing cross-functional teams on tooling, testing plans and strategies, and guiding them through execution

Highly competitive salary
Retirement/ 401K/ Pension Savings globally
Competitive Paid time off packages including birthday's off!
Access to a comprehensive mental health care program
Health benefits from day one of employment
Work from home reimbursements
Optional global WeWork membership for those who want a change from their home office and hubs in London and Toronto
Robust training and onboarding program
Coverage and support of personal development initiatives (conferences, courses, books etc)
Access to StackAdapt programmatic courses and certifications to support continuous learning
An awesome parental leave program
A friendly, welcoming, and supportive culture
Our social and team events!