SDET 4 driving quality and reliability for AI-enabled VoiceAI platform at Phreesia. Lead testing strategies for AI agents and LLM-powered workflows in a modern CI/CD environment.
Responsibilities
Design, build, and maintain comprehensive automated test suites, iterating continuously within a CI/CD environment to ensure fast, reliable feedback
Define and execute testing strategies for AI-enabled Voice AI agents and LLM-driven workflows, validating behavior, accuracy, performance, and reliability
Conduct end-to-end testing across front-end, back-end, API, and performance layers to ensure system robustness in a distributed, message-based architecture
Leverage AI-assisted tools and techniques to support test creation, test data generation, failure analysis, and exploratory testing where appropriate
Perform root cause analysis for test and production failures, collaborating closely with engineers and SDETs to drive timely resolution
Integrate automated testing into CI/CD pipelines and troubleshoot pipeline failures related to test execution and environment issues
Assist in troubleshooting and resolving production escalations, contributing to system stability and long-term product quality
Participate in code and merge request reviews for both application and test code, providing actionable feedback to uphold quality standards
Monitor application performance, reliability, and operational health using AWS tools such as CloudWatch, X-Ray, and CloudTrail, identifying and addressing issues proactively
Support products and services over their full lifecycle, from initial development through production and ongoing maintenance
Test and validate systems built on cloud-native and serverless technologies, including AWS Lambda, SQS, SNS, DynamoDB, OpenSearch, as well as Azure and GCP services
Collaborate with Product Managers, developers, and other SDETs to design effective test strategies and scenarios within an agile development process
Test microservices and APIs in highly distributed systems, ensuring correctness, resilience, and observability
Work with database architecture and operations, including DynamoDB, SQL-based systems, and NoSQL stores such as MongoDB
Develop and maintain quality and test automation frameworks for .NET / .NET Core applications
Continuously iterate on and improve test automation processes, frameworks, and tooling to increase coverage, reliability, and efficiency
Proactively identify gaps in test coverage, quality risks, or tooling, and propose solutions to address them
Create, maintain, and execute test plans and scenarios; curate high-quality test data; and monitor results to ensure comprehensive validation
Mentor junior SDETs and QA engineers, sharing best practices in testing, automation, and effective AI-assisted quality engineering
Requirements
5+ years of experience in Quality Assurance or as a Software Development Engineer in Test (SDET), testing complex, distributed systems
3+ years of professional programming experience using C# and TypeScript, including writing and maintaining automated API and integration test suites
Hands-on experience testing or validating Voice AI agents and LLM-enabled workflows in production or pre-production environments
Demonstrated experience leveraging AI-assisted tools to support testing activities such as test design, data generation, exploratory testing, and failure analysis
Experience with infrastructure as code, containerization, and CI/CD pipelines (Docker, GitLab CI/CD preferred)
Strong experience in UI automation using frameworks such as Playwright, Cypress, or Selenium
Cloud experience with AWS, GCP, or Azure, with AWS strongly preferred
Solid understanding of testing patterns, automation frameworks, and quality strategies across functional, integration, and performance testing
Familiarity with modern web technologies, with a willingness and curiosity to learn new tools and frameworks as systems evolve
Experience building and testing .NET / .NET Core–based applications in a cloud-native environment
Benefits
Remote First: 100% Remote work + home office expense reimbursements+ monthly reimbursement for cell phone, internet and wellness.
Take time when you need time: Flexible PTO + company holidays
Top class healthcare benefits: Variety of healthcare benefits for you and your family (and your pets!) starting day one
Care about your families: Generous top-up for parental leave benefits
Support personal development: Continuing education and professional certification reimbursement
Connecting in person: Various offsite events and activities for team to connect and meet in person, to support team building and engagement.
Giveback to community: Local in-person volunteer events, and give back programs to our communities.
Recognition and perks: We have a company wide recognition tool (Phireworks) to celebrate milestones, recognize achievements and strengthen your bond with your teams. You can accumulate points and redeem them for a wide catalogue of items!
Diversity and inclusive environment: At Phreesia, all employees are encouraged to bring their authentic self to work, feel supported and perform at their best. We have a variety of Employee Resources Groups (ERGs) which bring together individuals from a wide range of backgrounds, experiences and perspectives, and seek to foster a sense of shared community and empowerment for employees who share a common social identity, such as gender, race, ethnicity, and sexual orientation. Opportunity to join an Employee Resource Group.
Test Engineer developing and executing functional and integration tests for new features in Ford's Phoenix Infotainment system. Collaborating with Agile teams to deliver product capabilities and automated tests.
QA Engineer specializing in API automation and microservices testing at Hatch Innovations. Collaborating with global teams and ensuring quality of backend services in a hybrid environment.
Test Engineering Specialist developing and maintaining system verification for Telesat's LEO constellation communication system. Participating in testing activities with partners and maintaining the lab.
Software Development Engineer in Test at Best Buy Canada focusing on automated QA practices and testing. Collaborate with teams to ensure high software quality during deployments.
Internship for Automation Test Engineer testing state - of - the - art product in AI - driven Fraud and Risk Platform. Collaborate with QA to enhance testing through automation using modern tools.
Intern, Test Engineer contributing to high - quality software products at PICTON Investments. Gain hands - on testing experience with web and SaaS applications in a dynamic environment.
Senior Software Engineer in Test ensuring quality for platform services that power Kong's products. Collaborating with cross - functional teams to deliver reliable and secure software solutions.
QA Automation Engineer at Axya building testing frameworks for AI - driven procurement solutions. Collaborating with teams to ensure quality and performance of web platforms and APIs.
SDET QA Automation role requiring 8+ years experience with Selenium/Cypress/Playwright, Java/Python, API testing, CI/CD tools, and automation frameworks. Performance testing experience is beneficial.