QA Strategist shaping testing, observability, and LLM evaluation for Blue J’s AI-powered tax research software. Improving delivery confidence across engineering, product, and customer teams.
Responsibilities
Advocate for end users during discovery, planning, and design
Surface unknowns, clarify requirements, and identify overlooked use cases
Collaborate with Product and domain experts to devise real-world test scenarios
Define success measures and ensure systems are observable, debuggable, and actionable
Lead exploratory and regression testing for new features
Investigate bugs and user feedback, identify root causes, and develop solutions with developers
Expand LLM evaluation tools and use production metrics to assess algorithmic improvements
Collaborate with QA peers and developers across engineering pillars to solve systemic and non-deterministic quality issues
Attend quarterly in-person meetings and occasional team get-togethers
Requirements
5+ years of experience in a quality-focused role
Technical background understanding how software systems function beyond test cases
Comfortable reading and debugging production code
Ability to write maintainable tools to automate QA tasks and support testing
Ability to write scripts or SQL queries to analyze millions of rows
Experience with or knowledge of Node.js, TypeScript, Python, LLM APIs, AWS, Terraform, Kubernetes, Docker, Datadog, Sentry, Mixpanel, GitHub, CircleCI, PostgreSQL, Elasticsearch, DynamoDB, S3, Kafka, NATS, and SNS/SQS
Initiative, ownership, and ability to prioritize high-impact tasks
Strong written and oral communication skills
User-first mindset focused on product impact
All candidates must be eligible to work in Canada
Must live within driving distance of the Toronto office
Additional valuable experience includes data pipelines at scale, RAG systems, heterogeneous data ingestion, fault-tolerant distributed systems, high-growth startups or agile environments, and LLMs/prompt evaluation
Benefits
Equity compensation
Thoughtful benefits package supporting employees and their families
Flexible hybrid environment with the option to work remotely most of the time
Travel and accommodation covered for onboarding if based outside the local commuting area
Meaningful informal leadership opportunity
Continuous learning culture
Inclusive and accessible hiring experience with accommodations available
Infrastructure penetration tester conducting enterprise, cloud, and red - team assessments for cybersecurity consultancy Packetlabs. Reporting vulnerabilities and guiding clients on remediation and defensive improvements.
Hardware penetration tester assessing embedded and IoT device vulnerabilities for Packetlabs, a cybersecurity consulting firm. Reverse engineering firmware and advising clients on practical remediation.
Senior videogame tester validating Seismic Squirrel’s narrative - driven games across PC, console, and mobile platforms. Leading QA, defect reporting, and test planning in Edmonton’s hybrid test lab.
Quality assurance and regulatory affairs intern supporting Saputo’s dairy product compliance, labeling, and technical specification projects. Collaborating with plants and departments in an on - site internship.
Quality Engineer developing automated UI and API testing solutions for Avanade’s large - scale software systems. Creating test plans, executing regression testing, and reporting defects.
Analyste AQ testant les API, données et intégrations AWS de Tecsys, fournisseur mondial de logiciels cloud pour la santé et la distribution. Automatisation des tests et validation de la qualité de bout en bout.
QA Analyst testing APIs, data flows, and integrations for Tecsys, a cloud - based AI software provider serving healthcare and distribution supply chains. Automating tests and validating performance across AWS systems.
Senior QA Analyst testing ERP solutions and related applications for SMS Equipment, North America’s largest Komatsu dealer. Driving enterprise testing, defect resolution, and QA improvements.
Claims quality specialist auditing personal and commercial claims for Sedgwick, a global claims management company. Reviewing case files, data quality, vendor performance, and compliance.