Resume Score

Check how well your resume matches this job before you apply.

Sign in to check score

About the role

  • QA Strategist shaping testing, observability, and LLM evaluation for Blue J’s AI-powered tax research software. Improving delivery confidence across engineering, product, and customer teams.

Responsibilities

  • Advocate for end users during discovery, planning, and design
  • Surface unknowns, clarify requirements, and identify overlooked use cases
  • Collaborate with Product and domain experts to devise real-world test scenarios
  • Define success measures and ensure systems are observable, debuggable, and actionable
  • Lead exploratory and regression testing for new features
  • Investigate bugs and user feedback, identify root causes, and develop solutions with developers
  • Write lightweight tools supporting testing workflows
  • Occasionally fix bugs in production code
  • Coach and enable developers on quality practices
  • Expand LLM evaluation tools and use production metrics to assess algorithmic improvements
  • Collaborate with QA peers and developers across engineering pillars to solve systemic and non-deterministic quality issues
  • Attend quarterly in-person meetings and occasional team get-togethers

Requirements

  • 5+ years of experience in a quality-focused role
  • Technical background understanding how software systems function beyond test cases
  • Comfortable reading and debugging production code
  • Ability to write maintainable tools to automate QA tasks and support testing
  • Ability to write scripts or SQL queries to analyze millions of rows
  • Experience with or knowledge of Node.js, TypeScript, Python, LLM APIs, AWS, Terraform, Kubernetes, Docker, Datadog, Sentry, Mixpanel, GitHub, CircleCI, PostgreSQL, Elasticsearch, DynamoDB, S3, Kafka, NATS, and SNS/SQS
  • Initiative, ownership, and ability to prioritize high-impact tasks
  • Strong written and oral communication skills
  • User-first mindset focused on product impact
  • All candidates must be eligible to work in Canada
  • Must live within driving distance of the Toronto office
  • Additional valuable experience includes data pipelines at scale, RAG systems, heterogeneous data ingestion, fault-tolerant distributed systems, high-growth startups or agile environments, and LLMs/prompt evaluation

Benefits

  • Equity compensation
  • Thoughtful benefits package supporting employees and their families
  • Flexible hybrid environment with the option to work remotely most of the time
  • Travel and accommodation covered for onboarding if based outside the local commuting area
  • Meaningful informal leadership opportunity
  • Continuous learning culture
  • Inclusive and accessible hiring experience with accommodations available
  • Competitive salary

Job title

Job type

Full Time

Experience level

Mid levelSenior

Salary

CA$120,000 - CA$135,000 per year

Degree requirement

No Education Requirement

Tech skills

AWSDistributed SystemsDockerDynamoDBElasticSearchJavaScriptKafkaKubernetesNode.jsPostgresPythonSQLTerraformTypeScript

Location requirements

HybridTorontoCanada

Report this job

Found something wrong with the page? Please let us know by submitting a report below.