Senior Consultant/Manager – Data Engineer

Posted last week

Apply Now

Resume Score

Check how well your resume matches this job before you apply.

Sign in to check score

About the role

  • Data Engineer building PySpark and Databricks pipelines for Electric Mind’s financial services and wealth management clients. Debugging transformations, validating data, and supporting Azure-based environments.

Responsibilities

  • Build and maintain data pipelines on Databricks for a financial services/wealth management data platform
  • Write and maintain hand-written PySpark transforms for silver and gold layer tables, including customer, account, and transaction data models
  • Work with Lakeflow Declarative Pipelines and Auto CDC / SCD Type 2 patterns for change-data-capture merges
  • Query and validate data in Unity Catalog across dev/qa/uat using SQL warehouses
  • Debug failed pipeline runs by reading event logs, tracing bad records, and fixing schema drift or data quality issues
  • Maintain reference/lookup tables and PowerShell/Python utility scripts
  • Write and update unit/integration tests for transform logic
  • Participate in GitLab code reviews using feature branches and merge requests
  • Keep technical documentation current
  • Work alongside a senior engineer and help maintain healthy Databricks environments

Requirements

  • Python — comfortable writing clean, readable code; understands functions, modules, basic OOP
  • Databricks
  • Some exposure to Apache Spark / PySpark, or strong SQL skills plus a willingness to learn Spark quickly
  • Working knowledge of SQL (joins, aggregations, window functions)
  • Basic Git workflow: branches, commits, pull/merge requests
  • Comfortable reading other people's code and stack traces, and debugging methodically (not guess-and-check)

Job title

Job type

Full Time

Experience level

Senior

Salary

CA$110,000 - CA$160,000 per year

Degree requirement

No Education Requirement

Tech skills

ApachePySparkPythonSparkSQLUnity

Location requirements

HybridTorontoCanada

Report this job

Found something wrong with the page? Please let us know by submitting a report below.