Principal Databricks Data Engineer

Posted via LinkedIn Recruiter (not a company profile)

Posted last week

Apply Now

Resume Score

Check how well your resume matches this job before you apply.

Sign in to check score

About the role

  • Hiring a Principal Databricks Data Engineer in Toronto, ON for a full-time permanent role. Requires 12-18 years of data engineering experience and expertise in Databricks and Spark.

Responsibilities

  • We are hiring a Principal Databricks Data Engineer for a full-time permanent role in Toronto, ON. The role involves modernizing legacy Cloudera platforms to Databricks Lakehouse, redesigning ingestion and transformation patterns, refactoring legacy Hive/Impala logic into PySpark and Spark SQL ELT pipelines, ensuring data reconciliation and audit integrity during migration, and designing enterprise Data Warehouse and Lakehouse architectures.

Requirements

  • 12–18 years of overall data engineering experience; 8+ years in enterprise Data Warehouse and Data Lake platforms; 5+ years hands-on with Databricks and Spark at scale; strong experience in modernizing legacy Cloudera platforms (CDH/CDP, Hive, HBase, Impala, Spark) to Databricks Lakehouse; ability to redesign ingestion, transformation, and consumption patterns from HDFS-based architecture to cloud object storage and Delta Lake; refactor legacy Hive/Impala logic into PySpark and Spark SQL ELT pipelines; ensure data reconciliation, audit integrity, and consistency during migration; design and govern enterprise Data Warehouse and Data Lake/Lakehouse architectures.

Job title

Job type

Full Time

Experience level

Senior

Salary

Not specified

Degree requirement

No Education Requirement

Tech skills

DatabricksSparkClouderaHiveHBaseImpalaPySparkSpark SQLDelta LakeHDFS

Location requirements

Linkedin Recruiter PostTorontoOntario Toronto

Report this job

Found something wrong with the page? Please let us know by submitting a report below.