Administrateur principal de plateformes de données chez EDC, société d’État aidant les entreprises canadiennes à réussir à l’étranger. Conception, exploitation et gouvernance de plateformes de données infonuagiques.
LP
Databricks Architect – GenAI
Posted via LinkedIn Recruiter (not a company profile)
Posted 4 months ago
About the role
- Senior Databricks Lead/Data Architect needed for a large-scale data and AI modernization program in the Greater Toronto Area. Must be hands-on with lakehouse architecture, legacy ETL migration, and team leadership.
Responsibilities
- We are hiring a senior, deeply hands-on Databricks Lead / Data Architect to drive the Databricks workstream of a large-scale data and AI modernization program for a major Canadian enterprise retail client. This is a build-and-lead role: you will own the technical direction of Databricks-based solutions end to end — architecture, lakehouse design, data engineering, migration of legacy ETL workloads, and production operations — while remaining personally hands-on in code and design. You will work side by side with the client’s VP of Data & AI and the AVP of Data Platforms & Integration and their teams, acting as the senior technical authority who turns strategy into delivered, production-grade outcomes. The immediate focus is modernizing a large on-premise ETL estate (IBM DataStage) to an Azure-native lakehouse on Azure Data Factory and Databricks, and then scaling the platform to power enterprise analytics and AI use cases. Key Responsibilities: Architect the lakehouse: Design and own scalable, secure Databricks Lakehouse architecture on Azure (Delta Lake, Unity Catalog, medallion bronze/silver/gold, ADLS Gen2) aligned to enterprise standards. Stay hands-on: Personally build and review PySpark / Spark SQL pipelines, Delta Live Tables, notebooks, and orchestration — setting the engineering bar, not just directing it. Lead legacy migration: Drive conversion of complex legacy ETL (DataStage) workloads to Databricks/PySpark and ADF, including patterns, accelerators, and reusable frameworks for code conversion and validation. Own performance & cost: Optimize cluster configuration, job performance, partitioning, and cost; establish FinOps and right-sizing practices on Databricks. Embed governance: Implement data governance, lineage, quality, and access control through Unity Catalog and Purview; ensure security, privacy, and compliance by design. Enable analytics & AI: Design Gold-layer semantic models and feature pipelines that serve BI (Power BI), advanced analytics, and ML/GenAI use cases (MLflow, Azure ML). Lead the squad: Provide technical leadership and mentoring to data engineers; define best practices, coding standards, CI/CD (Azure DevOps), and review processes. Partner with the client: Work closely with the client’s VP (Data & AI), AVP (Data Platforms & Integration), platform architects, and business stakeholders to translate requirements into delivery roadmaps and measurable outcomes.
Requirements
- 12+ years in data engineering / data platform architecture, with 4+ years of deep, hands-on Databricks delivery. Expert-level Databricks: Spark (PySpark & Spark SQL), Delta Lake, Delta Live Tables, Unity Catalog, Workflows, performance tuning, and cluster/cost optimization. Strong Azure data stack: Azure Data Factory, ADLS Gen2, Azure Key Vault, Azure DevOps (CI/CD), and Azure networking/security fundamentals. Proven migration track record: Led at least one large-scale migration from legacy ETL (e.g., DataStage, Informatica, Teradata) to a cloud lakehouse, including complex transformation logic. Lakehouse design depth: Medallion architecture, dimensional & semantic modelling, SCD handling, surrogate keys, and data quality / reconciliation frameworks. Engineering rigor: CI/CD, version control (Git), automated testing/validation, observability, and production support of mission-critical pipelines. Leadership with hands-on credibility: Demonstrated ability to lead engineers and engage senior client stakeholders while still contributing code and designs directly. Preferred: Databricks certifications (e.g., Databricks Certified Data Engineer Professional / Solutions Architect) and relevant Azure certifications (DP-203, AZ-305). Experience in retail, supply chain, merchandising, or financial-services data domains. Familiarity with IBM DataStage, DB2, Oracle, and legacy on-prem ETL estates. Exposure to agentic AI / GenAI patterns, MLOps/LLMOps, and AI-assisted code migration tooling. Experience operating a warm-standby DR and high-availability data platform.
Job title
Job type
Experience level
Salary
Not specified
Degree requirement
No Education Requirement
Tech skills
Location requirements
Report this job
Found something wrong with the page? Please let us know by submitting a report below.
Senior Data Platform Administrator shaping and operating EDC’s secure enterprise data platforms. Supporting Canadian businesses through Export Development Canada’s trade - finance solutions.
Staff full - stack engineer architecting GitLab’s data products, integrations, and knowledge graph. Building APIs, marketplace data access, and AI - powered developer intelligence.
Data Governance Architect shaping governance strategies and cloud data architectures for Lovelytics’ enterprise data and AI consulting clients. Leading technical delivery, presales, migrations, and governance implementations on Databricks.
Senior Data Engineer owning Databricks lakes, pipelines, and AI data infrastructure. Building Quandri’s AI operating system for insurance agencies and brokerages.
Senior Data Engineer architecting Azure and Databricks data platforms for TTEC Digital’s client experience solutions. Building governed pipelines, APIs, MCP integrations, and agentic AI data services.
Senior Data Engineer building Palantir Foundry data solutions for Unit8, a Swiss AI and data analytics consultancy. Supporting client delivery and establishing its Canadian market presence.
Staff Software Engineer building AWS data infrastructure and integrations for Solink’s cloud video - security platform. Driving scalability, architecture, and engineering mentorship.
Senior Azure Fabric Data Engineer building reliable pipelines and modern data platforms for Data Elephant, a Canadian data, analytics, and AI consultancy. Delivering trusted datasets for reporting, AI, and machine learning.
Senior Data Architect leading Microsoft Fabric implementations for Data Elephant, a Canadian data, analytics, and AI consulting firm. Designing secure Azure, OneLake, and Fabric platforms for clients.