Lead Databricks Engineer/Architect at Bounteous, designing and scaling cloud-based lakehouse platform while collaborating with data science teams.
Responsibilities
Architect and lead the implementation of an enterprise lakehouse on Databricks (Delta Lake, Unity Catalog, Photon, Workflows) across one or more major clouds (AWS, Azure, or GCP).
Design scalable batch and streaming data pipelines using PySpark, Spark SQL, Structured Streaming, and Delta Live Tables; establish patterns for ingestion from operational systems, event streams, and third-party APIs.
Define and enforce platform standards for data modeling (medallion architecture), CI/CD, code quality, testing, observability, and cost optimization.
Lead the governance strategy using Unity Catalog — fine-grained access control, data lineage, audit, and PII handling — in partnership with security and compliance.
Optimize Spark workloads for performance and cost: cluster sizing, Photon, autoscaling, file layout, Z-ordering, caching, and query tuning.
Partner with ML engineers and data scientists to operationalize models using MLflow, feature stores, and model serving on Databricks.
Own the cloud infrastructure footprint for the platform: networking, IAM, secrets, encryption, and Terraform/IaC for Databricks workspaces and supporting services.
Mentor a team of data engineers; lead architecture reviews, code reviews, and technical design sessions; raise the bar on engineering practices.
Engage with stakeholders across analytics, product, and finance to translate business needs into a roadmap for the data platform.
Requirements
8+ years of data engineering experience, with 4+ years building production workloads on Databricks.
Deep expertise in Apache Spark (PySpark and Spark SQL) — including performance tuning, partitioning strategy, and the Catalyst/Photon execution model.
Strong hands-on experience with Delta Lake, Unity Catalog, Databricks Workflows, and Delta Live Tables.
Production experience on at least one major cloud (AWS, Azure, or GCP), including networking, IAM, storage (S3/ADLS/GCS), and compute primitives.
Proficiency in Python and SQL; comfort with Scala is a plus.
Experience designing medallion (bronze/silver/gold) architectures and dimensional models for analytics.
Strong CI/CD and DevOps practice: Git, Terraform, Databricks Asset Bundles or dbx, automated testing of data pipelines.
Track record of leading technical projects end-to-end and mentoring engineers.
Excellent written and verbal communication; able to drive alignment with both engineering and business stakeholders.
Benefits
Bounteous is willing to sponsor eligible candidates for employment visas.
Accommodation will be provided at any point throughout the hiring process for candidates with accommodation needs.
We welcome applications from all qualified candidates.
Technical Solutions Architect designing Juniper and Aruba enterprise networking solutions for Arrow’s channel partners. Leading technical sales engagements, demonstrations, and solution architecture across North America.
Global Solutions Engineer leading MSSP partner architecture and AI security evaluations at Sophos. Building demos, proofs of value, and global technical enablement.
Conseiller SAP concevant des architectures intégrées pour un client manufacturier. Gouvernance, intégrations ERP/SCADA, transformation numérique et sélection de solutions.
Data Solution Architect designing scalable Databricks lakehouses and pipelines. Guiding cloud data engineering delivery for Lovelytics’ enterprise consulting clients.
Solution Architect leading enterprise AI architecture, governance, and delivery standards for Bits In Glass and ConveneAI. Shaping client engagements and scaling reusable AI accelerators.
Data and AI Customer Solutions Architect selling cloud, analytics, and AI solutions at Softchoice, an IT solutions provider. Leading executive sales cycles, partner co - selling, and AI adoption strategies.
Data Integration Engineer building resilient API, webhook, and data pipelines for SafelyYou’s AI - powered senior - care platform. Supporting partner integrations, backfills, monitoring, and technical documentation.
Systems Integration Engineer leading multidisciplinary rail systems integration for AtkinsRéalis. Supporting design, construction, testing, commissioning, and stakeholder coordination.
Business Solutions Manager maintaining JTI’s consumer applications and digital platforms across the Americas. Driving system rollouts, AI adoption, consumer data ecosystems, and budget efficiency.
Databricks Solution Architect designing enterprise Lakehouse and AI platforms for Bits In Glass. Building prototypes, guiding customers, and shaping technical strategies across Canada.