Senior Data Engineer designing scalable Databricks lakehouse architectures for Bits In Glass, an AI and automation consulting firm. Leading pipelines, governance, cloud modernization, and client data solutions.
About the role
- Senior Data Engineer designing scalable Databricks Lakehouse architectures for Bits In Glass, an AI and automation consulting firm. Building governed pipelines, streaming solutions, and cloud data platforms for client modernization.
Responsibilities
- Design and implement end-to-end data architectures, including data lakes, data warehouses, and analytics platforms
- Define scalable, secure, and high-performance data integration and transformation strategies
- Collaborate with business stakeholders, data engineers, and analytics teams to translate requirements into technical solutions
- Develop data models, ETL/ELT pipelines, and frameworks for structured and unstructured data
- Provide technical leadership and mentorship to data engineers and developers
- Ensure compliance with data governance, security, and privacy standards
- Optimize existing data architectures and processes for performance and reliability
- Stay current with cloud data services and emerging technologies
- Act as a trusted advisor to clients on architecture decisions and data modernization best practices
Requirements
- 5+ years of experience in data architecture, data engineering, or analytics solution design
- Hands-on experience with data lake and warehouse technologies, such as Databricks, Snowflake, Redshift, and Synapse
- Deep understanding of data modeling, data integration, and ETL/ELT design
- Proficiency in SQL and one or more programming languages: PySpark/Python or Scala
- Experience with complex data transformations and optimization within Spark
- Solid understanding of data governance, security, and privacy best practices
- Proven experience designing, implementing, and optimizing large-scale ingestion pipelines using Databricks Autoloader
- Deep practical knowledge of building and managing reliable, self-managing ETL/ELT pipelines using Delta Live Tables
- Experience building high-throughput, low-latency streaming data ingestion solutions using Apache Kafka, Spark Structured Streaming, and Databricks Streaming
- Extensive experience applying and enforcing Medallion architecture (Bronze, Silver, Gold layers) within Databricks
- Experience designing and implementing CI/CD pipelines using tools such as Azure DevOps, GitHub Actions, or GitLab CI for Databricks workflows, notebooks, and cluster configurations
- Experience planning and executing data migration projects from traditional data warehouses into the Databricks Lakehouse
- Strong working knowledge of at least one major cloud provider, AWS or Azure, including data storage, networking, and security concepts relevant to Databricks deployment
- Ability to engage with clients, present technical solutions, and communicate complex ideas clearly
- Bachelor’s or Master’s degree in Computer Science, Information Systems, Engineering, or a related field
- Excellent problem-solving, communication, and collaboration skills
Benefits
- Supportive, driven, collaborative team environment
- Great Place to Work recognition
- Opportunities to grow skills and deepen expertise
- Meaningful work solving real-world business challenges
- Career exposure to AI and automation consulting, cloud, data, and business application technologies
- Full-time employment
Job title
Job type
Experience level
Salary
Not specified
Degree requirement
Tech skills
Location requirements
Report this job
Found something wrong with the page? Please let us know by submitting a report below.
Senior Data Engineer designing scalable Databricks Lakehouse architectures for Bits In Glass, an AI and automation consulting firm. Building streaming pipelines, modernizing data platforms, and advising clients.
Data Engineer leading ERP - to - MindBridge ETL and financial data pipelines. Supporting clients and product teams for MindBridge’s audit analytics platform.
Azure platform administrator operating Desjardins’ data platform for data - driven financial services. Managing IaC, deployments, security, and shared production infrastructure.
Senior Data Engineer building trusted Snowflake data products for Granum’s landscaper and arborist software ecosystem. Operating governed pipelines that power analytics, AI, and operational decisions.
Data Engineer building Azure ETL/ELT pipelines and governed data architecture for Manulife, an international financial services provider. Improving data integration, automation, and platform efficiency.
Data Engineer building Azure ETL/ELT pipelines and governed data architecture for Manulife’s financial services business. Supporting scalable analytics, automation, and operational excellence.
Data Engineer building ETL pipelines and analytics infrastructure for CAE’s simulation and training solutions. Supporting data quality, modelling, dashboards, and reporting across multidisciplinary teams.
Senior Data Engineer building scalable pipelines and AI - ready infrastructure for CINC Systems’ community association management software. Enabling reliable analytics, automation, and intelligent product experiences.
Senior Data Engineer building AWS, Snowflake, Python, PySpark, and SQL data products for Sun Life’s Canadian financial - services business. Enabling analytics, data science, governance, and client insights.