Senior Data Engineer designing scalable Databricks Lakehouse architectures for Bits In Glass, an AI and automation consulting firm. Building governed pipelines, streaming solutions, and cloud data platforms for client modernization.
About the role
- Senior Data Engineer designing scalable Databricks lakehouse architectures for Bits In Glass, an AI and automation consulting firm. Leading pipelines, governance, cloud modernization, and client data solutions.
Responsibilities
- Design and implement end-to-end data architectures, including data lakes, data warehouses, and analytics platforms
- Define data integration and transformation strategies for scalability, security, and performance
- Translate business requirements into technical solutions supporting analytics, reporting, and AI initiatives
- Develop data models, ETL/ELT pipelines, and frameworks for structured and unstructured data
- Provide technical leadership and mentorship to data engineers and developers
- Promote best practices in data management and governance
- Ensure compliance with data governance, security, and privacy standards across platforms
- Optimize existing data architectures and processes for performance and reliability
- Stay current with cloud data services and emerging technologies
- Act as a trusted advisor to clients on architecture decisions and data modernization best practices
- Collaborate with business stakeholders, data engineers, and analytics teams to align solutions with client goals
Requirements
- 5+ years of experience in data architecture, data engineering, or analytics solution design
- Hands-on experience with data lake and warehouse technologies, such as Databricks, Snowflake, Redshift, and Synapse
- Deep understanding of data modeling, data integration, and ETL/ELT design
- Proficiency in SQL and one or more programming languages, including PySpark/Python or Scala
- Experience with complex data transformations and optimization within Spark
- Understanding of data governance, security, and privacy best practices
- Experience designing, implementing, and optimizing large-scale ingestion pipelines using Databricks Autoloader
- Practical knowledge of building and managing reliable, self-managing ETL/ELT pipelines using Delta Live Tables
- Experience building high-throughput, low-latency streaming data ingestion solutions using Apache Kafka, Spark Structured Streaming, and Databricks Streaming
- Extensive experience applying and enforcing Medallion architecture within a Databricks environment
- Experience designing and implementing CI/CD pipelines using tools such as Azure DevOps, GitHub Actions, or GitLab CI for Databricks workflows, notebooks, and cluster configurations
- Experience planning and executing data migration projects from traditional data warehouses into the Databricks Lakehouse
- Strong working knowledge of at least one major cloud provider, AWS or Azure, including data storage, networking, and security concepts relevant to Databricks deployment
- Ability to engage with clients, present technical solutions, and communicate complex ideas clearly
- Bachelor’s or Master’s degree in Computer Science, Information Systems, Engineering, or a related field
- Excellent problem-solving, communication, and collaboration skills
- Nice to have: AI/ML integration and data science workflows
- Nice to have: data cataloging and metadata management tools
- Nice to have: prior consulting or client-facing experience in a technology services firm
Benefits
- Supportive, collaborative team environment
- Great Place to Work recognition
- Opportunities to deepen expertise and grow skills
- Meaningful work solving real-world business challenges
- Professional development through exposure to modern platforms, cloud services, AI, and automation consulting
- Full-time employment
Job title
Job type
Experience level
Salary
Not specified
Degree requirement
Tech skills
Location requirements
Report this job
Found something wrong with the page? Please let us know by submitting a report below.
Senior Data Engineer designing scalable Databricks Lakehouse architectures for Bits In Glass, an AI and automation consulting firm. Building streaming pipelines, modernizing data platforms, and advising clients.
Data Engineer leading ERP - to - MindBridge ETL and financial data pipelines. Supporting clients and product teams for MindBridge’s audit analytics platform.
Azure platform administrator operating Desjardins’ data platform for data - driven financial services. Managing IaC, deployments, security, and shared production infrastructure.
Senior Data Engineer building trusted Snowflake data products for Granum’s landscaper and arborist software ecosystem. Operating governed pipelines that power analytics, AI, and operational decisions.
Data Engineer building Azure ETL/ELT pipelines and governed data architecture for Manulife, an international financial services provider. Improving data integration, automation, and platform efficiency.
Data Engineer building Azure ETL/ELT pipelines and governed data architecture for Manulife’s financial services business. Supporting scalable analytics, automation, and operational excellence.
Data Engineer building ETL pipelines and analytics infrastructure for CAE’s simulation and training solutions. Supporting data quality, modelling, dashboards, and reporting across multidisciplinary teams.
Senior Data Engineer building scalable pipelines and AI - ready infrastructure for CINC Systems’ community association management software. Enabling reliable analytics, automation, and intelligent product experiences.
Senior Data Engineer building AWS, Snowflake, Python, PySpark, and SQL data products for Sun Life’s Canadian financial - services business. Enabling analytics, data science, governance, and client insights.