Senior Data Engineer designing scalable Databricks Lakehouse architectures for Bits In Glass, an AI and automation consulting firm. Building governed pipelines, streaming solutions, and cloud data platforms for client modernization.
Responsibilities
Design and implement end-to-end data architectures, including data lakes, data warehouses, and analytics platforms
Define scalable, secure, and high-performance data integration and transformation strategies
Collaborate with business stakeholders, data engineers, and analytics teams to translate requirements into technical solutions
Develop data models, ETL/ELT pipelines, and frameworks for structured and unstructured data
Provide technical leadership and mentorship to data engineers and developers
Ensure compliance with data governance, security, and privacy standards
Optimize existing data architectures and processes for performance and reliability
Stay current with cloud data services and emerging technologies
Act as a trusted advisor to clients on architecture decisions and data modernization best practices
Requirements
5+ years of experience in data architecture, data engineering, or analytics solution design
Hands-on experience with data lake and warehouse technologies, such as Databricks, Snowflake, Redshift, and Synapse
Deep understanding of data modeling, data integration, and ETL/ELT design
Proficiency in SQL and one or more programming languages: PySpark/Python or Scala
Experience with complex data transformations and optimization within Spark
Solid understanding of data governance, security, and privacy best practices
Proven experience designing, implementing, and optimizing large-scale ingestion pipelines using Databricks Autoloader
Deep practical knowledge of building and managing reliable, self-managing ETL/ELT pipelines using Delta Live Tables
Experience building high-throughput, low-latency streaming data ingestion solutions using Apache Kafka, Spark Structured Streaming, and Databricks Streaming
Extensive experience applying and enforcing Medallion architecture (Bronze, Silver, Gold layers) within Databricks
Experience designing and implementing CI/CD pipelines using tools such as Azure DevOps, GitHub Actions, or GitLab CI for Databricks workflows, notebooks, and cluster configurations
Experience planning and executing data migration projects from traditional data warehouses into the Databricks Lakehouse
Strong working knowledge of at least one major cloud provider, AWS or Azure, including data storage, networking, and security concepts relevant to Databricks deployment
Ability to engage with clients, present technical solutions, and communicate complex ideas clearly
Bachelor’s or Master’s degree in Computer Science, Information Systems, Engineering, or a related field
Excellent problem-solving, communication, and collaboration skills
Benefits
Supportive, driven, collaborative team environment
Great Place to Work recognition
Opportunities to grow skills and deepen expertise
Meaningful work solving real-world business challenges
Career exposure to AI and automation consulting, cloud, data, and business application technologies
Data Platform & Analytics Capability Lead building, positioning, and scaling Avanade’s enterprise data and analytics capability in Canada. Shaping client solutions, opportunities, partnerships, and delivery standards.
Senior Data Engineer scaling blockchain, product analytics, and experimentation infrastructure for Phantom’s global open - markets trading app. Driving data quality and architecture across a remote team.
Data engineering intern supporting ETL, data quality, and transformation projects at Canadian asset manager PICTON Investments. Collaborating across business teams using SQL, Python, databases, and cloud platforms.
Data engineering student building Azure, SQL, Python, and Power BI solutions for CMHA Peel Dufferin’s mental health services. Supporting reporting, ETL pipelines, dashboards, automation, and data - driven decisions.
Senior Product Architect guiding AWS, SaaS, data and financial technology at FCC, a Canadian agriculture lender. Shaping secure enterprise roadmaps and integrations across financial applications.
Data Engineer designing warehouse models, Python pipelines, and Power BI reporting for Silk & Snow’s sleep products. Transforming raw data into actionable business insights.
Senior Data Engineer building SQL, dbt, and Airflow data pipelines for Penn Interactive’s online gaming and sports platforms. Delivering trusted analytics and recommendations to business stakeholders.
Data Architect governing World Vision Canada's nonprofit enterprise data architecture. Defining cloud data standards, governance, security and reusable platforms remotely within Canada.
Senior software engineer building backend data services, APIs, microservices, and pipelines for Akamai’s distributed cloud platform. Optimizing databases and reliability across high - scale workloads.
Principal Data Architect defining scalable, governed data platforms for Robots & Pencils’ enterprise AI systems. Setting architecture direction, supporting clients, and mentoring senior technical teams.