Senior Data Engineer designing scalable Databricks lakehouse architectures for Bits In Glass, an AI and automation consulting firm. Leading pipelines, governance, cloud modernization, and client data solutions.
Responsibilities
Design and implement end-to-end data architectures, including data lakes, data warehouses, and analytics platforms
Define data integration and transformation strategies for scalability, security, and performance
Translate business requirements into technical solutions supporting analytics, reporting, and AI initiatives
Develop data models, ETL/ELT pipelines, and frameworks for structured and unstructured data
Provide technical leadership and mentorship to data engineers and developers
Promote best practices in data management and governance
Ensure compliance with data governance, security, and privacy standards across platforms
Optimize existing data architectures and processes for performance and reliability
Stay current with cloud data services and emerging technologies
Act as a trusted advisor to clients on architecture decisions and data modernization best practices
Collaborate with business stakeholders, data engineers, and analytics teams to align solutions with client goals
Requirements
5+ years of experience in data architecture, data engineering, or analytics solution design
Hands-on experience with data lake and warehouse technologies, such as Databricks, Snowflake, Redshift, and Synapse
Deep understanding of data modeling, data integration, and ETL/ELT design
Proficiency in SQL and one or more programming languages, including PySpark/Python or Scala
Experience with complex data transformations and optimization within Spark
Understanding of data governance, security, and privacy best practices
Experience designing, implementing, and optimizing large-scale ingestion pipelines using Databricks Autoloader
Practical knowledge of building and managing reliable, self-managing ETL/ELT pipelines using Delta Live Tables
Experience building high-throughput, low-latency streaming data ingestion solutions using Apache Kafka, Spark Structured Streaming, and Databricks Streaming
Extensive experience applying and enforcing Medallion architecture within a Databricks environment
Experience designing and implementing CI/CD pipelines using tools such as Azure DevOps, GitHub Actions, or GitLab CI for Databricks workflows, notebooks, and cluster configurations
Experience planning and executing data migration projects from traditional data warehouses into the Databricks Lakehouse
Strong working knowledge of at least one major cloud provider, AWS or Azure, including data storage, networking, and security concepts relevant to Databricks deployment
Ability to engage with clients, present technical solutions, and communicate complex ideas clearly
Bachelor’s or Master’s degree in Computer Science, Information Systems, Engineering, or a related field
Excellent problem-solving, communication, and collaboration skills
Nice to have: AI/ML integration and data science workflows
Nice to have: data cataloging and metadata management tools
Nice to have: prior consulting or client-facing experience in a technology services firm
Benefits
Supportive, collaborative team environment
Great Place to Work recognition
Opportunities to deepen expertise and grow skills
Meaningful work solving real-world business challenges
Professional development through exposure to modern platforms, cloud services, AI, and automation consulting
Data Platform & Analytics Capability Lead building, positioning, and scaling Avanade’s enterprise data and analytics capability in Canada. Shaping client solutions, opportunities, partnerships, and delivery standards.
Senior Data Engineer scaling blockchain, product analytics, and experimentation infrastructure for Phantom’s global open - markets trading app. Driving data quality and architecture across a remote team.
Data engineering intern supporting ETL, data quality, and transformation projects at Canadian asset manager PICTON Investments. Collaborating across business teams using SQL, Python, databases, and cloud platforms.
Data engineering student building Azure, SQL, Python, and Power BI solutions for CMHA Peel Dufferin’s mental health services. Supporting reporting, ETL pipelines, dashboards, automation, and data - driven decisions.
Senior Product Architect guiding AWS, SaaS, data and financial technology at FCC, a Canadian agriculture lender. Shaping secure enterprise roadmaps and integrations across financial applications.
Data Engineer designing warehouse models, Python pipelines, and Power BI reporting for Silk & Snow’s sleep products. Transforming raw data into actionable business insights.
Senior Data Engineer building SQL, dbt, and Airflow data pipelines for Penn Interactive’s online gaming and sports platforms. Delivering trusted analytics and recommendations to business stakeholders.
Data Architect governing World Vision Canada's nonprofit enterprise data architecture. Defining cloud data standards, governance, security and reusable platforms remotely within Canada.
Senior software engineer building backend data services, APIs, microservices, and pipelines for Akamai’s distributed cloud platform. Optimizing databases and reliability across high - scale workloads.
Principal Data Architect defining scalable, governed data platforms for Robots & Pencils’ enterprise AI systems. Setting architecture direction, supporting clients, and mentoring senior technical teams.