Sr. Data Engineer needed to lead Databricks-based data platform modernization for a top financial client, migrating trading and market data from legacy systems.
Responsibilities
Design and develop data ingestion pipelines (batch and real-time) leveraging Databricks. Build and optimize Spark/Scala-based data processing workflows. Implement streaming ingestion using Kafka or APIs. Develop medallion architecture models (bronze, silver, gold). Create and maintain materialized views. Ensure data quality through validation, deduplication, and reconciliation. Contribute to reusable frameworks and collaborate with internal teams to deliver scalable solutions. Support BAU tasks and platform buildout.
Requirements
Extensive hands-on experience with Databricks as SME. Advanced proficiency in Apache Spark and Scala. Proven ability to develop data pipelines (batch and streaming). Knowledge of real-time ingestion frameworks such as Kafka or APIs. Strong understanding of data architecture, pipeline design, and data modeling best practices. Experience with data quality, validation, and reconciliation workflows. Ability to interpret complex technical requirements. Effective communication skills. Delivery-focused mindset.
Senior Product Manager overseeing data platform and intelligence capabilities at Wattpad. Collaborating with cross - functional teams to build systems for smarter product decisions.
Hands - on Manager, Data Architecture leading data analytics and data management for Optiom, a Canadian specialty insurer. Building a governed cloud data platform and analytics function.
Senior Backend Engineer enhancing health data platform at Molecular You. Owning platform quality, driving technology adoption, and ensuring security compliance.
AVP, Enterprise Data Architecture at Sun Life, managing enterprise data architecture and strategy. Collaborating with cross - functional teams to drive client - centric solutions leveraging data and analytics.
Data Architect designing enterprise - level data storage solutions for Servus Credit Union. Collaborating with teams to enhance data practices supporting the organization's goals.
Data Migration Engineer at gaiia owning the technical execution of data migrations for telecom service providers. Focusing on transforming customer data from legacy systems into gaiia's platform.
Product Manager responsible for the Data Platform at SandboxAQ, enhancing scientific discovery capabilities. Leading product management for data lifecycle in advanced materials and drug discovery.
Associate Data Engineer working with a global team to build reliable data pipelines. Collaborating on meaningful projects that support real business decisions within a hybrid work environment.
Commissioning Program & Data Engineer working on controlled, data - driven commissioning programs at Bird. Responsible for tracking tests, issue logs, and analyzing commissioning data.