Data Engineer building PySpark and Databricks pipelines for Electric Mind’s financial services and wealth management clients. Debugging transformations, validating data, and supporting Azure-based environments.
Responsibilities
Build and maintain data pipelines on Databricks for a financial services/wealth management data platform
Write and maintain hand-written PySpark transforms for silver and gold layer tables, including customer, account, and transaction data models
Work with Lakeflow Declarative Pipelines and Auto CDC / SCD Type 2 patterns for change-data-capture merges
Query and validate data in Unity Catalog across dev/qa/uat using SQL warehouses
Debug failed pipeline runs by reading event logs, tracing bad records, and fixing schema drift or data quality issues
Maintain reference/lookup tables and PowerShell/Python utility scripts
Write and update unit/integration tests for transform logic
Participate in GitLab code reviews using feature branches and merge requests
Keep technical documentation current
Work alongside a senior engineer and help maintain healthy Databricks environments
Full Stack Data Engineer migrating legacy insurance licensing and compensation systems. Building ETL pipelines, data models, reconciliations, reporting, and downstream feeds for Manulife.
Senior Data Engineer building Azure Synapse big data pipelines for Quantiphi, an AI - first digital engineering company. Supporting healthcare cloud migrations, large - scale processing, and data platforms.
Senior Data Engineer building Azure, Databricks, and Snowflake pipelines for 73 Strings’ private - capital data platform. Owning ingestion, transformation, quality, CI/CD, and production delivery.
Senior Developer building C#, SQL, and cloud tooling to migrate legal, billing, and trust - accounting data. Supporting complex client migrations for a technology company serving financial services firms.
Data Migration Lead moving legal, billing, and trust accounting data into a legal technology platform. Leading analysts, conversion tooling, customer migrations, and compliance controls.
Senior Software Engineer building scalable AWS microservices and APIs for Morningstar's global investment data platform. Modernizing distributed systems and enabling reliable financial data delivery.
Data infrastructure engineer building searchable robotics datasets and quality pipelines at Mecka AI. Managing distributed robot fleet infrastructure, deployments, monitoring, governance, and cloud costs.
Senior Data Migration Engineer leading PACS, RIS, VNA, and cloud migrations for INFINITT’s healthcare IT business. Ensuring secure data integrity across clinical imaging environments.
Data Engineering Advisor building Azure data pipelines and analytics solutions for Desjardins. Designing ingestion, transformation, and data management systems for business needs.
Staff Software Engineer building Instacart’s advertising data platform for grocery delivery and household goods. Designing compliant, scalable systems for advertisers, data scientists, and sales teams.