Senior Data Engineer building Azure Synapse big data pipelines for Quantiphi, an AI-first digital engineering company. Supporting healthcare cloud migrations, large-scale processing, and data platforms.
Responsibilities
Design and deliver robust big data pipelines for structured and unstructured data across multiple geographies
Help healthcare organizations achieve business goals through advanced data ingestion technologies, cloud services, and DevOps practices
Collaborate with Cloud Engineering, Software Engineering, and ML Engineering architects
Create platforms, solutions, and applications addressing healthcare trends such as digital diagnosis, software as a medical product (SaMD), and AI marketplaces
Develop migration, storage, and processing utilities using Azure Synapse Analytics
Design and build cloud migration strategies for cloud and on-premise applications
Diagnose and troubleshoot distributed systems problems within the Azure ecosystem
Build tools to ingest and process several terabytes or petabytes of data per day
Optimize performance and cost within Azure Synapse
Design and develop storage and compute solutions for large customers using Azure Synapse
Collaborate with Infrastructure, Network, Engineering, DevOps, SiteOps, and cloud customer teams
Build automation, testing, monitoring, administration, and data operations tooling across Azure Synapse workspaces and cloud clusters
Lead technology teams and mentor junior members
Requirements
6+ years of experience (Experience Level)
3+ years of hands-on Data Engineering experience
Strong knowledge of data structures, distributed systems, data modeling, and databases
Strong proficiency in SQL and hands-on experience with Python/PySpark
Extensive experience developing Big Data solutions for migration, storage, and processing
Proven track record implementing solutions on cloud platforms
Experience building and supporting large-scale systems in a production environment
Strong experience developing ETL/ELT pipelines using Azure Data Factory (ADF) and Synapse Pipelines
Deep expertise in Azure, specifically Azure Synapse Analytics, including SQL Pools, Spark Pools, Data Explorer, and Pipelines
Experience with Azure Data Lake Storage (ADLS Gen2)
Strong experience with Apache Spark, especially Azure Synapse Spark Pools
Experience with Azure Synapse SQL Pools, Azure Cosmos DB, Azure Data Explorer, and other Azure data services
Experience with Azure Data Factory, Apache Airflow, or similar orchestration frameworks
Experience with Azure Event Hubs, Azure Service Bus, or Kafka
Ability to lead technology teams and provide mentorship/support
Experience leading multiple large projects
Deep understanding of Agile development
Effective communication with technical and non-technical stakeholders
Experience working with insurance data/domain is good to have
Experience with Apache Airflow, Databricks, dbt, Microsoft Fabric, or Snowflake is good to have
Experience with Kubernetes/AKS is good to have
Knowledge of Terraform/Infrastructure as Code and Azure DevOps is good to have
Experience with data governance, data quality, lineage, and metadata management is good to have
Team management and mentoring
Ability to mentor and guide junior team members
Ability to communicate complex subjects and technical plans clearly
Ability to write clear reports accessible to varied audiences
Benefits
Upskilling and professional development opportunities
Exposure to cutting-edge AI, ML, data, and cloud technologies
Opportunities to work with Fortune 500 companies
Work in a research-focused organization with 60+ patents filed
Collaborate with passionate, talented colleagues
Opportunities to make an impact at a fast-growing AI-first digital engineering company
Senior Data Engineer building and migrating enterprise data platforms on Google Cloud. Designing BigQuery pipelines and architectures for consulting clients through production hand - off.
Full Stack Data Engineer migrating legacy insurance licensing and compensation systems. Building ETL pipelines, data models, reconciliations, reporting, and downstream feeds for Manulife.
Senior Data Engineer building Azure, Databricks, and Snowflake pipelines for 73 Strings’ private - capital data platform. Owning ingestion, transformation, quality, CI/CD, and production delivery.
Senior Developer building C#, SQL, and cloud tooling to migrate legal, billing, and trust - accounting data. Supporting complex client migrations for a technology company serving financial services firms.
Data Migration Lead moving legal, billing, and trust accounting data into a legal technology platform. Leading analysts, conversion tooling, customer migrations, and compliance controls.
Senior Software Engineer building scalable AWS microservices and APIs for Morningstar's global investment data platform. Modernizing distributed systems and enabling reliable financial data delivery.
Data infrastructure engineer building searchable robotics datasets and quality pipelines at Mecka AI. Managing distributed robot fleet infrastructure, deployments, monitoring, governance, and cloud costs.
Senior Data Migration Engineer leading PACS, RIS, VNA, and cloud migrations for INFINITT’s healthcare IT business. Ensuring secure data integrity across clinical imaging environments.
Data Engineering Advisor building Azure data pipelines and analytics solutions for Desjardins. Designing ingestion, transformation, and data management systems for business needs.
Staff Software Engineer building Instacart’s advertising data platform for grocery delivery and household goods. Designing compliant, scalable systems for advertisers, data scientists, and sales teams.