Senior Data Engineer improving General Motors vehicle data pipelines, monitoring, and observability. Supporting reliable fleet software operations across Oshawa and Markham hybrid offices.
LP
Data Engineer (Azure Data Factory & Databricks Focus)
Posted via LinkedIn Recruiter (not a company profile)
Posted 6 months ago
About the role
- Data Engineer role focusing on Azure Data Factory & Databricks for building data pipelines. 11-month contract, hybrid work in Oshawa, $90-$95/hour.
Responsibilities
- Design, develop, and productionize modular, scalable ELT/ETL pipelines using Azure Data Factory and Databricks. Build and maintain data lake and data warehouse solutions supporting analytics, applications, and innovation. Cleanse, transform, and optimize large datasets using Python, PySpark, Spark SQL, and SQL. Develop curated, business-centric common data models in collaboration with Data Architects and Data Modelers. Implement data quality, lineage, and governance controls throughout the data lifecycle. Ingest data from multiple sources including CSV, JSON, XML, REST APIs, and enterprise systems. Optimize pipeline performance, reliability, and scalability. Troubleshoot data ingestion, transformation, latency, accuracy, and integrity issues. Collaborate with Business Analysts, Data Scientists, Senior Data Engineers, and Architects. Support BI and analytics use cases, including dimensional modeling and aggregation optimization. Participate in CI/CD pipelines, DevOps workflows, and automated testing strategies. Contribute to metadata management and data cataloging. Provide Tier-2 support for production data pipelines and datasets. Participate in peer code reviews and agile SCRUM ceremonies.
Requirements
- Proven experience building new data pipelines using Azure Data Factory (ADF) for orchestration and Azure Databricks (Spark/PySpark) for transformations. Hands-on expertise with Azure Data Factory and Azure Databricks. Strong programming skills in Python, PySpark, Spark SQL, and SQL. Experience building data lakehouse and data warehouse pipelines. Strong understanding of data structures, data integration patterns, and processing frameworks. Knowledge of data governance, security, and data quality principles. Experience working in an Agile/SCRUM environment. Bachelor's degree in Computer Science, Software Engineering, or related field.
Job title
Job type
Experience level
Salary
Degree requirement
Tech skills
Location requirements
Report this job
Found something wrong with the page? Please let us know by submitting a report below.
Senior software engineer building General Motors’ vehicle data platform across ECUs and cloud. Developing control, data, logging, and SDK software for connected vehicles.
Senior Data Architect integrating Alberta’s environmental regulatory, Databricks, and geospatial platforms. Designing governed data flows for regulators, decision - makers, and Albertans.
Senior Data Architect leading secure, scalable data architecture for Manulife’s financial services organization. Designing cloud, lakehouse, analytics, machine learning, and generative AI data platforms.
Data engineering advisor developing scalable data platforms and pipelines for Desjardins. Designing, integrating, and improving data systems aligned with business needs.
Senior Data Engineer owning NimbleRx’s compliant healthcare data platform. Building scalable pipelines, warehouse models, and AI tooling for pharmacy and patient engagement operations.
Administrateur principal de plateformes de données chez EDC, société d’État aidant les entreprises canadiennes à réussir à l’étranger. Conception, exploitation et gouvernance de plateformes de données infonuagiques.
Senior Data Platform Administrator shaping and operating EDC’s secure enterprise data platforms. Supporting Canadian businesses through Export Development Canada’s trade - finance solutions.
Staff full - stack engineer architecting GitLab’s data products, integrations, and knowledge graph. Building APIs, marketplace data access, and AI - powered developer intelligence.
Data Governance Architect shaping governance strategies and cloud data architectures for Lovelytics’ enterprise data and AI consulting clients. Leading technical delivery, presales, migrations, and governance implementations on Databricks.