Senior Data Engineer leading scalable data infrastructure design at Spellbook. Collaborating with distributed teams to transform contract data into actionable insights.
LP
Data Engineer (Azure Data Factory & Databricks Focus)
Posted via LinkedIn Recruiter (not a company profile)
Posted last month
About the role
- Data Engineer role focusing on Azure Data Factory & Databricks for building data pipelines. 11-month contract, hybrid work in Oshawa, $90-$95/hour.
Responsibilities
- Design, develop, and productionize modular, scalable ELT/ETL pipelines using Azure Data Factory and Databricks. Build and maintain data lake and data warehouse solutions supporting analytics, applications, and innovation. Cleanse, transform, and optimize large datasets using Python, PySpark, Spark SQL, and SQL. Develop curated, business-centric common data models in collaboration with Data Architects and Data Modelers. Implement data quality, lineage, and governance controls throughout the data lifecycle. Ingest data from multiple sources including CSV, JSON, XML, REST APIs, and enterprise systems. Optimize pipeline performance, reliability, and scalability. Troubleshoot data ingestion, transformation, latency, accuracy, and integrity issues. Collaborate with Business Analysts, Data Scientists, Senior Data Engineers, and Architects. Support BI and analytics use cases, including dimensional modeling and aggregation optimization. Participate in CI/CD pipelines, DevOps workflows, and automated testing strategies. Contribute to metadata management and data cataloging. Provide Tier-2 support for production data pipelines and datasets. Participate in peer code reviews and agile SCRUM ceremonies.
Requirements
- Proven experience building new data pipelines using Azure Data Factory (ADF) for orchestration and Azure Databricks (Spark/PySpark) for transformations. Hands-on expertise with Azure Data Factory and Azure Databricks. Strong programming skills in Python, PySpark, Spark SQL, and SQL. Experience building data lakehouse and data warehouse pipelines. Strong understanding of data structures, data integration patterns, and processing frameworks. Knowledge of data governance, security, and data quality principles. Experience working in an Agile/SCRUM environment. Bachelor's degree in Computer Science, Software Engineering, or related field.
Job title
Job type
Experience level
Salary
Degree requirement
Tech skills
Location requirements
Report this job
Found something wrong with the page? Please let us know by submitting a report below.
Data Engineer role focused on building and optimizing data pipelines within Microsoft Fabric at AIM. Collaborating closely with analysts and developers to deliver scalable data solutions.
Senior Software Engineer focusing on data engineering and machine learning for Workday. Involves data ingestion, governance, and collaboration using modern technologies.
LP
Data Warehouse Tester / Developer
LinkedIn Recruiter Post
Hiring Data Warehouse Tester/Developer in Toronto, ON (Hybrid). Requires 6 - 8 years exp in Databricks, PySpark, Python, SQL, Azure, ADF, DevOps, Informatica.
Lead Data Engineering and Analytics team to oversee data infrastructure design and governance for robotic deliveries. Collaborate cross - functionally to enable data - driven decision - making across departments.
Senior Data Architect delivering enterprise - scale data and analytics solutions at 3Pillar. Leading design and delivery of analytics platforms in mixed prem and cloud environments.
Cloud Data Engineer responsible for modern Data & AI solutions on Microsoft Azure. Collaborating with clients and teams to develop production - ready data platforms and support analytics.
Senior Data Engineer at Solana Foundation collaborating with blockchain engineers on data indexing and pipeline creation. Ensuring efficient data processing and metrics formulation for decentralized applications.
Senior Engineer on Data Platform team designing and building systems for data flow at Movable Ink. Collaborating with engineering, analytics, and infrastructure teams to power data ingestion and processing.
Senior Data Engineer responsible for designing and maintaining event streaming pipelines at Movable Ink. Working with modern technologies to enhance data availability and reliability.