Data Engineer responsible for designing and maintaining enterprise data warehouse for various projects. Collaborating with stakeholders to ensure efficient data flow and integration.
Responsibilities
Design, develop, and maintain scalable Synapse pipelines for data ingestion, processing, and transformation in Azure data platform.
Develop and optimize T-SQL and Python-based data processing scripts and integrate them with Synapse pipelines for ETL tasks and data transformations.
Design and develop mapping dataflow solutions for complex data transformations.
Develop and maintain data storage solutions, ensuring efficient data retrieval and storage on Azure Data Lake and databases.
Implement and maintain API integrations, including REST APIs, for seamless data flow between systems.
Design and test the end-to-end process for data ingestion, data transformation, data cleansing, data delivery, data quality and data timeliness.
Implement and ensure data security and privacy measures, complying with industry standards and regulations.
Monitor and optimize the performance of Azure data services, Azure Synapse pipelines, Function Apps, and Logic Apps.
Improve the scalability, efficiency, and cost-effectiveness of data pipelines.
Requirements
Bachelor's degree or equivalent in the field of Computer Science or Engineering
3+ years of Proven experience as a Data Engineer, specifically working with Microsoft Azure technologies.
Proficient in T-SQL, Pyspark, Mapping Dataflow and Python programming language
Hands-on experience with Azure data services, including Azure Data Factory, Azure Data Lake, Azure Function App, Azure Logic App, Azure SQL Database, Azure Synapse Analytics, mapping dataflow, and Event Hub.
Understanding of data warehousing concepts and best practices.
Experience with Data modeling.
Experience in working with structured, semi-structured and unstructured data.
Aarorn Technologies seeks a Datastage ETL contractor with 5+ years of experience for a hybrid role in Toronto. Must have IBM InfoSphere DataStage, SQL, Unix scripting, and scheduling skills.
Azure Databricks Developer needed for HR tech projects at a top financial client. Design ETL solutions, support data integrations, and modernize HR data platforms.
Senior Software Engineer, Data responsible for building data pipelines and maintaining infrastructure for analytics and ML. Collaborate with teams to optimize data systems at Narvar.
Data Architect leading conversations and decisions on data modeling at a global consulting firm specializing in Data & Analytics. Transforming complex processes into clear, reusable information models.
Senior Data Engineer at Supabase managing data pipelines from source to analysis. Collaborating with growth, finance, and product teams to drive business insights.
Associate Data Architect collaborating with stakeholders to design scalable data architecture for AXIS Capital. Maximizing profit through data - driven decision making and implementing effective data governance strategies.
Data Engineer II at Finning supporting production data platforms and integrations using Snowflake, Databricks, Azure services. Collaborating with teams to ensure operational stability and reliability.