Azure Data Engineer-Databricks /ADF

Posted via LinkedIn Recruiter (not a company profile)

Posted 2 hours ago

Apply Now

Resume Score

Check how well your resume matches this job before you apply.

Sign in to check score

About the role

  • Azure Data Engineer needed for a financial client to develop pipelines using ADF, Databricks, and Azure Functions, and migrate to Unity Catalog.

Responsibilities

  • Develop and maintain pipelines using Azure Data Factory (ADF), Databricks, and Azure Functions (C#). Build and optimize Spark/Pandas notebooks. Configure and manage Unity Catalog for data governance. Migrate Databricks workspace from hive metastore to Unity Catalog. Design delta tables with security controls. Implement Delta Live Tables (nice to have). Manage DevOps using Azure DevOps and GitHub Enterprise. Work with Azure Entra ID for SSO/RBAC. Deploy and manage Azure PaaS resources. Collaborate with team and lead technical sessions. Ensure compliance with data governance and security standards.

Requirements

  • 7+ years with ADF, Databricks, Azure Functions (C#). Extensive Python/notebooks on Databricks/Spark/Pandas. Expert knowledge of Unity Catalog. Experience with Medallion architecture. Proven track record migrating Databricks to Unity Catalog. Ability to design delta tables. Strong DevOps skills. In-depth Azure Entra ID experience. Proficiency with Azure SQL PaaS, ADLS, Synapse. Familiarity with highly regulated environments. Ability to work independently and lead a team. Financial services experience (asset). Excellent communication skills.

Job title

Job type

Contractor

Experience level

Senior

Salary

$86.96/hr – $104.35/hr

Degree requirement

No Education Requirement

Tech skills

Azure Data FactoryDatabricksAzure FunctionsC#SparkPandasPythonUnity CatalogDelta Live TablesAzure DevOpsGitHub EnterpriseAzure Entra IDAzure SQLADLSSynapse

Location requirements

Linkedin Recruiter PostTorontoOntario Toronto

Report this job

Found something wrong with the page? Please let us know by submitting a report below.