Lead Data Engineer designing and optimizing AI - powered data platforms at Brillio. Collaborating with experts to build intelligent workflows and ensuring data security.
LP
Azure Databricks Architect
Posted via LinkedIn Recruiter (not a company profile)
Posted 4 months ago
About the role
- Azure Databricks Architect designing Azure data solutions with Databricks, Data Factory, Synapse Analytics. Builds Medallion architecture, data pipelines, AI/ML workflows, and serves as technical authority.
Responsibilities
- Platform Architecture Designing and implementing Azure based data solutions using Databricks Data Factory Synapse Analytics and Data Lake services. Design build and optimize a Medallion architecture Bronze Silver Gold supporting both HR and enterprise data domains. Develop data pipelines APIs and orchestration frameworks that enable seamless data ingestion transformation and delivery. Establish and maintain data governance metadata and lineage frameworks using Databricks Unity Catalog and Azure Purview. Build and maintain ETLELT frameworks data APIs and Realtime workflows connecting diverse HR and enterprise systems. Leverage AI and ML capabilities within Databricks including MLflow feature stores and the Agent Framework Agent Bricks to build intelligent data agents for automation monitoring and analytics enrichment. Design and implement AI agent pipelines that can reason over HR data models automate transformations and enhance analytical outcomes. Integrate CICD pipelines and infrastructureascode approaches Terraform Bicep or Azure DevOps. Serve as the technical authority for Azure and Databricks architecture within HR Technology. Partner with HR Analytics and Infrastructure leaders to define and execute the data and AI modernization roadmap. Establish standards patterns and reusable frameworks for data engineering AI agents and automation. Mentor engineers and champion a culture of innovation experimentation and delivery excellence.
Requirements
- Education bachelor’s or master’s degree in computer science information systems or related field. Experience Minimum 12 years of hands-on experience architecting and delivering largescale data platforms using Azure and Databricks. Proven success in deploying Medallion architectures governed data pipelines and Ai driven data workflows Expertise in Azure Data Lake Data Factory and Synapse Analytics. Databricks Delta Lake Unity Catalog Spark optimization. Python SQL PySpark and API development REST GraphQL. Infrastructure as code Terraform Bicep and DevOps automation. AIML frameworks within Databricks including Agent Bricks and MLflow. Strong grounding in data governance security and compliance best practices. Excellent communication and collaboration skills ability to translate complex architecture into business outcomes. Knowledge of Azure Purview Power BI and data cataloging tools. Experience implementing Databricks Unity Catalog Delta Live Tables and AI agent orchestration within enterprise contexts. Familiarity with HR data models and effective design for analytics and workforce intelligence. Understanding of LLM integration and AI agent frameworks for data enrichment and automation.
Job title
Job type
Experience level
Salary
Not specified
Degree requirement
Tech skills
Location requirements
Report this job
Found something wrong with the page? Please let us know by submitting a report below.
LP
Data Migration Specialist / Developer (ETL)
LinkedIn Recruiter Post
Hiring Data Migration Specialist / Developer (ETL) for contract hybrid role in Oakville, ON. Requires 9+ years experience in data migration, data engineering, and cloud.
AI Platform Engineer leading multidisciplinary teams building AI - powered applications and intelligent workflows at Valtech. Shape the future of experience through Data, AI, and emerging solutions.
LP
Datastage ETL
LinkedIn Recruiter Post
Aarorn Technologies seeks a Datastage ETL contractor with 5+ years of experience for a hybrid role in Toronto. Must have IBM InfoSphere DataStage, SQL, Unix scripting, and scheduling skills.
LP
Azure Databricks Developer-HR Platforms
LinkedIn Recruiter Post
Azure Databricks Developer needed for HR tech projects at a top financial client. Design ETL solutions, support data integrations, and modernize HR data platforms.
Senior Software Engineer, Data responsible for building data pipelines and maintaining infrastructure for analytics and ML. Collaborate with teams to optimize data systems at Narvar.
Data Architect leading conversations and decisions on data modeling at a global consulting firm specializing in Data & Analytics. Transforming complex processes into clear, reusable information models.
Senior Data Engineer at Supabase managing data pipelines from source to analysis. Collaborating with growth, finance, and product teams to drive business insights.
LP
Data Architect – Snowflake
LinkedIn Recruiter Post
Data Architect - Snowflake needed for full - time hybrid role in Markham, ON. Expertise in Snowflake on AWS, data migration, and ETL required.
LP
Datastage ETL
LinkedIn Recruiter Post
Hiring Datastage ETL contractor for hybrid role in Toronto. Requires 5+ years experience with IBM DataStage, SQL, Unix scripting.