Senior Data Engineer

Posted via LinkedIn Recruiter (not a company profile)

Posted last week

Apply Now

About the role

  • Senior Data Engineer role in Mississauga, ON. Requires 6+ years experience with Azure, Databricks, Scala, Spark, and database design.

Responsibilities

  • Senior Data Engineer role focused on big data analytics and data engineering. Responsibilities include working with Databricks platform, Spark framework (RDD/DataFrame/SQL), Azure cloud services (Data Lake Storage, Azure Data Factory), database design & development, and implementing CI/CD pipelines for automated deployments. Additional focus on AI/ML development using Python, Langchain, Azure OpenAI, Vector Databases, and RAG (Retrieval-Augmented Generation) techniques.

Requirements

  • Bachelor’s or Master’s degree in Computer Science or related field. 6+ years of hands-on experience in software engineering/data engineering roles. Strong experience with Databricks platform for big data analytics. Proficient in Scala programming language; solid understanding of Spark framework (RDD/DataFrame/SQL). Hands-on experience working on Azure cloud services (e.g., Data Lake Storage, Azure Data Factory). Strong background in Database design & development; excellent SQL skills (T-SQL/PLSQL etc.). Understanding of CI/CD pipelines for automated deployments on cloud platforms. Experience working within Agile/Scrum methodologies. Advanced knowledge of Python for scripting, automation, and AI/ML development. Familiarity with Langchain for integrating and orchestrating large language model workflows. Hands-on experience with Azure OpenAI to leverage generative AI capabilities in applications. Understanding of Vector Databases for efficient storage and retrieval of high-dimensional data embeddings. Knowledge of RAG (Retrieval-Augmented Generation) techniques.

Job title

Job type

Contractor

Experience level

Senior

Salary

Not specified

Degree requirement

Bachelor's or Master's degree in Computer Science or related field

Tech skills

Azure Data FactoryAzure Data Lake StorageDatabricksScalaSparkSQLPythonLangchainAzure OpenAIVector DatabasesRAG

Location requirements

Linkedin Recruiter PostMississaugaOntario Mississauga

Report this job

Found something wrong with the page? Please let us know by submitting a report below.