IAM Engineer at Kraken ensuring seamless, secure access management and compliance across the enterprise. Collaborating on cross - functional projects and mentoring junior engineers.
About the role
- Databricks Data Engineer developing and optimizing data pipelines using the Databricks platform. Collaborating with cross-functional teams to deliver high-quality data solutions for analytics and reporting.
Responsibilities
- Design, develop, and maintain scalable data pipelines using Databricks and Apache Spark.
- Build and optimize ETL/ELT workflows to ingest, transform, and curate large datasets.
- Implement data solutions following Medallion Architecture (Bronze, Silver, Gold).
- Develop high-performance PySpark and Spark SQL transformations.
- Optimize Delta Lake tables using partitioning, optimization, vacuuming, and time travel capabilities.
- Orchestrate and monitor Databricks Workflows and scheduled jobs.
- Collaborate with analytics, engineering, and product teams to deliver reliable data products.
- Apply software engineering best practices, including version control, automated testing, and CI/CD.
- Utilize AI-assisted development tools for code generation, debugging, documentation, and productivity while validating outputs to ensure quality and accuracy.
- Contribute to data quality, performance tuning, and operational excellence across the platform.
Requirements
- 7+ years of experience in data engineering
- Bachelor's degree in Computer Science, Software Engineering, or a related field (or equivalent work experience)
- Hands-on experience with Databricks Workspaces, Clusters, Jobs, and Unity Catalog
- Strong knowledge of Delta Lake, including ACID transactions, Time Travel, OPTIMIZE, and VACUUM
- Experience building and managing Databricks Workflows
- Strong experience with PySpark, Spark SQL, and Scala
- Expertise designing and building ETL/ELT pipelines
- Experience implementing Medallion Architecture (Bronze/Silver/Gold)
- Programming Languages: Python (Advanced), SQL (Advanced), Scala (Strong working knowledge)
- Experience with at least one of the following cloud platform: AWS (Amazon S3, AWS Glue, Redshift), Azure (Azure Data Lake Storage Gen2 (ADLS Gen2),Azure Data Factory (ADF), Azure DevOps)
- Experience using AI-powered developer tools for coding, debugging, testing, and documentation
- Understanding of responsible AI usage, code validation, and quality assurance.
- Git version control, CI/CD pipelines, Unit testing and automated validation for data transformations, Strong software engineering and code quality practices.
Benefits
- Join one of the world’s fastest-growing AI-first digital engineering companies and make a real impact at scale.
- Lead and collaborate with a high-energy team of talented, driven individuals solving complex, meaningful challenges.
- Work with Fortune 500 companies and disruptive innovators in a research-driven environment with 60+ patents.
- Stay ahead of the curve by gaining hands-on experience with cutting-edge AI, ML, data, and cloud technologies while continuously upskilling.
Job title
Job type
Experience level
Salary
Not specified
Degree requirement
Tech skills
Location requirements
Report this job
Found something wrong with the page? Please let us know by submitting a report below.
Senior Agent Engineer responsible for agent engineering in AI tools at Sourcegraph's code intelligence platform. Driving technical direction, product quality, and mentorship within a global distributed team.
LP
Java Engineer
LinkedIn Recruiter Post
Hiring a Java Engineer for a full - time role in Mississauga, ON, supporting Citi Bank with enterprise cloud engineering and financial applications.
AJO Engineer designing and optimizing customer engagement solutions using Adobe Experience Cloud technologies for Valtech. Collaborating with marketing and data teams to create scalable, data - driven campaigns.
Protection & Controls Electrical Engineer - in - Training at Qualus focusing on substation protection, control, and metering system designs. Collaborating within a multidisciplinary team for utility projects.
EMC Engineer responsible for EMC qualification and certification in aerospace. Executing engineering assignments and guiding team members to support compliance and certification activities.
M365 Productivity & Collaboration Engineer delivering Microsoft 365 productivity and collaboration solutions in a large enterprise environment. Design, engineer, and maintain endpoint and collaboration solutions for enterprise technology alignment.
Early - Career Scientist or Engineer applying academic knowledge to site assessments and remediation. Collaborating with a diverse client base to solve environmental challenges.
SailPoint ISC Engineer responsible for setup and management of SailPoint Identity Security Cloud components. Engaging with stakeholders to enhance identity governance workflows.
Intermediate Municipal Engineer managing civil engineering projects at GEMTEC. Collaborating with teams and ensuring project delivery within budget and scope.