Senior Data Engineer developing and optimizing data pipelines on Databricks for healthtech company. Collaborating with stakeholders and ensuring data integrity through efficient processing.
Responsibilities
Develop, manage, and optimize data pipelines on the Databricks platform.
Debug and troubleshoot Spark applications to ensure reliability and performance.
Implement best practices for Spark compute and optimize workloads.
Write clean, efficient, and reusable Python code using object-oriented programming principles.
Design and build APIs to support data integration and application needs.
Develop scripts and tools to automate data processing and workflows.
Integrate, query, and manage data within MongoDB.
Ensure efficient storage and retrieval processes tailored to application requirements.
Optimize MongoDB performance for large-scale data handling.
Work closely with data scientists, analysts, and other stakeholders to understand data needs and deliver solutions.
Proactively identify and address technical challenges related to data processing and system design.
Requirements
Proven experience working with Databricks and Spark compute.
Proficient in Python, including object-oriented programming and API development.
Familiarity with NoSQL (MongoDB preferred), including querying, data modeling, and optimization.
Strong problem-solving skills and ability to debug and optimize data processing tasks.
Experience with large-scale data processing and distributed systems.
Staff Software Developer leading the design and delivery of backend components at Varicent. Responsible for scalable service design and technical mentoring within a collaborative environment.
Senior .Net Developer focusing on high - load social discovery products with AI integrations. Leading architecture, development, and quality assurance in a remote global environment.
Senior Software Engineer focusing on backend systems for an e - commerce startup. Responsible for designing and building services that support core workflows and operations.
Senior Backend Programmer at Behaviour Interactive developing cloud - based gaming services. Collaborating with teams to enhance online gaming experiences in a dynamic studio environment.
Security Platform Developer at Sun Life responsible for automating security processes and developing workflows using Python and Ansible. Collaborating on security initiatives and managing enterprise projects for improved service delivery.
Senior Drupal Engineer developing backend integrations and maintaining CI/CD pipelines. Working remotely with global teams to enhance platform stability and observability.
Software Engineer contractor supporting Black Book's Business Systems team in building scalable applications. Focused on developing and maintaining the Transactional Billing Automation platform for improved operations.
Sr Python Developer needed for hybrid role in Mississauga, ON. Must have Python, REST APIs, FastAPI/Django/Flask, Gen AI, AI/ML, microservices, AWS, CI/CD, Docker, and database experience.
Senior Software Engineer developing Java and Angular applications for 3Pillar Global. Collaborating with teams to drive innovative software solutions with a focus on client success.