Senior Data Engineer developing and optimizing data pipelines on Databricks for healthtech company. Collaborating with stakeholders and ensuring data integrity through efficient processing.
Responsibilities
Develop, manage, and optimize data pipelines on the Databricks platform.
Debug and troubleshoot Spark applications to ensure reliability and performance.
Implement best practices for Spark compute and optimize workloads.
Write clean, efficient, and reusable Python code using object-oriented programming principles.
Design and build APIs to support data integration and application needs.
Develop scripts and tools to automate data processing and workflows.
Integrate, query, and manage data within MongoDB.
Ensure efficient storage and retrieval processes tailored to application requirements.
Optimize MongoDB performance for large-scale data handling.
Work closely with data scientists, analysts, and other stakeholders to understand data needs and deliver solutions.
Proactively identify and address technical challenges related to data processing and system design.
Requirements
Proven experience working with Databricks and Spark compute.
Proficient in Python, including object-oriented programming and API development.
Familiarity with NoSQL (MongoDB preferred), including querying, data modeling, and optimization.
Strong problem-solving skills and ability to debug and optimize data processing tasks.
Experience with large-scale data processing and distributed systems.
Senior Backend Java Engineer at Capgemini Engineering, focusing on projects in the hospitality sector. Collaborates with Machine Learning team, utilizing cutting - edge tech stack.
Senior Software Developer responsible for building high - correctness Rust services for Narvar. Collaborating across teams while owning systems from architecture to operations.
Consultant Developer role focusing on client delivery of quality software solutions. Working with teams to implement solutions that are well - tested and easily maintained.
Growth Product Designer for pdf.net, building the acquisition engine and optimizing conversion through design and data. Collaborating with PMs and engineers in a fully remote, high - autonomy environment.
Senior .Net programmer at Exposant 3 developing and modifying business solutions in a hybrid work environment. Committed to mutual success and team collaboration.
Staff Backend Engineer focusing on adaptive telemetry and databases for Grafana Labs. Responsible for technical strategy, project delivery, and system reliability.
Senior Backend Engineer building authentication features and services at GitLab. Working on token issuance, access management tooling, and security improvements for user authentication.
Senior Backend Programmer at Behaviour Interactive developing backend systems for '7 Days to Die'. Leading integration of online services and collaborating with cross - disciplinary teams.