About the role

Databricks Platform Engineer responsible for owning and optimizing the Databricks platform. Collaborating with various teams to support workflows and ensuring performance and security.

Responsibilities

Administer, configure, and maintain Azure Databricks workspaces, including user management, cluster setup, resource allocation, provisioning cloud services, and backup and recovery strategy
Own and evolve platform automation, including Infrastructure as Code (Terraform) and Azure DevOps CI/CD pipelines for Databricks and dependent cloud resources
Monitor system performance and troubleshoot issues related to Databricks clusters, jobs, and data pipelines
Implement best practices for security, compliance, and cost management within Azure Databricks environments
Collaborate with data engineers, data scientists, and other stakeholders to support their workflows and optimize platform usage
Partner with cloud networking and security teams to design and operate secure connectivity patterns
Integrate Databricks with various SaaS products and services
Administer and maintain Power BI workspaces, datasets, and reports
Automate routine administrative tasks using scripting (Python and/or Bash) and Databricks REST APIs
Manage data access controls, auditing, and monitoring to maintain data integrity and privacy
Stay current with new features, updates, and best practices in Databricks, Azure, and relevant SaaS platforms
Document procedures, configurations, and troubleshooting steps for internal knowledge sharing.

Bachelor’s degree in Computer Science, Information Technology, or a related field (or equivalent experience)
Proven experience administering Azure Databricks (or cross-cloud equivalent)
Solid understanding of Azure cloud services (e.g., Azure Data Lake, Azure Synapse Analytics, Entra, Azure DevOps)
Proficiency with Infrastructure as Code using Terraform (preferred)
Working knowledge of cloud infrastructure concepts—storage, networking, and identity and access management (IAM)
Experience integrating and managing SaaS products within a cloud-based data ecosystem
Strong scripting and automation skills (Python and/or Bash)
Knowledge of big data concepts, Spark architecture, and ETL processes
Familiarity with data security, compliance, and governance best practices
Agility, innovation, responsiveness and a continuous improvement mindset through proactive problem-solving
Relevant certifications (e.g., Azure Administrator, Databricks Certified Associate) are a plus.