Senior Software Engineer, SRE – Production Engineering

Posted 2 days ago

Apply Now

Resume Score

Check how well your resume matches this job before you apply.

Sign in to check score

About the role

  • Senior Software Engineer building bare-metal provisioning, hardware lifecycle, and reliability automation for NVIDIA DGX Cloud’s large-scale GPU infrastructure.

Responsibilities

  • Build automation for bare-metal provisioning, hardware validation, firmware and software upgrades, repair, and cluster lifecycle management
  • Develop tools that interact with BMC and Redfish interfaces to monitor hardware health, manage server state, and assist recovery workflows
  • Handle and advance NVIDIA NVL72 systems and BlueField-3 or later DPUs throughout cloud partner and on-premises environments
  • Diagnose failures across servers, DPUs, GPU systems, CPU systems, networking, Linux, and Kubernetes; turn recurring issues into automated detection and repair
  • Define validation and handoff criteria so new capacity enters production safely and consistently
  • Take part in on-call duties, incident response, root-cause analysis, and follow-up to implement permanent solutions
  • Work with hardware, networking, platform, data center operations, and partner teams to resolve issues across ownership boundaries

Requirements

  • 12+ years building software for or operating production infrastructure, including substantial hands-on bare-metal experience
  • Strong Go or Python skills, with a record of delivering production automation and services
  • Direct experience with BMC and Redfish in server provisioning, health inspection, power management, or fault diagnosis
  • Practical experience working directly with NVIDIA GPU hardware, such as NVL72 systems, and BlueField-3 or newer DPUs
  • Experience with Linux, firmware and driver management, network boot, and the server lifecycle from initial provisioning through repair
  • Experience managing production reliability through on-call duties, incident response, observability, and durable solutions
  • Ability to debug failures across hardware, host operating systems, networking, and distributed services
  • Clear communication and demonstrated ownership of problems that span multiple teams
  • BS/MS in Computer Science or equivalent experience in a practical setting
  • Experience operating BlueField DPUs in DPU mode, including host-to-DPU connectivity and lifecycle debugging, with DPU or equivalent experience considered
  • Background with NVLink, InfiniBand, Spectrum-X, or GPU cluster performance validation
  • Experience building safe, repeatable workflows for rack-scale bringup, firmware upgrades, hardware replacement, and customer handoff
  • Experience with Kubernetes, GitOps, Argo CD, SLOs, and fleet-wide automation

Benefits

  • Equity
  • Benefits

Job type

Full Time

Experience level

Senior

Salary

Not specified

Degree requirement

Bachelor's Degree

Tech skills

CloudKubernetesLinuxPythonGo

Location requirements

RemoteCanada

Report this job

Found something wrong with the page? Please let us know by submitting a report below.