Staff Software Engineer building Cerebras’s production ML inference APIs. Enabling reliable model serving across GPU and Cerebras AI accelerator backends.
Responsibilities
Design, implement, and maintain production ML inference APIs for chat completions, text generation, streaming, model configuration, tool calling, structured outputs, multimodal inputs, and emerging inference capabilities
Create consistent request and response semantics across GPU prefill, Cerebras decode, and other heterogeneous inference backends
Integrate emerging foundation models, tokenizers, prompt formats, sampling methods, attention variants, multimodal inputs, and model-specific features
Maintain compatibility with widely adopted inference interfaces while designing Cerebras-specific extensions
Establish versioning, deprecation, validation, and backward compatibility practices
Extend and integrate custom inference services with vLLM, PyTorch, Hugging Face libraries, AMD ROCm, and Cerebras runtime components
Build control and data paths coordinating GPU prefill with Cerebras decode, including request routing, state transfer, error handling, retries, and lifecycle management
Optimize streaming, time to first token, latency, throughput, batching, serialization, tokenization, scheduling, and communication between serving components
Build validation systems for tokenization, sampling, logits, generated outputs, precision changes, model upgrades, determinism, and backend compatibility
Define service indicators and build logging, tracing, metrics, dashboards, health checks, and diagnostic tooling
Build configuration, SDKs, documentation, examples, debugging tools, and self-service workflows
Partner with compiler, runtime, kernel, cloud, product, and solutions teams and work directly with customers
Requirements
5+ years of software engineering experience, including substantial individual-contributor ownership of production software or distributed systems
Strong programming ability in Python and Go
Experience developing performance-sensitive or highly concurrent services in C++, Rust, or a similar systems language
Experience building stable APIs with validation, error handling, observability, compatibility, and versioning practices
Experience integrating software across service, framework, runtime, and infrastructure boundaries
Experience designing or maintaining OpenAI-compatible, gRPC, REST, or streaming inference APIs
Experience with Linux, containers, Kubernetes or comparable orchestration systems, CI/CD, and operating latency-sensitive production services
Ability to diagnose correctness, reliability, and performance issues across distributed serving systems
Strong communication and cross-functional execution skills
Bachelor's degree in computer science, Computer Engineering, Electrical Engineering, or a related discipline, or equivalent practical experience
Preferred: experience with vLLM, SGLang, PyTorch, Hugging Face Transformers, Triton, TensorRT-LLM, or another open-source ML systems project
Preferred: experience creating API conformance, model-quality, numerical-comparison, determinism, or performance-regression test systems
Preferred: experience building SDKs, developer tools, model registries, configuration systems, or self-service ML platforms
Preferred: experience with multi-model or multi-tenant inference platforms, routing, admission control, fairness, quotas, rate limiting, and capacity-aware scheduling
Preferred: understanding of model-specific tokenization, chat templates, generation configuration, logits processing, stopping criteria, tool calling, structured generation, and constrained decoding
Preferred: experience with disaggregated prefill/decode architectures, KV-cache transfer, prefix caching, chunked prefill, memory-aware admission control, or request scheduling
Preferred: experience designing, building, or operating production APIs and services for machine learning, large language models, or other data-intensive applications
Preferred: familiarity with BF16, FP8, FP4, INT8, or INT4
Benefits
Build a breakthrough AI platform beyond the constraints of the GPU
Publish and open source cutting-edge AI research
Work on one of the fastest AI supercomputers in the world
Job stability with startup vitality
Simple, non-corporate work culture that respects individual beliefs
Associate Software Developer building AI, instrument - integration and data - pipeline software for Miraterra’s soil - intelligence platform. Supporting farmers, researchers and policymakers with actionable soil insights.
Senior engineer building agentic AI features, backend services, and customer workflows for Samsara’s connected - operations platform. Improving safety, efficiency, and reliability for physical operations.
Senior engineer building agentic AI detections and workflows for Samsara’s connected - operations IoT platform. Shipping backend, API, and customer - facing software for safer physical operations.
Senior software engineer building AI - native, full - stack features for Life360’s family safety platform. Serving aging parents and expanding products across connected trackers and pets.
Senior software developer building secure account systems for businesses, families, and joint owners. Improving regulated financial onboarding at Canadian fintech Wealthsimple.
Senior Engineer shaping Megaport’s global Network as a Service edge compute and networking platform. Architecting high - performance VNF infrastructure and distributed systems.
Senior software developer building creative tools, ad rendering, delivery systems, and performance services for Equativ’s digital advertising platform. Hybrid Montreal role collaborating across Product, Design, Data, and Engineering.
Senior software developer building scalable advertising systems at Equativ, an independent digital advertising platform. Developing creative pipelines, web delivery, video experiences, and supporting services.
Senior backend engineer building scalable, reliable systems for Canals, which automates wholesale distribution workflows with AI. Owning complex projects and guiding technical direction.
Principal Software Engineer leading cloud - native, full - stack architecture and modernization. Improving Veradigm healthcare tools for scheduling, messaging, payments, and patient engagement.