Member of Technical Staff — Model Optimization and Inference (Experienced) at Nuance Labs · LumiJobs

Responsibilities

We can train a great model. The next problem is making it fast enough to actually use in a real-time conversation — and that gap is enormous. A model that responds in 3 seconds is a demo. A model that responds in under 500ms is a product.
We’re looking for someone who specializes in taking trained models and squeezing every last millisecond out of them. You understand the full stack from model weights to serving infrastructure — quantization, KV cache optimization, kernel-level acceleration, batching strategies — and you know which lever to pull for which problem. You’ve worked with vLLM, SGLang, or similar frameworks at scale and have strong opinions about where they fall short.
This posting is aimed at experienced engineers and researchers who’ve operated at a senior to senior-staff level at big tech, a leading AI lab, or a high-traffic inference team. Everyone at Nuance is MTS — we don’t run title ladders — but we’re hiring people who have already done this work at scale.
Our stack is more complex than a standard LLM deployment: we’re serving a full-duplex multimodal system that must satisfy strict real-time latency constraints. There’s a lot of unsolved optimization work here, and we need someone who finds that genuinely exciting.
Own end-to-end inference optimization across our model stack — LLMs, audio models, and diffusion-based components
Implement and tune KV cache strategies for long-context conversations, including eviction policies, compression, and memory-efficient attention
Evaluate, deploy, and extend inference serving frameworks (vLLM, SGLang, TensorRT-LLM, etc.) for our specific workloads
Profile and benchmark end-to-end latency and throughput; identify and systematically eliminate bottlenecks
Build internal tooling that makes optimization work faster and more rigorous — profiling viewers, end-to-end inference test harnesses, and other infrastructure that helps the team move quickly
Accelerate diffusion model inference — consistency models, step distillation, caching strategies, and custom kernel optimizations
Apply and develop quantization techniques (INT8, INT4, GPTQ, AWQ, and beyond) to reduce memory footprint and increase throughput without meaningfully degrading quality
Work closely with research and infrastructure to ensure new models ship with optimized serving from day one

Member of Technical Staff — Model Optimization and Inference (Experienced)

Responsibilities

Qualification

Required

Preferred

Benefits

Company

Nuance Labs

More roles at Nuance Labs