P90 time-to-first-token fell to 3.8 seconds from 59 seconds for GLM-5.1 after Fixstars tuned inference on a single node with eight NVIDIA H200 GPUs.
The gain came after Fixstars replaced tensor parallelism with a DP-EP layout, using “DP Attention” to avoid MLA’s replicated compressed KV cache consuming memory across ranks.
NVFP4 quantization, KV-cache-aware routing and CPU cache offloading together expanded node KV-cache capacity by about 3.5x, creating room for more simultaneous requests.
Stable concurrency rose from a target of 30 to more than 50 on a benchmark based on real coding-agent workloads, showing the 754B-parameter open-weight MoE model can run more responsively on one node.