NVIDIA's open-source Model Optimizer library now documents an end-to-end W4A4 NVFP4 recipe: 4-bit post-training quantization of both weights and activations, combined with quantization-aware distillation (QAD), demonstrated on Qwen3.6-35B-A3B. The result NVIDIA cites in its release notes is up to 1.30× vLLM throughput over BF16 with 3.1× smaller checkpoints, while recovering most of the accuracy that aggressive quantization costs.
The more interesting part is the target. 4-bit weights have been standard for a while; the hard step was 4-bit activations. Activations vary more than weights and are sensitive to outliers, which is why many production recipes stop at W4A16 or W8A8. NVFP4 adds block scaling that makes the quantization practical on Blackwell hardware with native FP4 support.
QAD is the mechanism that brings accuracy back: instead of calibrating a quantized model once, training continues against the original behaviour so that the errors introduced by aggressive quantization get corrected. NVIDIA had already reported in June that it quantized Nemotron 3 Ultra (550B) to NVFP4 with the same library, claiming up to 5.9× higher throughput on decode-heavy inference than a 754B FP4 model at comparable BF16 accuracy. Other parts of the toolchain cover local Hessian weight scales and AutoQuantize, an automatic mixed-precision assignment method.
The library itself is broad: it accepts Hugging Face, PyTorch or ONNX inputs and composes quantization, distillation, pruning, neural architecture search, speculative decoding and sparsity, exporting to TensorRT-LLM, TensorRT, vLLM and SGLang. The practical stake is inference arithmetic — 4-bit checkpoints that cut memory by roughly three quarters and lift throughput without sacrificing accuracy determine which models are economical to serve on which hardware.
Treat the numbers as vendor figures: NVIDIA reports on its own benchmarks and its own models. The direction is nonetheless clear. After 4-bit inference became the default, the work has moved to the activations, and the tooling for it is open.
