An arXiv preprint reports higher average benchmark scores for a new way of selecting NVFP4 scales across several text-only language-model tests. The method, called H-Scale, uses calibration activations to choose hardware-valid per-group scales with an estimate of how scale changes affect a layer's output. Its target is layer-output perturbation rather than plain weight reconstruction.
In practical terms, the change happens during offline preparation, when the system decides which local scale to pair with each group of weights. The benchmark comparison kept inference cost unchanged. The added work was in the scale search itself, and the reported offline runtimes rose in the tested examples.
A search guided by model outputs
At the core of H-Scale is a diagonal second-order proxy, used as a sensitivity estimate. Using calibration activations, the search weighs how a candidate scale is likely to change a layer's output and favors choices expected to limit that perturbation. That differs from choosing a scale only because it gives a smaller numerical reconstruction error for the weights.
All evaluated methods used 4,096 FineWeb calibration samples with a sequence length of 8,192. H-Scale then searched a default 16-position window of E4M3-representable local-scale candidates, with nine steps toward smaller scales and six toward larger ones. Every candidate is exactly representable in E4M3, keeping the refinement inside the format's available scale values.
The strongest gains appeared in average scores
In a layer-wise validation, H-Scale reduced average activation loss by 32.5% relative to Min-Max. Best Scale, an expensive oracle reference, reached 40.8%, while Weight-Guided Scale reached 12.6%. The paper describes H-Scale's result as about 80% of the oracle gain. Activation loss here measures how much a layer's outputs change after quantization, so the test checks a different target from weight error alone.
The clearest benchmark result came from Qwen3-30A3-Thinking. H-Scale had a higher average score for every tested post-training quantization pipeline and narrowed the gap to the BF16 reference, whose average was 81.06. GPTQ combined with H-Scale reached 81.22. Each setting was evaluated in three independent decoding runs and scored by their mean.
On the eight-task Qwen3-30A3-Instruct suite, H-Scale yielded a higher average for every baseline. The paper also reports higher observed averages for Qwen3-4B-Instruct and LLaMA-3.1-8B-Instruct across the tested pipelines or baselines. The individual benchmark results were mixed, however, and gains depended on the method being tested.
What the extra tests reveal
An ablation comparing H-Scale with scale-only optimization gave the Hessian-guided version the highest average score for all three tested baselines. Scale-only optimization was less stable and could hurt 4over6 and ArcQuant. The comparison reinforces the paper's focus on layer-output effects rather than weight reconstruction.
The candidate-window sweep found a sharp difference between one-sided and bidirectional searches. With no upward steps, the median recovered fraction was 64.63%. With at least six upward steps, the median reached 100.00%, and the 10th-percentile result was above 99.6%. Widening the total window to 20 or 24 candidate positions changed recovery only marginally.
The evidence has a clear boundary
Reducing the calibration set did not erase the reported pattern in one additional check. With all other settings unchanged, H-Scale still improved over the corresponding NVFP4 baseline on Qwen3-30A3-Instruct when the set was reduced from 4,096 samples to 128.
Readers should treat the benchmark gains as results from a bounded computational evaluation. It covers NVFP4 with groups of 16 values and text-only language models. Whether H-Scale's scale-contraction behavior transfers to other formats or group sizes, or to multimodal models, remains open. The runtime evidence is also offline and does not establish end-to-end production throughput or latency.
The document is an arXiv preprint, and its front matter lists Qwen Team, Alibaba Inc. as the affiliation. No separate funding statement is reported in the supplied text.
Paper data and sources
Original title: H-Scale: Hessian-Guided Scale Refinement for NVFP4 Sub-Byte LLM Inference
Authors: Hao Yu, Zheng Li, Dayiheng Liu, Jianwei Zhang
Journal/Repository: arXiv
Status: Preprint, not yet peer-reviewed
First online: 2026-08-28
DOI: Not available
Original paper · Full text