Quantized Models
torch/torchao API reference, so there is no local source to draw from for this page. Treat framework-specific claims as illustrative rather than confirmed against a specific PyTorch version.A quantized tensor stores values at lower precision than the model was trained in - most commonly 8-bit or 4-bit integers instead of 16- or 32-bit floats - along with the small amount of extra metadata needed to recover an approximate real value from each stored integer. Opening a quantized checkpoint in Tensormorph works like any other, but several views and inspections behave differently once dtype metadata says a tensor is quantized rather than a plain float tensor.
What's stored alongside the integers
A quantized tensor is never just integers - it carries the parameters needed to dequantize:
- Scale - a float multiplier converting a stored integer back toward the original range.
- Zero-point - an integer offset, for schemes that aren't symmetric around zero.
- Granularity - whether scale/zero-point are shared per-tensor, per-channel (one pair per output channel - common for weight quantization), or per-group (a small run of adjacent values sharing one pair - common for low-bit weight-only quantization).
Tensormorph's Scalar Inspector shows both the raw stored integer and the dequantized approximate value for any scalar you inspect, side by side, rather than only one or the other.
Reading quantization in the Architecture and Matrix/Heatmap views
The quantization boundaries overlay (see Architecture Editor) draws a visible line wherever dtype or quantization scheme changes across a module - useful for spotting, at a glance, which layers a quantization pass actually touched versus left at full precision. In the Matrix/Heatmap view, a dedicated quantization shading mode colors each element by its quantization granularity group rather than by value, making per-channel or per-group boundaries visible directly on the tensor itself.
Quantization diff
Comparing a quantized model against its full-precision source is a distinct operation from an ordinary weight diff, because the "expected" difference is the quantization error itself, not a meaningful change - see Quantization Diff. The interesting signal is where the error is unusually large relative to a tensor's own scale, which is where quantization silently degraded something rather than merely rounding it.
Related resources
- Dtype and Quantization - the underlying dtype model quantized tensors extend.
- Quantization Diff - comparing a quantized model against its full-precision source.
- GPU Backend Compatibility - low-bit quantized kernels are not uniformly supported across every compute backend.