# Sparsity

The zero-count statistic and its role in Tensormorph's rendering are confirmed design facts; the specific sparsity-pattern classification described below (structured vs. unstructured) is standard domain vocabulary applied to this product, not sourced from a local reference document - there is no local PyTorch `torch.sparse` API reference in this workspace to adapt from.

Sparsity - the fraction of a tensor's values that are exactly zero - is measured directly from the **zero count** already computed as part of every tensor's [tile statistics](/docs/inspect-and-analyze/tensor-statistics), so checking how sparse a tensor is never requires a separate pass over its data.

## Why sparsity is worth checking

A tensor's sparsity ratio alone is a useful signal for several different situations that all produce zeros for different reasons:

-   **Pruning** - a model that's had unimportant weights zeroed out deliberately, where sparsity is the point.
-   **Quantization artifacts** - aggressive low-bit quantization can round small values to exactly zero as a side effect, which looks the same as pruning in a zero-count but has a different cause.
-   **MoE routing** - an expert that's rarely or never selected by the router will show near-total sparsity in its accumulated contribution, distinct from sparsity in the expert's own weights.

Because these look identical in a raw zero-count, distinguishing them usually means cross-referencing sparsity against [outliers and anomalies](/docs/inspect-and-analyze/outliers-and-anomalies) or against a [quantization diff](/docs/compare-and-morph/quantization-diff), rather than reading the zero-count in isolation.

## Structured vs. unstructured sparsity

Two tensors with the same overall sparsity ratio can look completely different once you look at *where* the zeros are:

-   **Unstructured sparsity** - zeros scattered without a regular pattern. Visually, this shows up as noise in the Matrix/Heatmap view's sparsity-mask shading mode.
-   **Structured sparsity** - zeros following a regular pattern (whole rows, columns, or fixed-size blocks zeroed together, or an N:M pattern such as 2:4). Structured sparsity is what most sparse-compute hardware paths can actually accelerate; unstructured sparsity generally can't be exploited for a speedup even at a high ratio.

The sparsity-mask shading mode is what makes this distinction visible at a glance - a structured pattern reads as a visible grid or stripe pattern in the heatmap, where unstructured sparsity reads as texture with no discernible regularity.

## Related resources

-   [Tensor Statistics](/docs/inspect-and-analyze/tensor-statistics) - the zero-count statistic sparsity is measured from.
-   [Matrix and Heatmap Editor](/docs/editors/matrix-and-heatmap-editor) - where the sparsity-mask shading mode lives.
-   [Quantization Diff](/docs/compare-and-morph/quantization-diff) - for distinguishing quantization-induced zeros from deliberate pruning.