SVD, PCA and Effective Rank

🚧

This page's placement in Tensormorph's "explicit analysis" priority lane is a confirmed design fact; the specific analyses described (SVD, PCA, effective rank) are standard linear-algebra techniques applied to that lane, not sourced from a local reference document.

Unlike the tile statistics computed automatically for every tensor during indexing, singular value decomposition, principal component analysis, and effective-rank estimation all require reading substantially more of a tensor's actual data - they aren't mergeable summaries you can compute once and combine. Tensormorph treats them as explicit analysis: work you request on a specific selection, not something running continuously in the background.

Why this is a separate tier

Tensormorph's compute scheduler prioritizes work by proximity to what you're actually looking at - a hovered scalar first, then the current selection, then visible tiles, then near-visible prefetch - with explicit analysis deliberately scheduled after all of that and before generic background indexing. A rank computation on a large matrix is real work; running it unprompted for every tensor in a large model would starve the interactive rendering path it sits behind in priority. Requesting it is a conscious tradeoff: you're asking for a specific, deeper answer about a specific tensor, at the cost of it not being instant.

Effective rank

A weight matrix's nominal rank (the smaller of its two dimensions, for a full-rank matrix) is rarely the interesting number - what usually matters is how much of the matrix's behavior is captured by a small number of dominant singular values. Tensormorph's effective-rank estimate is computed from the singular value spectrum (for example, the number of singular values needed to capture a chosen fraction of total variance), and is surfaced directly in the Inspector panel once requested for a selection.

This is particularly relevant when evaluating whether a tensor is a good candidate for low-rank decomposition - during quantization, during a LoRA-style morph operation, or simply to understand how much of a layer's capacity is actually being used.

Reading an SVD/PCA result

Requesting an SVD or PCA analysis on a selected matrix surfaces:

The singular value spectrum as a curve (a sharp drop-off indicates strong effective low-rank structure; a flat spectrum indicates the matrix is using close to its full rank).
The top-k principal directions, available for further inspection as their own vectors.

Related resources

Tensor Statistics - the always-on statistics tier this page's analyses sit above.
Model Morphing - low-rank structure is directly relevant to LoRA-style morph operations.
Expert Similarity - another explicit-analysis-tier computation, comparing rather than decomposing.

NaN and Inf detection Expert similarity