# Expert Similarity

The MoE Expert Galaxy view and its use of a similarity-driven layout are confirmed design facts; the specific similarity metrics described below are standard techniques applied to that view, not sourced from a local reference document.

In a Mixture-of-Experts layer, two experts can end up nearly identical after training (redundant capacity that isn't earning its parameter cost) or sharply divergent (genuine specialization). Telling these apart is what expert-similarity analysis is for, and it's the computation that drives the default spatial layout of the [MoE Expert Galaxy](/docs/working-with-models/mixture-of-experts-models) view.

## Computing similarity

Like [SVD, PCA and effective rank](/docs/inspect-and-analyze/svd-pca-and-effective-rank), pairwise expert similarity is an **explicit analysis** - it requires comparing actual weight data across every pair of experts in a layer, not just combining precomputed tile statistics. Requesting it for a layer computes a similarity score for every expert pair (typically a cosine similarity or correlation between flattened weight vectors, or between each expert's dominant principal directions), producing a full pairwise similarity matrix for the layer's experts.

## Reading the result

The similarity matrix is viewable directly as a small heatmap (one row/column per expert), but its main use is driving the **Expert Galaxy**'s idle-state layout: experts position themselves in 3D space so that similar experts cluster together and dissimilar ones sit apart, turning an *n*-by-*n* similarity matrix into something you can read spatially at a glance rather than by scanning a grid of numbers.

## What high similarity suggests

Two experts with very high similarity are worth double-checking against a few different explanations, since the fix differs by cause:

-   **Under-trained routing** - the router rarely sends distinct-enough inputs to both, so neither has had the chance to specialize away from the other.
-   **Genuine redundancy** - both experts converged to a similar function because the task didn't need the extra capacity, which is a legitimate pruning candidate.
-   **Initialization proximity** - early in training, or for a freshly-initialized model, experts can be similar simply because they haven't diverged yet.

Cross-referencing a routing trace from [Runtime Traces](/docs/profiling/runtime-traces) - how often each expert is actually selected - is usually what separates "under-trained" from "genuinely redundant."

## Related resources

-   [Mixture-of-Experts Models](/docs/working-with-models/mixture-of-experts-models) - the Expert Galaxy view this analysis feeds.
-   [SVD, PCA and Effective Rank](/docs/inspect-and-analyze/svd-pca-and-effective-rank) - a related explicit-analysis-tier computation.
-   [Runtime Traces](/docs/profiling/runtime-traces) - capturing actual routing frequency to interpret a similarity result.