Structural Diff
A weight diff assumes two tensors are the same shape and asks how their values differ. A structural diff asks a prior question: do the two checkpoints even have the same shape of model - same layers, same modules, same tensor shapes and dtypes - before any value-level comparison is meaningful.
What a structural diff surfaces
Comparing two checkpoints' Semantic Model Graphs directly, rather than their weight values, surfaces:
- Added or removed layers/modules - present in one checkpoint's tree but not the other's.
- Shape changes - the same-named tensor with a different shape between the two (a resized embedding table after vocabulary expansion, for instance).
- Dtype changes - the same-named, same-shaped tensor with a different dtype (commonly from a quantization pass applied to one side only).
Each of these appears in the Outliner as a per-node structural marker (added/removed/changed-type), independent of and prior to whatever the Diff view shows for value-level changes on the tensors that do still match.
Why this matters before diffing values
A weight diff computed across two structurally different checkpoints without first checking structure produces numbers that are technically defined but not meaningful - comparing a tensor at the "same" name across two checkpoints whose architectures actually diverged there is comparing the wrong things, not measuring a real change.
This is also the distinction Merge Conflicts draws between numerically resolvable and structural conflicts - a structural diff is what determines which category a given difference falls into before any resolution approach is chosen.
Related resources
- Weight Diff - the value-level comparison that structural diffing is a precondition for.
- Merge Conflicts - where the resolvable-vs-structural distinction determines available resolution options.
- Model, Architecture, Layer and Module - the Semantic Model Graph a structural diff compares.