Skip to content
Docs
Core Concepts
Model morphing

Model Morphing

Morphing is Tensormorph's term for building a new model out of weight-space operations on existing ones - interpolating between two checkpoints, merging a task-specific delta into a base model, or combining several fine-tunes into one. It has its own dedicated 3D view (Morph, alongside Architecture, Layer Stack, Tensor Volume, MoE Expert Galaxy, Parameter Map, Diff, and Runtime Flow), and its own artifact model, distinct from ordinary checkpoint editing.

Expression DAG, not an edit history

💡

A morph is authored as an expression DAG - a graph of operations over model/tensor inputs - and only turns into actual output weights when you materialize it. Until materialized, a morph is a description, not a checkpoint.

This separation matters for two reasons: the same recipe can be re-materialized against an updated base checkpoint without re-authoring it, and a recipe is small and diffable (it's a graph of operations, not gigabytes of weights) even when what it produces is enormous. A materialized result is written out as an ordinary checkpoint like any other; the DAG that produced it is saved separately as recipe.json.

Common operation types

Nodes in a morph DAG represent weight-space operations - the exact catalog is still being finalized, but the operation family generally includes:

  • Task-vector addition - adding the difference between a fine-tune and its base back onto a different base, isolating "what fine-tuning changed" from the base weights themselves.
  • Interpolation - blending two checkpoints, either linearly or along a spherical path (SLERP) that respects the geometry of high-dimensional weight space better than a straight line does.
  • Selective merging (in the spirit of techniques like TIES or DARE) - merging multiple task vectors while resolving sign conflicts or dropping redundant small updates, rather than naively averaging everything.

Each node in the graph is inspectable in the Morph view the same way a computation-graph node is inspectable elsewhere in Tensormorph - its inputs, its operation type, and (once materialized) the resulting statistics.

Validating a morph result

A materialized morph is not assumed correct just because it ran without error. See Comparing Two Model Revisions and Model Diff for comparing a morph's output against its base, and CI Validation for running a validation suite against a materialized result as part of a pipeline rather than only interactively.

Related resources

Model diffLevel of detail and streaming