# Announcing Tensormorph 1.0 \[#announcing-tensormorph-10] > Tensormorph 1.0 introduces a professional Tensor IDE for exploring, comparing, debugging, and understanding open-weight AI models. Today we're releasing Tensormorph 1.0, an IDE for opening, navigating, inspecting, comparing, and debugging open-weight model checkpoints. This is Tensormorph's first public release — everything below is new in the sense that none of it has shipped before, but it isn't a first draft: 1.0 is the point where the project's underlying data model, addressing scheme, and view architecture are stable enough to build on. Modern open-weight models are distributed as `.safetensors` files, sharded checkpoints, and architecture manifests — a few hundred gigabytes of typed arrays and a JSON index. Understanding one today usually means a one-off Python script: load a tensor, print its shape, plot a histogram in a separate window, repeat, losing the surrounding architectural context each time. Source code stopped being edited this way decades ago, once IDEs made an entire codebase navigable, searchable, and debuggable as one connected system; model weights, by comparison, are mostly still inspected as opaque arrays. Tensormorph 1.0 turns an open-weight model from a collection of files and arrays into a navigable computational system — one address space running from a whole model down to a single scalar, with the same selection following you through every view. Tensormorph 1.0 is a first public release, not a finished product. Some capabilities described below — model morphing's full operation catalog, Lob-side review and conflict resolution, a public plugin registry — are confirmed architectural directions with parts of their interaction design still settling. Each section notes what's stable to build on today versus what's still in motion. ## Why a Tensor IDE? \[#why-a-tensor-ide] None of the individual pieces of a model-inspection workflow are new. Printing a tensor's shape, plotting a distribution, diffing two checkpoints — these are all things a few lines of Python already do. What's missing is the connective tissue between them: - A script that prints `torch.Size([4096, 4096])` for `layer.12.attn.q_proj.weight` doesn't know that tensor is one of 32 nearly-identical projections, or where it sits inside the attention block that owns it. - A histogram in one notebook cell and a heatmap in the next aren't the same *view* of the same object — two disconnected plots that happen to describe the same tensor, neither surviving past the notebook session. - A 400 GB sharded checkpoint doesn't fit in memory, so "just load it and look" stops being an option well before the interesting questions start. - Comparing a base checkpoint against a fine-tune, a quantized export, or a merge result means writing custom alignment code before any diffing can begin — get the alignment wrong and the resulting numbers look meaningful without being so. - A tensor's static values and its runtime behavior — activations, gradients, routing decisions — usually live in separate tools entirely, so connecting "this weight looks unusual" to "this weight misbehaves at runtime" is manual, slow work. Tensormorph addresses these by giving every one of those questions the same underlying resource model. A model, once opened, is read into a typed structure Tensormorph calls the **Semantic Model Graph** — every layer, module, tensor, tile, and scalar gets a stable address, and every view (a 3D structural view, a 2D heatmap, a 1D distribution plot, a scalar readout) is a different projection of that same graph rather than its own private copy. Select a module in one view, and it's highlighted, framed, and summarized everywhere else, automatically. ## What's new in Tensormorph 1.0 \[#whats-new] ### A unified tensor workspace \[#a-unified-tensor-workspace] Everything in a Tensormorph session — the loaded model, its architecture manifest, an open comparison pair, a captured runtime trace — is addressed through one shared **InspectionContext**. Whatever you've selected, filtered, or pinned lives in that single context; no view maintains its own private notion of "what's currently interesting." Click a module in the tree-based **Outliner** and it's simultaneously highlighted in the 3D structural view and summarized in the Inspector panel, with no explicit synchronization step — that's a property of one shared context underneath every view, not a feature you turn on. ### Architecture-aware navigation \[#architecture-aware-navigation] Every drill-down in Tensormorph follows the same address space, from a whole checkpoint to a single number: ```text Model → Architecture → Layer → Module → Tensor → Tile/Block → Row/Col/Channel → Scalar ``` The Outliner gives you this as a tree; the 3D structural view gives you the same hierarchy spatially; a fuzzy-search command palette (`Ctrl+P`) and a filter-by-predicate mode let you jump or narrow directly. Selection history steps back through what you've navigated to, the same way browser back/forward does for pages. A human-readable path like `layer.12.attn.q_proj.weight` is convenient to type and read, but it's not a permanent identity — a renamed layer or a restructured checkpoint can change it. The underlying address in the Semantic Model Graph is what's stable; the readable path is a rendering of that address, not the address itself. ### Coordinated 3D, 2D, 1D, and scalar inspectors \[#coordinated-3d-2d-1d-and-scalar-inspectors] Tensormorph deliberately doesn't collapse structure, values, signals, and exact numbers into a single overloaded view — each level of detail gets a view built for the question it actually answers: - **3D views** cover spatial structure and relationships: the whole-model **Architecture** view, a **Layer Stack**, a per-tensor **Tensor Volume** view, the **MoE Expert Galaxy**, a **Parameter Map**, a **Diff** view, a **Morph** view, and a **Runtime Flow** view — eight views, one shared spatial language. - **The 2D Matrix and Heatmap view** reads one tensor slice precisely — signed value, magnitude, sparsity, quantization granularity, diff, correlation, SVD reconstruction, and outlier shading modes, with cell-exact values on hover and no loss of precision as you zoom in. - **The 1D Signal and Distribution view** covers curves a 2D grid can't represent well: a raw value plot, a log-scale distribution histogram, a per-layer or per-expert profile for spotting the outlier, a singular-value spectrum, and a delta plot for a 1D diff. - **The Scalar Inspector** is the finest-grained readout: exact value, full address back to the model, shape/dtype/device metadata, view-vs-copy status, and — for a quantized value — both the raw stored integer and its dequantized approximation, side by side. These are coordinated projections of one tensor resource, not four separate tools you reconcile by hand — switching between them (`1`/`2`/`3` for 3D/2D/1D) never loses the current selection. ### The address grid \[#the-address-grid] Underneath every 3D and 2D view is a shared integer coordinate system — the **address grid**, derived directly from the Semantic Model Graph's own hierarchy rather than an arbitrary, user-configurable spacing. A tile snaps to its tile boundary; a layer snaps to its layer boundary. Selection, measurement, and comparison all read from this same grid, which is also what keeps two structurally different models comparable in the same space: their grids align at every level of the hierarchy even when their shapes differ. Overlays — shape labels, stride markers for view-vs-copy tensors, quantization-boundary lines, and the grid itself — are independent, toggleable layers drawn on top of whatever shading mode is active, so a label is never a reason to switch shading modes. ### Tensor-aware interaction \[#tensor-aware-interaction] The 3D views support the interaction vocabulary you'd expect from a serious spatial tool, applied specifically to tensor workflows rather than as a generic 3D-scene editor: orbit, pan, zoom, frame-selection (`F`) and frame-whole-model (`Home`) with both perspective and orthographic cameras; hover and click-precise picking down to a single tile or scalar; selection by type, by structural link ("select linked experts"), or by box/lasso, kept in sync with the Outliner regardless of which one you used; isolate (`/`) and hide (`H`) to narrow or exclude the current selection without changing what's actually selected underneath; a Measure tool reading out shape or distance on the address grid, and a Probe tool for inspecting a scalar under the cursor without disturbing the current selection; and filtering by shape, dtype, module type, expert, magnitude, sparsity, quantization error, or diff state, expressed through the same query language used everywhere else in Tensormorph (see [TQL](#matrix-multiplication-as-a-visual-operation) below). Weight editing and interpolation tools are scoped to the dedicated Morph view specifically — every other 3D view only ever renders derived statistics, so nothing done while exploring the Architecture or Tensor Volume views can accidentally alter a weight. ### Matrix multiplication as a visual operation \[#matrix-multiplication-as-a-visual-operation] A `MatMul` node — the operation underlying every projection, attention score, and MLP layer in a transformer — has a single spatial convention throughout Tensormorph: its two input matrices render as perpendicular planes, and the output renders as a third plane, with the shared contraction dimension highlighted as the common edge between them. $$ C[i, j] = \sum_k A[i, k] \cdot B[k, j] $$ Once you've learned to read one `MatMul` this way, it generalizes: the same convention applies to an isolated `q_proj` multiplication or to tracing how Q, K, V, and output projections connect into a full attention module in the **Runtime Flow** view. Grouping multiple `MatMul` nodes into a layer or a head is how you move from one multiplication to a model-level computational graph, without switching visual languages. Tensormorph's query and filter language, **TQL (Tensor Query Language)**, is what makes a selection like this precise and reusable rather than a one-off click. TQL is plain ASCII text everywhere you type it — the command palette (`:` to filter, `?` for natural-language input translated to TQL), the Console REPL, a breakpoint condition, or a saved query — and renders as typeset notation wherever there's room to show it: ```text # Structural + statistical filter layer.type == "attention" and tensor.absmax > 10 # Selection by name pattern, scoped to a module model["layer.12"].tensor("*.q_proj.weight") ``` Tensormorph does not render every scalar of a large model simultaneously, and it never will by design. A trillion-parameter checkpoint is explored through level-of-detail streaming and aggregated statistics — see [Multiresolution rendering and streaming](#multiresolution-rendering-and-streaming) — not by placing a geometric primitive for each parameter. ### Attention and Mixture-of-Experts inspection \[#attention-and-mixture-of-experts-inspection] An attention score tensor shaped `[batch, head, query, key]` is a rank-4 object — Tensormorph doesn't attempt to render four spatial dimensions at once. Instead, the **Tensor Volume** view resolves the extra axes through slicing (a chosen batch/head shown as a 3D volume) or small-multiples (a grid of 2D slices across an axis), and grouping by head, query-key range, or causal/padding mask is a filter and slicing operation, not a separate mode. The same approach covers grouped-query, multi-query, and latent-attention structures — what changes is how many key/value heads a `ModelAdapter` recognizes per query head, not the inspection mechanism. Mixture-of-Experts models get their own dedicated 3D view, the **MoE Expert Galaxy**, because "which experts actually fire, and how similar are they to each other" is usually the question that matters most for this architecture family. In its idle state, experts are laid out by pairwise similarity — computed from flattened weight vectors or dominant principal directions — so redundant experts cluster together and specialized ones sit apart, readable at a glance instead of by scanning an *n*-by-*n* matrix. With a captured routing trace loaded, routing weight renders as flow between the gate and the experts it activated, and selecting a single token highlights that token's actual path through the layer. A `ModelAdapter` recognizes common MoE routing patterns automatically on import, so the Expert Galaxy is available immediately for a recognized architecture; an unrecognized architecture still opens and is fully inspectable through the fallback adapter, just without the specialized MoE views. ### Multiresolution rendering and streaming \[#multiresolution-rendering-and-streaming] A tensor IDE that renders one object per parameter stops working somewhere in the low millions of elements, long before a real model's scale. Tensormorph is built around level-of-detail (LOD) streaming instead, on a hierarchy that mirrors the address space itself: ```text Model summary → Layers → Modules → Tensors and experts → Blocks and tiles → Visible scalars ``` Every tensor carries a fixed set of **tile statistics** — moments, extremes (including absmax), L1/L2 norms, NaN/Inf/zero counts, and a log-scale histogram — computed once during indexing and reused by every view that shades, colors, or summarizes a tensor. These statistics are **monoid-mergeable**: a coarser region's statistics are computed directly from its constituent finer tiles', without re-reading raw values. That's what makes an overview of an entire layer cheap regardless of scale — zooming out means reading a coarser, already-correct level of the same statistics pyramid, not loading less data and hoping it's representative. A fixed compute-scheduler priority — cursor scalar, then current selection, then visible tiles, then near-visible prefetch, then explicit analysis, then saved queries, then background indexing — keeps panning and zooming responsive before you've finished navigating. Some questions can't be answered from mergeable statistics alone — effective rank, principal components, pairwise expert similarity — and those are **explicit analysis**: requested deliberately for a specific selection, not computed automatically for every tensor in a large model. Streaming extends to how a checkpoint opens in the first place: a format like `.safetensors` stores a small JSON header up front, so Tensormorph can populate the entire tensor tree — every name, shape, and dtype — from one small read, before a single byte of weight data is fetched. Trillion-parameter interactivity is the design goal this architecture is built toward; it's an architectural property of 1.0, not a specific benchmark result published with this release. ### Comparison and morphing \[#comparison-and-morphing] Once two checkpoints are loaded as an aligned pair, every view in Tensormorph gains a comparison mode — a dedicated **Diff** view spatially, a Diff shading mode in the 2D view, and a Delta sub-mode in the 1D view — rather than requiring a separate comparison-specific tool. Comparison works at two levels: - **Numeric diff** — a per-element $\Delta W = W_B - W_A$ comparison between two aligned tensors, rendered with one consistent diverging convention throughout Tensormorph: cyan where the candidate increased relative to baseline, red where it decreased, dark where nothing changed. - **Semantic diff** — a best-effort classification of *why* two tensors differ, distinguishing a change that looks like genuine signal from one that looks like quantization rounding or an inconsequential expert-index permutation. This layer sits on top of the numeric diff and never replaces it; the raw heatmap is always available even when nothing about a difference can be further characterized. Before any diff runs, the two checkpoints must be **aligned** — matched tensor-for-tensor by name and shape. For two revisions of the same architecture this is usually direct; for architecturally different checkpoints, alignment is where a comparison most commonly goes wrong, so Tensormorph surfaces it explicitly rather than silently guessing. Building a new model from weight-space operations on existing ones — interpolating between two checkpoints, merging a fine-tuned delta into a different base, combining several fine-tunes into one — is what Tensormorph calls **morphing**. A morph is authored as an expression DAG over model and tensor inputs and only becomes actual output weights when you materialize it; until then it's a small, diffable description, not a checkpoint. A materialized result is written out as an ordinary checkpoint, with the DAG that produced it saved alongside as `recipe.json` so it can be re-materialized later against an updated base. Morphing's expression-DAG model and materialize step are stable in 1.0. The specific catalog of operation types — task-vector addition, linear and spherical (SLERP) interpolation, selective merging in the spirit of TIES or DARE — is still being finalized, so treat the exact set of available operations as evolving rather than frozen. Model version history itself isn't something Tensormorph stores. **Lob**, a separate tensor-native version-control system, is what actually holds weight history; a checkpoint loaded through Lob carries a **commit pin** back to the exact commit and content hash it was materialized from, and Tensormorph flags a **stale-note** if Lob's history moves past that pin rather than silently comparing you against outdated state. Reviewing a proposed weight change means loading a Lob-side **weight-space pull request** as an ordinary comparison pair and applying the same diff tooling described above — the review *content* is fully supported today; a dedicated approve/request-changes UI, and Lob's own branch and merge semantics, are still being defined. ### GPU-accelerated analysis \[#gpu-accelerated-analysis] Tile statistics, histograms, norms, diff calculation, quantization-error analysis, heatmap generation, similarity computation, and LOD aggregation all run through one scheduler, which places each operation on whichever available backend can serve it at the lowest cost given where the data currently sits. Tensormorph 1.0 ships this scheduler against five backends — **CPU, wgpu/WebGPU, CUDA, ROCm/HIP, and Metal** — with an `AcceleratorBackendPlugin` interface for adding another accelerator without changing core scheduling logic. Kernel coverage isn't identical everywhere — low-bit quantized operations in particular aren't uniformly implemented across all five yet — so where a backend lacks a kernel, Tensormorph falls back to a slower but correct one rather than failing the request; trading an exact result for an approximation to fit a memory budget is always surfaced explicitly, never substituted silently. CPU is a Tier-1 backend, not a degraded fallback for machines without a GPU. Indexing a very large checkpoint is bandwidth-bound on storage reads, not compute-bound, so a GPU doesn't meaningfully speed up that phase — GPU backends earn their keep in interactive rendering and statistics-refinement, where parallel throughput on already-resident data matters. ### Open formats and extensibility \[#open-formats-and-extensibility] Tensormorph reads `.safetensors` files (including sharded checkpoints with an index), Hugging Face Hub references by revision, and ONNX graphs, with recognition handled by a `ModelAdapter` per format or model family and a generic fallback adapter for anything unrecognized. Alongside these, Tensormorph produces its own small family of derived artifacts: an indexed tensor table (`.tmidx`), a level-of-detail tile pyramid (`.tmtiles`), precomputed A/B statistics (`.tmpair`), a captured runtime profile (`.tmrun`), and a project file (`.tmproj`) holding bookmarks, saved queries, and workspace layout. Every one of these except `.tmproj` is a derived cache, regenerable from source — safe to exclude from version control, and safe to delete for the disk space back. Extensibility runs through a versioned plugin ABI, **`tm-plugin/1`**, with three core traits: `ModelAdapter` for recognizing a new architecture, `RuntimeProbe` for supplying live data from a new runtime source, and `TensorMetric` for adding a new computed statistic or shading mode. Plugins run sandboxed in WASM by default — portable, no separate trust decision per plugin — or through a signed native path for workloads, like custom CUDA kernels, a sandbox genuinely can't support. The `tm-plugin/1` ABI and its three core traits are the stable, documented extension surface for 1.0. A public plugin marketplace or signed-publisher registry is not part of this release — plan on sharing a plugin directly within your team for now. ## Stability in 1.0 \[#stability-in-10] "1.0" means a narrower thing than "finished": the conceptual contracts a workflow, integration, or plugin can be built against without expecting them to shift underneath you. - **Tensor resource identity** — every node in the Semantic Model Graph has a stable address, independent of the human-readable path used to display it. - **The model hierarchy** — Model → Architecture → Layer → Module → Tensor → Tile/Block → Row/Col/Channel → Scalar is the address space every view, query, and plugin operates over. - **Selection semantics** — one shared InspectionContext, not a per-view selection model. - **The coordinate and grid system** — the address grid every 3D/2D view snaps to and measures against. - **View synchronization** — a selection made in any one view is guaranteed visible in every other. - **Tiling and LOD boundaries** — the tile as the unit of streaming, statistics, and rendering, and the mergeable-statistics contract that makes LOD work. - **Extension points** — the `tm-plugin/1` ABI and its `ModelAdapter` / `RuntimeProbe` / `TensorMetric` traits. - **Project and workspace representation** — `.tmproj` as the durable, source-of-truth artifact; everything else regenerable. What isn't frozen: the full morph operation catalog, Lob's branch and merge UI, a public plugin distribution mechanism, and the exact rank-4+ projection strategies in the Tensor Volume view — active design areas 1.0 deliberately leaves room for. ```mermaid graph TD A["Checkpoint: safetensors / sharded / Hub"] --> B["ModelAdapter → Semantic Model Graph"] B --> C[InspectionContext] C --> D["3D views: Architecture, Tensor Volume, MoE Galaxy, Diff, Morph"] C --> E["2D: Matrix / Heatmap"] C --> F["1D: Signal / Distribution"] C --> G[Scalar Inspector] B --> H["Tile statistics (mergeable)"] H --> I["LOD pyramid (.tmtiles)"] I --> D I --> E I --> F J["Compute scheduler: CPU / WebGPU / CUDA / ROCm / Metal"] --> H K["Lob (external VCS)"] -. commit pin .-> B ``` | Capability | Status in 1.0 | | ------------------------------------------------------------------ | ------------------------------------------------------ | | Semantic Model Graph, address space, InspectionContext | Ships now | | 3D/2D/1D coordinated views, address grid, overlays | Ships now | | Tile statistics, LOD streaming, explicit analysis tier | Ships now | | `.safetensors`, sharded checkpoints, Hugging Face Hub, ONNX import | Ships now | | CPU, WebGPU, CUDA, ROCm, Metal compute backends | Ships now | | Weight diff, structural diff, quantization diff, alignment | Ships now | | `tmorph index` CLI | Ships now | | `tmorph query` / `validate` / `render` CLI subcommands | Experimental — shape confirmed, not a frozen reference | | Model morphing (expression DAG, materialize, `recipe.json`) | Ships now, operation catalog still expanding | | MoE Expert Galaxy, expert similarity | Ships now, for recognized architectures | | Runtime traces (`RuntimeProbe`, breakpoints, probe levels) | Experimental | | Lob commit pins and stale-notes | Ships now | | Lob review / conflict-resolution UI | Planned — Lob's own branch/merge model is unsettled | | Public plugin registry | Planned | | Tile Server for remote/thin-client viewing | Planned | ## Getting started \[#getting-started] ```bash [INSTALL_COMMAND] [OPEN_MODEL_COMMAND] model.safetensors ``` A first session generally follows the same arc, whether the model is a small single-file checkpoint or a large sharded one: 1. Open a local `.safetensors` file, or paste a Hugging Face Hub reference (`org/model` or `org/model@revision`) — the model tree populates from the header alone, before any weight bytes load. 2. Navigate to a layer, then a module, in the Outliner or directly in the Architecture view — the two stay in sync in both directions. 3. Select a tensor, read its shape/dtype/statistics in the Inspector panel, then switch to the Matrix and Heatmap view for a precise, cell-exact read of its values. 4. Click a cell to open the Scalar Inspector, and note its full address back up to the model. 5. Load a second checkpoint with **Compare With…**, set it against your original as baseline, and read the Diff view's shaded heatmap. 6. Save bookmarks, saved queries, and your current layout to a `.tmproj` file so the session picks up where you left it. For scripting, indexing, and CI use without a display, `tmorph index ` builds the `.tmidx` file a session streams from — see the [CLI and Automation](/docs/cli-and-automation/tensormorph-cli) documentation for what's confirmed there today. ## Migration from pre-1.0 releases \[#migration-from-pre-10-releases] Tensormorph 1.0 is this project's first public release, so there's no prior public version to migrate from, and no breaking-change list to publish alongside it. If you've worked from an internal preview or early prototype build, treat the checklist below as a starting point rather than a confirmed diff: - **Workspace/project schema** — `[CONFIRM_BEFORE_PUBLISHING: pre-1.0 project file compatible with 1.0's `.tmproj` schema, or does it need conversion?]` - **Tensor resource addressing** — `[CONFIRM_BEFORE_PUBLISHING: did any pre-1.0 build address tensors differently than the Semantic Model Graph scheme above?]` - **Renamed commands or view identifiers** — `[CONFIRM_BEFORE_PUBLISHING: list identifiers/names changed on the way to 1.0.]` - **Cache invalidation** — `.tmidx`/`.tmtiles`/`.tmpair`/`.tmrun` are derived caches; re-run `tmorph index` on anything from a pre-1.0 build rather than assuming compatibility. - **Extension API** — `[CONFIRM_BEFORE_PUBLISHING: did `tm-plugin/1` exist in an earlier, incompatible form?]` - **Deprecated experimental formats** — `[CONFIRM_BEFORE_PUBLISHING: list formats a pre-1.0 build read that 1.0 no longer does.]` If 1.0 is the first Tensormorph build you've used, there's nothing to migrate. ## What's next \[#whats-next] 1.0 establishes the foundation; several directions build directly on it without changing the contracts above: - **Runtime activation and KV-state inspection**, extending today's static-checkpoint tooling to a live forward or backward pass, built on the confirmed `RuntimeProbe` trait and probe-level model. - **Deeper Lob integration** — in-app resolution and approval for weight-space pull requests and conflicts, once Lob's own branch and merge semantics are finalized — plus Lob maturing as its own project, with Tensormorph continuing to anchor to it via commit pins rather than absorbing it. - **A Tile Server and thin web client** for lightweight remote viewing of a checkpoint's tiles and statistics without a full desktop session — a confirmed but lower-priority roadmap item. - **Collaborative review**, building on the weight-space pull request model already supported today. - **Additional accelerator backends** via `AcceleratorBackendPlugin`, and additional `ModelAdapter`s for architectures not yet recognized out of the box. - **A public plugin distribution mechanism**, once the direct-sharing model in 1.0 shows where a registry would help. - **Custom high-dimensional projection strategies** for rank-4+ tensors in the Tensor Volume view, beyond today's selector and small-multiples approaches. None of these are commitments with dates attached — they're the directions 1.0's architecture was built to support. ## Thank you \[#thank-you] Tensormorph 1.0 exists because of everyone who used a rough build, filed an issue, argued about where a boundary should sit, or explained how a real checkpoint is actually structured in practice. Thank you to our early users, to the open-source contributors who worked on the application and its documentation, to the researchers and model authors whose work this tool exists to make more legible, and to the scientific-visualization, GPU, and compiler communities whose prior work this project builds on — including `[CONTRIBUTOR_NAMES]` and our `[CONTRIBUTOR_COUNT]` contributors. - Documentation: \[] - Repository: \[] - Download: \[] - Discussions: \[] - Changelog: \[]