Memory

🚧

This page covers what's confirmed about Tensormorph's memory-residency model. Detailed per-kernel memory-traffic diagrams (cache hit rates, on-/off-chip transfer breakdowns) are not a confirmed part of Tensormorph's design - there is no local NVIDIA Nsight Compute reference in this workspace to draw that level of detail from.

Tensormorph's memory story is mostly the story already told in Level of Detail and Streaming: at any moment, only a small fraction of a large model is actually resident in fast memory, and what's resident is governed by a fixed priority order - hovered scalar, current selection, visible tiles, near-visible prefetch, explicit analysis, queries, then background indexing.

Where to check residency

The Compute panel (see GPU Backend Compatibility) is where backend and, by extension, memory placement is visible for a given operation. Beyond that, this page doesn't go further than what's confirmed - a dedicated memory-analysis view comparable to a kernel-level memory-traffic diagram isn't part of the documented design.

Why this matters at scale

The practical consequence of the LOD/residency model is that Tensormorph's memory footprint doesn't scale with model size the way naively loading a checkpoint would - opening a trillion-parameter model doesn't require a trillion parameters' worth of RAM or VRAM, because most of what's "loaded" at any given moment is coarse statistics and a small working set of tiles, not the full tensor data.

Related resources

Level of Detail and Streaming - the residency and prefetch model in full.
GPU Backend Compatibility - where a given operation's backend (and therefore memory location) is visible.
Sharded Checkpoints - memory considerations specific to a placement-sharded tensor's local vs. full view.

Gradients FLOPs