Shape, Rank, Stride and Layout
torch.Tensor API reference.Every tensor Tensormorph inspects carries the same fixed set of metadata, regardless of source format: shape, rank, dtype, device, and layout (including stride). This vocabulary is what the Scalar Inspector and Tensor Inspector panel speak natively.
Shape and rank
- Shape - the size of each dimension, e.g.
[batch, heads, seq, seq]for an attention score tensor. - Rank - the number of dimensions (4, in that example). Rank alone doesn't tell you what each axis means - a rank-3 tensor in a vision tower's patch embeddings and a rank-3 tensor in a text tower's token sequence are structurally identical but semantically unrelated; see Multimodal Models for why that distinction matters.
Storage and stride
A tensor's actual values live in a flat, one-dimensional storage buffer. Stride is what maps a multi-dimensional index back onto a position in that flat buffer - the number of storage elements to skip to move one step along each dimension. A tensor's shape and stride together are what let two entirely different-looking tensors share the same underlying storage.
Views vs. copies
A view shares storage with its base tensor - no data is copied, and editing through a view mutates the base tensor. A copy has its own independent storage.
This distinction is why Tensormorph's overlay system includes a dedicated stride-marker overlay: visually flagging "this tile is a view, not a copy" prevents an easy mistake - editing what looks like an isolated tile but is actually aliased into a much larger tensor. A tensor stops being contiguous (storage laid out exactly as shape/stride would suggest with no gaps) under some view operations, which matters for anything that needs to read a tensor's bytes as one unbroken run, like a fast-path export.
Dtype, device, and layout
- Dtype - the element type (float32, bfloat16, int8, and so on), orthogonal to shape and stride. See Dtype and Quantization for quantized dtypes specifically.
- Device - where the tensor's storage actually resides (CPU, a specific GPU) - see GPU Backend Compatibility.
- Layout - how a tensor's data is physically arranged (dense/strided by default; sparse layouts are a distinct case - see Sparsity).
Related resources
- Tensor, Matrix, Block and Scalar - where this metadata sits in the address space.
- Shards and Storage - extending storage across multiple devices.
- ONNX - a format whose tensors are always dense, with no stride concept at all.