# Shape, Rank, Stride and Layout

This page is written from general tensor-semantics domain knowledge (the vocabulary is standard across PyTorch and similar frameworks), not adapted from a local reference document - this workspace's local PyTorch checkout covers the pytorch.org marketing site, not the `torch.Tensor` API reference.

Every tensor Tensormorph inspects carries the same fixed set of metadata, regardless of source format: shape, rank, dtype, device, and layout (including stride). This vocabulary is what the [Scalar Inspector](/docs/editors/scalar-inspector) and [Tensor Inspector panel](/docs/editors/architecture-editor) speak natively.

## Shape and rank

-   **Shape** - the size of each dimension, e.g. `[batch, heads, seq, seq]` for an attention score tensor.
-   **Rank** - the number of dimensions (4, in that example). Rank alone doesn't tell you what each axis *means* - a rank-3 tensor in a vision tower's patch embeddings and a rank-3 tensor in a text tower's token sequence are structurally identical but semantically unrelated; see [Multimodal Models](/docs/working-with-models/multimodal-models) for why that distinction matters.

## Storage and stride

A tensor's actual values live in a flat, one-dimensional **storage** buffer. **Stride** is what maps a multi-dimensional index back onto a position in that flat buffer - the number of storage elements to skip to move one step along each dimension. A tensor's shape and stride together are what let two entirely different-looking tensors share the same underlying storage.

## Views vs. copies

A **view** shares storage with its base tensor - no data is copied, and editing through a view mutates the base tensor. A **copy** has its own independent storage.

This distinction is why Tensormorph's [overlay system](/docs/editors/architecture-editor#the-overlay-system) includes a dedicated stride-marker overlay: visually flagging "this tile is a view, not a copy" prevents an easy mistake - editing what looks like an isolated tile but is actually aliased into a much larger tensor. A tensor stops being **contiguous** (storage laid out exactly as shape/stride would suggest with no gaps) under some view operations, which matters for anything that needs to read a tensor's bytes as one unbroken run, like a fast-path export.

## Dtype, device, and layout

-   **Dtype** - the element type (float32, bfloat16, int8, and so on), orthogonal to shape and stride. See [Dtype and Quantization](/docs/core-concepts/dtype-and-quantization) for quantized dtypes specifically.
-   **Device** - where the tensor's storage actually resides (CPU, a specific GPU) - see [GPU Backend Compatibility](/docs/reference/gpu-backend-compatibility).
-   **Layout** - how a tensor's data is physically arranged (dense/strided by default; sparse layouts are a distinct case - see [Sparsity](/docs/inspect-and-analyze/sparsity)).

## Related resources

-   [Tensor, Matrix, Block and Scalar](/docs/core-concepts/tensor-matrix-block-scalar) - where this metadata sits in the address space.
-   [Shards and Storage](/docs/core-concepts/shards-and-storage) - extending storage across multiple devices.
-   [ONNX](/docs/working-with-models/onnx) - a format whose tensors are always dense, with no stride concept at all.