# Shards and Storage

[Shape, Rank, Stride and Layout](/docs/core-concepts/shape-rank-stride-layout) covers a tensor whose storage lives in one place. This page covers what happens once storage is split across files or devices - the foundation [Sharded Checkpoints](/docs/working-with-models/sharded-checkpoints) builds on.

## Two independent kinds of splitting

-   **File-level sharding** - a checkpoint's tensors spread across multiple files purely for manageable file size, with an index mapping each tensor name to its file. Every tensor still has one complete, unsplit storage; only its *file location* is split.
-   **Placement sharding** - a single logical tensor's storage genuinely divided across devices, described by a **device mesh** (the logical grid of ranks/devices) and a **placement** per mesh dimension:
    -   **Shard** - each device holds a distinct slice.
    -   **Replicate** - every device holds the full tensor.
    -   **Partial** - each device holds a partial value awaiting a reduction across devices.

## Local vs. full views

A placement-sharded tensor has both a local view (only the data resident on one device) and a full view (the reassembled logical whole). Tensormorph's views default to showing the full, seamless tensor without necessarily materializing it entirely on one device - reassembly happens as part of rendering, not as a separate manual step.

Requesting the full view of a very large placement-sharded tensor is a real cost, gated by the same [exactness-never-silently-degrades](/docs/reference/gpu-backend-compatibility#exactness-never-silently-degrades) principle as any other expensive request - Tensormorph tells you when it's substituting an approximation rather than doing so silently.

## Why storage splitting doesn't affect diffing

A [weight diff](/docs/compare-and-morph/weight-diff) always compares full, reassembled logical tensors - never raw per-shard or per-file bytes - specifically so that a change in *how* a checkpoint is split (a different shard count, a different device mesh) never shows up as a spurious difference between two checkpoints that are otherwise numerically identical.

## Related resources

-   [Sharded Checkpoints](/docs/working-with-models/sharded-checkpoints) - the format-level, applied continuation of this page.
-   [Level of Detail and Streaming](/docs/core-concepts/level-of-detail-and-streaming) - how partial/lazy loading interacts with sharded storage.
-   [GPU Backend Compatibility](/docs/reference/gpu-backend-compatibility) - reassembling a sharded tensor across a heterogeneous set of backends.