File Format API
Tensormorph reads several checkpoint formats out of the box (see Working with Models); a new format - a niche quantization container, a custom checkpoint layout - is added by implementing a format importer against the same zero-copy bar every built-in format is held to.
The zero-copy bar
Any format Tensormorph supports should allow: reading a tensor's metadata (name, shape, dtype) without reading its data, and reading a slice of a tensor's data without reading the whole tensor. A format importer that requires loading an entire file to answer "what tensors does this contain" is a materially worse fit than one that doesn't - see Safetensors for the format that sets this bar most clearly.
A format importer that can't meet this bar still works - Tensormorph will still open files in that format - but loses the near-instant tree population and lazy loading every other format gets, so meeting it is worth the extra implementation effort where the source format allows it at all (a header-then-data layout like safetensors' makes this straightforward; a monolithic pickle-style format makes it difficult or impossible).
What an importer provides
- Header parsing - enumerate every tensor's name, shape, dtype, and byte location from metadata alone.
- Slice-addressable reads - fetch a specific tile's bytes without reading surrounding data.
- Metadata mapping - where the format has its own metadata convention (ONNX's
metadata_props, for instance), map it into Tensormorph's minimal always-present metadata fields; see Minimal vs. rich metadata.
Activation on demand
Like other plugin types, a format importer only loads and runs when a matching file is actually encountered - declaring which file signatures or extensions it handles is enough for Tensormorph to route the right files to it without loading every installed importer's code at startup.
Related resources
- Safetensors - the reference example for meeting the zero-copy bar.
- Extension API - the plugin ABI a format importer is registered under.
- Custom Tensor Adapters - the next step after import: recognizing the checkpoint's architecture.