GPU Backend Compatibility
Tensormorph runs its indexing, statistics, and rendering compute across five backends: CPU, wgpu/WebGPU, CUDA, ROCm/HIP, and Metal - with additional accelerators addable through an AcceleratorBackendPlugin interface rather than requiring core changes for each new one.
CPU is first-class, not a fallback
CPU (specifically high-core-count server CPUs, e.g. EPYC-class) is a Tier-1 backend, not a degraded path used only when no GPU is available.
Indexing a multi-terabyte checkpoint is bandwidth- and NVMe-bound, not compute-bound - the bottleneck is reading bytes off storage, not the arithmetic performed on them. A GPU doesn't meaningfully speed up that phase, so treating CPU indexing as a lesser fallback would misrepresent where the actual cost lives. GPU backends earn their keep in the interactive rendering and statistics-refinement paths, where parallel throughput on already-resident data matters.
How backend selection works
Backend placement is driven by capability and cost, never by branching on a hardware name. A given operation (compute a tile's statistics, render a shading pass, run an explicit analysis) is scheduled to whichever available backend can do it and is cheapest to reach given where the data currently sits - not hardcoded to "use CUDA if present." This is what lets the same model, the same view, and the same query behave consistently whether you're on a workstation with an NVIDIA GPU, an Apple Silicon laptop, or a headless EPYC server with no GPU at all.
The Compute panel is where this is visible and, if needed, overridden. Its default is Compute: Auto; any override you set there is a scheduling constraint ("prefer this backend," "never use that one"), not a different code path - the same operation runs, just placed differently.
Exactness never silently degrades
If a requested computation would need to fall back from an exact result to an approximation (for example, to fit within a backend's memory budget), Tensormorph always surfaces that choice explicitly rather than silently substituting a cheaper answer. The fallback ladder - what the approximate path actually computes instead - is shown, not hidden, and moving from exact to approximate requires your consent.
Format and operation coverage varies by backend
Not every operation is implemented identically across all five backends - low-bit quantized kernels in particular (see Quantized Models) are not uniformly supported everywhere. Where a backend lacks a specific kernel, Tensormorph falls back to a slower but correct backend for that operation specifically, rather than failing the whole request; the Compute panel shows which backend actually served a given operation.
Related resources
- Level of Detail and Streaming - how LOD interacts with what's resident on which backend.
- Sharded Checkpoints - reassembling a placement-sharded tensor across a heterogeneous set of backends.
- Settings - persisting a Compute panel override across sessions.