FLOPs
A FLOP (floating-point operation) count estimates the raw computational cost of an operation from its shapes alone, without needing to actually run it - useful for comparing the cost of two architectures, or two morph candidates, before committing to running either.
Estimating from graph structure
Because an imported model is read into a graph of typed operations (see Model, Architecture, Layer and Module and, for graph-native formats, ONNX), a FLOP estimate for an operation follows directly from its type and its input/output shapes - a MatMul of an [M,K] matrix against a [K,N] matrix costs on the order of 2*M*K*N FLOPs, for instance. Because shape inference can populate every edge's shape before a model ever runs, a whole-graph FLOP estimate is available immediately on import, the same way shape labels are.
Where this is useful
- Comparing morph candidates - two candidates that are numerically similar can still have very different inference cost if one changed a layer's dimensions; a FLOP comparison surfaces that immediately, without needing a runtime benchmark.
- Spotting disproportionate cost - a Layer Profile of estimated FLOPs per layer can show that one unremarkable-looking layer is responsible for a large share of total compute cost.
- Sanity-checking quantization or pruning gains - a FLOP estimate before and after a compression pass gives an expected speedup figure to check actual measured performance against.
FLOPs are an estimate, not a measurement
A FLOP count describes theoretical arithmetic cost, not actual wall-clock time - real performance depends on the backend, memory bandwidth, and how well an operation's shapes fit the hardware's execution model. See GPU Kernels and Performance Baselines for measured, rather than estimated, performance.
Related resources
- Model Morphing - comparing compute cost between morph candidates.
- Performance Baselines - comparing measured, rather than estimated, performance.
- ONNX - the graph structure a FLOP estimate is computed over for graph-native formats.