← Back to Glossary
Quantization
Compressing model weights from high precision (16-bit floats) to lower precision (8-bit or 4-bit integers) to shrink memory size and boost speed.
Why it exists
Large models require substantial VRAM. Quantization slashes memory requirements by 2x to 4x with minimal accuracy loss.
How it works
Maps 16-bit floating point numbers to discrete 4-bit or 8-bit integers (e.g. GPTQ, AWQ, GGUF), reducing GPU memory bandwidth bottlenecks.
Related terms
Where you'll use this
roadmap