← Back to Glossary
LoRA / QLoRA
Low-Rank Adaptation / Quantized Low-Rank Adaptation
Parameter-efficient fine-tuning methods that freeze main model weights and only train tiny low-rank adapter matrices.
Why it exists
Full fine-tuning of 70B models requires massive GPU clusters. LoRA reduces trainable parameters by 99% so fine-tuning runs on a single consumer GPU.
How it works
Freezes main weights W and adds trainable rank-decomposition matrices A and B. QLoRA quantizes W to 4-bit while training adapters in higher precision.
Related terms
Where you'll use this
roadmap