← Back to Glossary
Chunking
Splitting long documents into smaller text segments so retrieval can pull exact paragraphs rather than entire files.
Why it exists
Embedding an entire 50-page book into one vector dilutes specific details. Chunking keeps search precise.
How it works
Splits text by character counts, token limits, paragraph breaks, or semantic shifts (e.g. 500-token chunks with 50-token overlap).