Chunking Strategies
18 min
A RAG (retrieval-augmented generation) system's quality depends heavily on how source documents get split into retrievable chunks. LLM Fundamentals' fixed-size sliding window is simple, but it can slice a sentence in half at a chunk boundary — a retrieved chunk might then contain a dangling, confusing fragment. Sentence-boundary-aware chunking fixes this by packing whole sentences together, only starting a new chunk when the next sentence would push past the size limit.
def chunk_by_sentences(sentences, max_chars):
chunks = []
current = []
current_len = 0
for sentence in sentences:
added_len = len(sentence) + (1 if current else 0)
if current and current_len + added_len > max_chars:
chunks.append(" ".join(current))
current = [sentence]
current_len = len(sentence)
else:
current.append(sentence)
current_len += added_len
if current:
chunks.append(" ".join(current))
return chunks
Notice a chunk can end up slightly UNDER max_chars if the next sentence wouldn't fit -- this function always prioritizes never splitting a sentence over hitting the size limit exactly, which is the entire point of this strategy over fixed-size windows.
Write `chunk_by_sentences(sentences, max_chars)`: pack a list of sentence strings into chunks (joined with a single space), never exceeding `max_chars` characters per chunk UNLESS a single sentence alone is already longer than `max_chars` (in which case it becomes its own oversized chunk — never split mid-sentence). Return the list of chunk strings.
LLM Fundamentals' `context-windows` module covered fixed-size sliding-window chunking. Why would a RAG pipeline prefer SENTENCE-boundary-aware chunking instead?
You can implement sentence-boundary-aware chunking, trading a little size precision for guaranteeing every chunk stays coherent.