Choosing the right strategy

There is no perfect chunk size that works for every document. The right strategy depends on your document type, query patterns, and the trade-off between precision and context preservation.

Great question! This is the "Citrus Mystery" problem. Imagine your manual says: "Citrus plant leaves are turning yellow due to acidic soil" in one paragraph, and "Fix by adding wood ash to raise soil pH" in the next. If these land in different chunks, a query about "fixing yellow leaves" might find the problem chunk but miss the solution. This is why chunk overlap and choosing the right chunk size are so important.

If overlap is nearly as large as the chunk itself, most of the content is duplicated across chunks. You end up storing and searching many near-identical vectors, which wastes storage, slows down search, and can return redundant results. A good rule of thumb is to keep overlap at 10-20% of the chunk size.