Recursive vs semantic splitting

The most common chunking method is recursive splitting. It's the industry standard because it's predictable and works well with structured text. It tries to split at paragraphs first, then sentences, then words, respecting natural text boundaries. To try it, we'll borrow the SentenceSplitter node parser from LlamaIndex, a RAG framework you'll use to build a full pipeline soon.

04_chunking.ipynb
python
from llama_index.core.node_parser import SentenceSplitter
from llama_index.core import Document

kitchen_manual = """
Chapter 1: Food Safety

All vegetables must be washed in cold water for at least 30 seconds.
Cross-contamination prevention: use separate cutting boards for
different ingredient types. Store raw ingredients below 4 degrees C.

Chapter 2: Cooking Temperatures

Grilled vegetables should reach an internal temperature of 75C.
Soups and stews must be brought to a rolling boil before serving.
"""

splitter = SentenceSplitter(
    chunk_size=250,     # Max characters per chunk
    chunk_overlap=30    # Overlap between consecutive chunks
)

nodes = splitter.get_nodes_from_documents(
    [Document(text=kitchen_manual)]
)

for i, node in enumerate(nodes):
    print(f"--- Chunk {i+1} ({len(node.text)} chars) ---")
    print(node.text[:100] + "...")

Recursive splitting with LlamaIndex SentenceSplitter

That is exactly what chunk overlap solves. Without overlap, a sentence at the boundary gets cut in half and neither chunk has the full meaning. With overlap, the end of one chunk is repeated at the start of the next, so the full sentence appears in at least one chunk.

The chunk_overlap parameter is your insurance against broken context. It repeats the last 30 characters of chunk N at the start of chunk N+1. This way, if a sentence spans a chunk boundary, the full context appears in at least one chunk.

Semantic splitting takes a different approach. Instead of splitting by structure, it uses embeddings to detect where the topic changes. It reads through the document and creates a new chunk whenever it detects a significant shift in meaning.

04_chunking.ipynb
python
from llama_index.core.node_parser import SemanticSplitterNodeParser
from llama_index.embeddings.litellm import LiteLLMEmbedding

embed_model = LiteLLMEmbedding(model_name=EMBEDDING_MODEL)

semantic_splitter = SemanticSplitterNodeParser(
    buffer_size=1,
    breakpoint_percentile_threshold=95,
    embed_model=embed_model
)

semantic_nodes = semantic_splitter.get_nodes_from_documents(
    [Document(text=kitchen_manual)]
)

for i, node in enumerate(semantic_nodes):
    print(f"--- Semantic Chunk {i+1} ({len(node.text)} chars) ---")
    print(node.text[:100] + "...")

Semantic splitting detects topic boundaries using embeddings

AI prompt: Try it with AI

Loading practice…

Quiz: Quiz

Loading practice…

Matching exercise: Match chunking concepts

Loading practice…