Chunking Strategies: Five Approaches — Fixed, Recursive, Structure-Based, Parent-Child, and Semantic — and Choosing by Evaluation, Not Intuition
Chunking is the step in the whole pipeline most often decided by gut feel, yet it affects results the most. Today implement five mainstream chunking approaches one by one, explain where each fits and fails, cover what overlap and contextual chunk headers are for, then hold all five up against the same ruler with a fixed set of questions.
Today's Goals
- Explain the assumptions and failure scenarios of fixed-length, recursive, structure-based, parent-child, and semantic chunking
- Explain what problems overlap and contextual chunk headers each remedy, and state the cost of each
- Design a set of questions that quantifies retrieval hit rate across different chunking approaches, and decide with data instead of intuition
Yesterday you unified PDF, HTML, and Markdown sources into one stream of nodes with heading paths, clearing the parsing stage. Today answers the question immediately after: at what granularity and where should that stream be cut so it can be retrieved? Come back and tick off the three goals.
Plain-Language Walkthrough
One: chopping vegetables — too large and the seasoning cannot get in, too small and you cannot taste what the dish is
My family's braised pork has an iron rule: cut the meat into two-centimetre cubes. Cut it at five and the outside is salty while the inside is still pale, with the sauce unable to get in; cut it at half a centimetre and it takes the flavor, but after braising the pot is full of shreds and you cannot tell belly from loin in what you pick up.
Chunking faces the same contradiction with different names on each end.
Too large and the seasoning cannot get in. Retrieval compares the question against a chunk. A two-thousand-character chunk may contain one sentence relevant to the question with over nineteen hundred characters of noise diluting that signal. And even when retrieved, what goes into the model's context is those two thousand characters — you paid a whole chunk's price for one sentence and took up room another chunk could have used.
Too small and you cannot taste what the dish is. A thirty-character chunk reads "the retention period is 30 days, after which it is permanently deleted." Precise in itself, and whose retention period? The recycle bin's? Logs? Backups? The moment the chunk was cut, it lost contact with its context and became an isolated statement nobody can place. Retrieval finds it and the model cannot explain it.
So chunking is never picking a number but finding a position between two quantities pulling against each other: signal-to-noise ratio and completeness. Which is why online advice like "just use 512 tokens with 50 overlap" all looks right and none of it works quite right — it omits the preconditions: what kind of document, what kind of question, and how much context budget retrieval was given.
Today's job is concrete: implement five mainstream approaches behind one interface, cut the same corpus with each, and measure their hit rates with the same set of questions. After today, how large a chunk should be is not a matter of opinion for you but the output of one command.
Interview questions
How do you decide on chunk size? Name two metrics you would look at, and one counterexample.你怎么决定切块大小?说出你会看的两个指标和一个反例。
Common in ChinaCommon overseasIntermediate#chunking#evaluation🔒 Unlock this chapter to see the answer keys.
What does parent-child chunking buy you, and when does it slow the system down instead?父子切块的收益是什么?它在什么情况下反而会拖慢系统?
Common in ChinaCommon overseasIntermediate#chunking#parent-child🔒 Unlock this chapter to see the answer keys.
What overlap ratio would you use, and what concretely goes wrong when the overlap is too large?重叠区设成块长的百分之多少合适?重叠过大会带来什么具体问题?
Common in ChinaCommon overseasBasic#chunking#overlap🔒 Unlock this chapter to see the answer keys.
Semantic chunking costs considerably more than recursive splitting. How would you prove to your team that the money is well spent?语义切分比递归切分贵不少,你怎么向团队证明这笔钱值得花?
Common in ChinaCommon overseasDeep dive#chunking#evaluation#cost🔒 Unlock this chapter to see the answer keys.
