Guides
RAG Chunking Strategies on GitHub 2027: Fix Retrieval Before Changing Models
This article, adapted from the Duaer documentation site, explains why document chunking is often the root cause of weak enterprise RAG results, compares common chunking strategies and their trade-offs, and offers practical methods for evaluation and rollout.
Why Chunking Is Harder Than It Looks
Many disappointing enterprise RAG projects fail not because the model is weak but because documents were cut into fragments that lost their original meaning. A team finds a chunking script on GitHub, runs it over the internal knowledge base, connects a vector database, checks three test questions, gets correct answers, and decides it is ready to launch. Two weeks later, the same system may confidently cite an outdated policy version or fail to find a clause that clearly appears in a PDF. The first instinct is often to switch to a larger model, but in most cases the real problem is upstream: chunking.
Chunking sounds like a minor preprocessing step, yet it determines what the retriever can ever find. A language model can only reason over the chunks it receives. If one chunk contains half of a rule and the exception sits in the next chunk, no model, however capable, can reason about that exception.
Common Strategies and Their Weak Points
Enterprise documents make this problem worse. Contracts rely on terms defined pages earlier. Product manuals place model numbers in headers and specifications in tables. A decision in meeting notes only makes sense alongside the agenda item above it. Tools that split purely by character count treat all of this as interchangeable text, and that is where answer quality starts to slip.
Most open-source repositories implement only a handful of methods. Rather than memorizing names, it helps to understand what each one trades away.
Fixed-length splitting: cut every N tokens, often with some overlap. It is simple and predictable, which makes it a reasonable baseline. Its weakness is that it ignores document structure and frequently slices tables or numbered steps in half.
Recursive splitting: try to split by paragraph first, then by sentence, and finally by character. It is the most common default in community code and works well on clean prose, but it has no awareness that a heading governs the paragraphs beneath it.
Structure-aware splitting: use headings, sections, list items, or page layout to determine boundaries. For policies, manuals, and regulated documents, this is often the best starting point, because authors have already organized the content into meaningful units. The drawback is its dependence on parsing quality. If a PDF parser flattens columns or drops heading levels, the structure-aware logic inherits those errors.
Semantic splitting: group sentences by vector similarity and place boundaries where the topic shifts. This sounds appealing, but it costs more to run, is harder to debug, and its boundaries are not always easy to explain to reviewers. Teams usually adopt it only after a fixed-length baseline fails, and the improvement is real only if the failure actually came from mixed topics within chunks.
Parent-child or small-to-large retrieval: match on small chunks for precision, then return the larger section they belong to. For enterprise knowledge bases this is often the most practical pattern, because it separates two questions: what to match on, and how much context the model should read.
Plausible-Looking Mistakes in Demos
The first mistake is accepting defaults as they are. The chunk sizes in example repositories were chosen for sample documents, not for your contracts, tickets, or product specifications. Treat any default as a hypothesis to be tested.
The second mistake is applying one strategy to every document type. A FAQ page, a 200-page regulation, and a price table should not all go through the same splitter. Even if the first version handles only three document types, route them by type.
The third mistake is tuning against a demo set. If a team picks chunk sizes using five questions whose answers the developers already know, the system is optimized for the team's memory, not for the questions employees will ask next month.
The fourth mistake is dropping metadata. A chunk without its source title, section path, effective date, version number, and access permissions cannot be filtered, cited, or audited. In enterprise settings, missing permission fields are often the most costly, because they turn a retrieval defect into a data exposure problem. For a broader discussion of pitfalls at the build stage, see the Enterprise RAG Knowledge Base Build Guide.
Evaluating Chunks by Retrieval Quality and Running a Feedback Loop
A chunk that looks clean may be useless in retrieval. The right test is whether the correct chunk comes back for the right question, and whether the model's answer is supported by that chunk.
Start with a golden question set of 50 to 150 questions drawn from real users, each paired with the passage that should answer it. Include some hard cases: questions that span two sections, questions that use outdated terminology, and questions where the correct answer is that the policy does not address the matter.
Evaluate retrieval first. Measure how often the reference passage appears in the top k results, and how often it ranks first. Only once retrieval is stable should you assess generation, checking whether each answer is grounded in the retrieved text. If recall is low, changing the model will not fix the problem.
Rerun this question set whenever the splitter, embedding model, or source corpus changes. Chunking adjustments often help one document type while quietly hurting another, and only a fixed test set makes that regression visible.
Chunking strategy is not a one-time setting. Documents change, new teams add data sources, and users ask questions the original test set did not cover. Review failed retrievals from production logs every month, add real failure cases to the golden set, and check whether the current strategy still holds. Much of the lasting value comes from this feedback loop rather than from which splitter was chosen at the start.
Evaluating a GitHub Chunking Tool Before Adoption
An open-source splitter is a good starting point, but it becomes code you will maintain yourself. Before adopting one, check four things. First, the license, and whether it permits internal use without unexpected complications. Second, maintenance activity: recent commits, open issues about parsing edge cases, and whether the maintainers respond. Third, parsing dependencies: many chunking tools depend on the underlying PDF, Word, or HTML parser, and those parsers often fail on scanned pages, multi-column layouts, and embedded tables. Fourth, output format: can it emit section paths, page numbers, and custom metadata, or does it return plain text only.
If the tool cannot preserve metadata, wrap it. A lightweight layer that attaches source, section, version, and access labels to each chunk is worth far more than a sophisticated splitting algorithm that discards that information. Teams that treat the pipeline as a chain of replaceable parts, rather than as one monolithic library, find it easier to change strategies later. The handoffs between parsing, chunking, indexing, and retrieval are where many projects stall, and the article Where RAG Integrations Break walks through the common problems one by one.
What You Can Do This Week
Pick one high-value document set, such as product manuals or HR policies, and compare two chunking approaches on it: a fixed-length baseline and a structure-aware method that respects headings. Build a 50-question golden set from questions real employees have asked, record top-5 recall for each approach, and write down the failures that remain. Before testing anything else, attach source and section metadata to every chunk. This small exercise will teach you more than any leaderboard in a repository README.
If you would rather run this kind of chunking and retrieval comparison without assembling every component yourself, you can test the workflow directly on Duaer. After signing up, you can configure document sources, compare splitting rules against your own questions, and see where retrieval breaks before you commit to a pipeline. Chunking strategy ultimately depends on your own documents, and a tested rule set will always outperform a default copied from elsewhere.
If you tell us about your business scenario, we will send back a written rollout plan tailored to it. Submission form: https://api.duaer.com/form/c59a907a-3697-4edf-a04d-3ea0fb72fdc8
Questions
Should I use fixed-length, recursive, or structure-aware chunking for RAG?
If your documents have clear headings and section structure, such as policies, manuals, and regulated filings, structure-aware chunking is usually the best starting point. Fixed-length chunking works well as a predictable baseline, and recursive splitting is a common default in community code. Whichever you choose, validate it against your own golden question set rather than relying on the default parameters in example repositories.
Can I use an open-source GitHub chunking script directly on internal enterprise documents?
You can use it as a starting point, but first check the license, maintenance activity, the quality of the underlying parser, and whether it can output section paths, page numbers, and custom metadata. If the script cannot preserve metadata, wrap it in a layer that attaches source, version, and access-permission tags to each chunk. Otherwise, filtering, citation, and auditing become very difficult later.
How do I tell whether my RAG chunking is performing well enough?
Start by building a golden set of 50 to 150 questions drawn from real users, and annotate the passage each question should hit. Measure retrieval first, including how often the reference passage appears in the top k results and how often it ranks first. Once retrieval is stable, check whether generated answers are grounded in the retrieved text. Rerun this test every time the splitting rules, embedding model, or source corpus changes.