The newest file is not always the governing document. A draft uploaded this morning may be less authoritative than an approved policy from last month, while a regional exception can override both for a particular customer. An AI agent needs a way to select the applicable version before it uses a passage as evidence. Semantic similarity alone cannot make that decision.
Product documentation and current access details were checked on October 11, 2026. The workflow examples are proposed evaluation designs, not reports of completed customer deployments.
- 01Separate revision from effective dateA document can be newly edited while describing a future or historical policy.
- 02Make scope explicitRegion, product and customer class can determine which rule applies.
- 03Filter before using evidenceOnly eligible document versions should compete to answer the current question.
- 04Expose unresolved conflictsAn agent should ask for the missing decision rather than silently choose a convenient source.
01 — Applicable scopeStart with the question the document must answer
A fictional support team asks whether an order delivered in September can still be returned. Its shared drive contains an approved September policy, an October draft, a regional addendum and an old training slide. All four discuss returns and may rank well in a semantic search. Only some can govern this customer’s question, and the answer depends on the relevant date and region.
Write the applicability question before searching for a passage. Is the user asking about today’s policy, the policy when the order was placed, or the policy attached to a particular contract? Those are different requests. If the business rule uses order date rather than delivery date, the retrieval system must carry that distinction into the search. The model should not quietly invent the date that makes its first result fit.
This guide focuses on competing internal versions. Our stale-web-evidence guide covers how to interpret dated external claims; the problem here is that several company documents can all be recent, accessible and relevant while carrying different authority. Resolve that authority explicitly rather than asking the model to infer organizational policy from filenames and confident prose.
Which rule applies now?
Select an approved version whose effective period and business scope match the request.
Which rule applied then?
Select the version governing the requested period and make that period visible in the answer.
02 — Document metadataStore the facts that make a version eligible
Give each governed document a stable family identifier and a distinct version identifier. The family groups revisions of the same policy; the version identifies the exact text used. Add approval status, effective period, owner and scope where those attributes affect the business decision. These fields should come from a maintained process, not from a language model guessing what the document probably means.
Separate file modification time from policy effective time. A typo correction can change a file today without changing the policy’s start date. A policy approved today may begin next month. Likewise, an archived document may remain the right evidence for a historical dispute. One timestamp cannot represent all of those facts without producing misleading answers for at least one type of question.
Start with fields your organization can maintain reliably. A complicated schema filled with guesses is worse than a small, accurate one with visible gaps. If the approval owner is unknown, record that uncertainty and define which questions can still be answered. The document-reading evidence guide is a useful companion because accurate extraction from a page does not establish that the page is entitled to govern the answer.
For the fictional policy family, imagine version 3 is approved on September 20 and becomes effective October 1. A version 4 draft is uploaded October 5 for discussion, while a regional addendum to version 3 begins October 8. A current question from that region needs version 3 plus the addendum. A question about September 25 may need version 2. The file uploaded October 5 wins neither question merely because it is newest. Writing this example down gives the team an unambiguous acceptance case before it chooses a search technology.
| Field | Purpose | Example value |
|---|---|---|
| Document family | Group related revisions | Customer returns policy |
| Version and status | Identify approved text | Version 3, approved |
| Effective period | Select the relevant time | Starts October 1; prior version retained |
| Business scope | Apply the correct exception | Specified region and product group |
| Owner | Resolve authority conflicts | Customer operations policy owner |
03 — Retrieval sequenceUse search filters for eligibility, ranking for relevance
Qdrant’s filtering documentation illustrates the distinction between similarity search and explicit payload constraints. A search can require matching metadata instead of treating every similar passage as eligible. The same general design is useful for policy retrieval: establish the allowed versions and scope, then rank relevant passages within that boundary.
A filter is only as reliable as the metadata it receives. If a draft was incorrectly marked approved during ingestion, the search engine cannot repair the organizational mistake by applying the filter correctly. Keep the source record and indexing process connected, and inspect representative indexed records after a policy update. A working query is not proof that the indexed status matches the source of truth.
Avoid relying solely on a prompt that tells the agent to ignore old documents. By the time the model receives a mixture of approved and superseded chunks, it must solve an authority problem that the application could have made explicit earlier. Still preserve enough document identity in the returned evidence for the model and reviewer to verify the selected version. Filtering should clarify the evidence boundary rather than make the selection invisible.
A document can be the governing version and still be inaccessible to the current user. Version filtering does not replace permission checks on retrieval and citations.
04 — Precedence designKeep local exceptions attached to their parent rule
A regional addendum can be newer than the main policy without replacing it everywhere. Model this as an explicit relationship: which rule it modifies, for which scope and during which period. The retrieval process may need both the base policy and the exception. Selecting only the most specific document can be incomplete if the exception assumes definitions supplied by the parent.
In the fictional returns example, a regional addendum might change the return window while leaving the packaging condition unchanged. An answer that quotes only the longer window can still omit a relevant requirement. The agent should identify the base rule and the applicable modification, then explain the combined result in ordinary language. It should not average conflicting periods or choose whichever document contains the clearest sentence.
Ask the policy owner to define precedence when two approved documents overlap. A technical team can implement a rule, but it should not invent one because retrieval requires a deterministic outcome. If the business has not settled the overlap, the assistant should surface that gap. A visible unresolved exception is more useful than a confident answer that quietly becomes an unofficial policy decision.
- Link the exception to the document family it modifies.
- Record the exact scope and effective period.
- Keep unchanged parent conditions available with the exception.
05 — Chunk lineagePreserve version identity down to the returned passage
Document splitting creates a second place for version identity to disappear. A useful passage may be stored separately from its title page, approval block and effective date. Carry the document family, version and source location into each indexed chunk. Otherwise, a search result can be semantically accurate while leaving the agent unable to tell which revision produced it.
When a policy is superseded, update eligibility for every relevant chunk, not just the document’s top-level listing. Test that an old passage cannot remain in the current-answer pool because a deletion or metadata update only reached part of the index. Preserve the historical version if the business needs historical answers, but make that a deliberate query path rather than an accidental mixture.
The stale-data read-path guide explains why a current source and an old searchable copy can disagree. For document versions, the practical check is to compare the governing record with the exact chunks returned for a known question. A citation link to the current document does not repair an answer that was generated from an older indexed passage.
A revealing indexing failure occurs when a document’s first chunk receives the new approval status but later chunks retain the old status. An ordinary question about the introduction may pass while a question about an exception retrieves superseded text. Test at least one question whose answer comes from a later section and inspect its lineage. The fix belongs in metadata propagation or indexing, not in a prompt that asks the model to distrust long documents. This is why the source-to-chunk relationship must be reviewable.
After replacing a policy, ask a question whose answer changed. Inspect the retrieved passage and its version identifier before judging the final response.
06 — Useful uncertaintyHandle conflicts without making the user do the search
An unresolved conflict should produce a useful next step. Tell the user which fact is missing and why it affects the answer. For example, the assistant may need the order’s region, or it may have found two approved policies with overlapping effective periods. These require different actions: ask the customer for a detail in the first case, and ask the policy owner to resolve authority in the second.
Do the independent work first. The agent can identify the relevant document family, collect the applicable passages and prepare the alternatives before requesting a decision. It should not simply announce that documents conflict and leave the user to repeat the search. A concise comparison of the two rules, their scope and their owners makes the missing decision concrete.
Also limit what can proceed while the conflict remains. Preparing a draft explanation may be acceptable; committing to a refund under an uncertain rule may not be. The application should preserve this distinction instead of treating every unresolved source issue as either total failure or permission to guess. The resulting behavior is both more helpful and easier for the team to maintain.
- State the unresolved fact or authority conflict.
- Show the candidate rules and their practical difference.
- Name the person or source that can settle the question.
07 — Acceptance casesBuild a version test set with deliberate contradictions
Create a small fictional document family specifically for testing. Include an approved current policy, a later draft, a future approved policy, a superseded version and a regional exception. Write questions whose expected source is known. This makes it possible to test the selection process independently from the model’s ability to summarize polished prose.
Include traps that resemble ordinary business files. Rename the draft with the word final. Give the superseded version a recent file modification time. Put an attractive concise answer in a training slide that has no policy authority. The system should continue selecting the governing document based on maintained metadata and scope, not on a filename or the convenience of a passage.
Score source selection separately from answer wording. A correct-looking answer from the wrong version is a failure because it may break on the next question. Our AI transformation service can help turn this test into an operating check for a document assistant. The examples here are proposed tests; they are not a claim that a particular retrieval product has already passed them.
Have the expected-answer record name both the governing version and the relevant passage. This prevents an evaluator from awarding full credit because two versions happen to give the same answer to an easy question. Add a question where the versions disagree, and another where the exception changes only one condition. The evaluation should reveal whether the system selected authority correctly before it summarized the text. When the policy owner changes the precedence rule, update the answer key deliberately and retain the reason for that change.
Ask the same policy question for two different effective dates. The answer should change only when the governing rule changes, with the selected period and version visible.
08 — Ongoing operationMake version ownership part of the publishing process
The system will drift if version metadata is treated as an optional cleanup after publication. Assign responsibility for approving a document, setting its effective period and marking what it supersedes. The retrieval team needs a dependable signal when those fields change. A policy that is approved in an email but remains marked draft in the source system creates ambiguity before any AI model sees it.
Keep a practical exception queue for missing metadata and conflicting approvals. Review the questions that the assistant could not answer because authority was unclear. They often reveal a document-management problem that affects people too. Fixing the underlying record can improve every future answer more reliably than adding another sentence to the prompt.
The goal is not to make every document library perfectly tidy before using AI. It is to identify which documents can govern which decisions, preserve that information through retrieval and expose the gaps that remain. Once the team can explain why a particular version was selected, the assistant has a much stronger basis for producing an answer that another person can trust.
- Assign an owner for version status and scope.
- Propagate changes into searchable chunks.
- Review unresolved authority conflicts as document-maintenance work.
Test one policy family before expanding the library
Choose a document whose revisions change a real business answer. Record its versions, effective periods and exceptions, then test current and historical questions against that family.
A reliable document agent should be able to show why its source applies. The answer becomes easier to trust when authority is carried through the system rather than guessed at the final paragraph.