Knowledge / Field guide
Keep a RAG index current when policies change
A new handbook may be searchable while its retired predecessor still answers questions. Plan the removal path alongside the upload path.
2XD Studio · · 2 min read
Give each source a durable identity
Keep a source ID independent of its file name. Store the version, owner, effective date, and access rules. A renamed file should update its existing entry; a policy with a future effective date should not quietly displace today's instructions.
Test a replacement and a deletion
In a pilot library, change a return window from 30 days to 14. Ask the question before and after the update and inspect the citations. Then delete the test policy and repeat. Check the index, answer caches, and any precomputed summaries. Removing the source document proves little if a cached answer still quotes it.
Set a freshness budget
Ask how long the team can tolerate an old answer. An annual reference guide may refresh overnight; a dispatch procedure may need a faster change signal. Show the last successful synchronization and alert an owner when it falls behind. For high-impact changes, provide a way to temporarily suppress the affected source while the index catches up.
Check the date that matters
A policy uploaded Friday may take effect Monday. A system that always picks the newest upload can give the wrong answer all weekend. Store the effective interval separately from the upload time, and ask the same test question for both dates. That also makes it possible to explain why a customer correctly received different instructions last week.
Reader questions
How do we handle a failed refresh?
Keep the last known state visible to operators. If stale material could cause harm, suspend answers from that source until it is reconciled.
Can we keep old versions for audit?
Yes. Separate the archive from the collection used for current answers and make historical retrieval an explicit action.
Further reading Microsoft Learn: retrieval-augmented generation