Why Do Answers Start to Conflict as a RAG Knowledge Base Grows?
Two Answers Before the Morning Meeting
Twenty minutes before the morning meeting, operations lead Xiao Zhou is asked a very specific question: when the same database connection spike alert appears again today, should the on-call team restart the service first, or isolate traffic first?
The team's knowledge base quickly finds two answers. A runbook from three years ago says, "restart and observe." Last month's incident review, however, records that "for this kind of symptom, limit traffic first; restarting directly will expand the impact."
Neither document is nonsense. The first corresponds to an earlier architecture, while the second comes from handling experience after dependency relationships changed. But when both are retrieved in the same search, Xiao Zhou does not get an answer. He gets two choices that both look like answers.
