An employee asks about an internal process, but the answer is spread across a handbook, a procedure and several updates. A general language model does not know which version the company considers current. To provide a useful answer, the application needs to find the relevant material and use it as context.
This approach is commonly called retrieval-augmented generation, or RAG. Understanding the connection between retrieval and generation helps explain why document quality matters and what a team should prepare before building an internal knowledge assistant.
Connecting retrieval with a model’s answer
In a simplified process, the application first searches for document passages relevant to the question. It then gives selected passages to a language model together with the request and response instructions. The model prepares an explanation using that context. The application can include links to the underlying sources.
Retrieval may use keywords, similarity of meaning or a combination of techniques. Sophisticated search still cannot repair contradictory procedures. If two versions of a rule are present without any indication of which applies, the underlying information remains ambiguous.
An example: finding the current procedure
Imagine a company handbook for new employees. Someone asks: “How do I request access to the learning platform?” An older document gives an email address, while the current version directs employees to an internal form. The assistant needs the applicable procedure, rather than whichever document repeats the question’s words most often.
A useful answer in this illustrative scenario includes the steps, a link to the form and an identifiable source. If eligibility depends on the employee’s team, the application uses permitted context or asks a follow-up question. It should not invent a rule absent from the documents.
To test the scenario, ask the same question in several ways: using the official name, informal language and a misspelled platform name. The team should check whether the correct procedure is retrieved each time and whether the outdated email address appears in the answer.
Preparing the documents
Begin with an inventory. Establish where the material lives, who owns it and how a current version is distinguished from an archived one. An initial release can benefit from one well-maintained section rather than immediate access to the entire file repository.
- Currency. Preserve a version, effective date or another reliable signal for selecting the applicable document.
- Structure. Keep headings, lists and the relationship between a rule and its exceptions. An isolated sentence may convey a different meaning.
- Text quality. Check extracted text from scans, particularly names, numbers and information originally arranged in tables.
- Ownership. Assign a person or team to correct inaccuracies and communicate changes.
- Permissions. Specify which groups may access each document and its retrieved passages.
This work is useful beyond the AI application. It establishes which information employees can rely on and who is responsible for keeping it accurate. It also gives the development team a more realistic view of the content available for answering questions.
Preserving context around a passage
Long materials are often divided into passages for retrieval. Each passage should retain its document title, section, version and source link. Otherwise the retrieved content may omit a condition in the preceding paragraph or an exception immediately after it.
For example, the sentence “the request is submitted by the manager” is incomplete without the section explaining which kind of access it covers. During evaluation, inspect the passages supplied to the model as well as the final response. This separates retrieval failures from errors made while explaining correctly retrieved information.
A useful review question is whether a colleague could reconstruct the answer from the displayed evidence. If the evidence is too fragmented to support the conclusion, improving the wording of the response will not solve the underlying problem.
Permissions belong in the application
A knowledge base may combine general procedures with material restricted to particular teams. The application needs to retrieve only content permitted for the requesting user. This check should happen before the content is included in the model’s response context.
Hiding a document’s title after generation is insufficient if its contents already shaped the answer. Design must therefore identify the source of access rights, how permission changes propagate and what happens when a document is removed. These are integration requirements rather than instructions that can be left solely to a language model.
Evaluating the knowledge base
Prepare questions with known supporting documents. Include questions with no answer, conflicting procedures and material outside a user’s permissions. Test content updates separately: a new version should become available for retrieval, and removed information should stop contributing to responses.
- Confirm that retrieval found the right document with enough surrounding context.
- Compare factual statements in the answer with the passages supporting them.
- Open the source link as the same user and check that it resolves to the applicable material.
- Assess whether the assistant asks for clarification or reports missing information when appropriate.
- Repeat relevant checks after content updates or retrieval configuration changes.
The person maintaining documents should participate in reviewing failures. They may discover an outdated procedure or missing explanation rather than a software defect. Distinguishing those cases helps the team assign corrections to the right owner.
Setting realistic expectations
A useful knowledge base helps employees find and understand current information. It still requires maintained documents. Its quality depends on content, retrieval, permissions and response evaluation together. For an initial release, choose one subject area and agree which questions it should handle.
If the assistant must also create a request or check its status after finding an instruction, it needs integration with business systems. That is a separate development step with additional requirements for permissions and confirmation of completed actions.
