RAG is not a feature that automatically grounds a model. It is a system with two dependent stages: retrieval must locate the right evidence, and generation must use that evidence faithfully for the question being asked. A citation can point to a document that does not support the claim. A larger model can explain the wrong context more fluently. Quality therefore has to be tested at the retrieval, generation, and product-behavior levels.
The first decision is a product one: which questions the system may answer, when it must decline, and who bears the cost of an error. That contract drives requirements for the corpus, pipeline, evaluation suite, and interface. Governance should make an answer reproducible and a bad change reversible; documentation created after release cannot substitute for those controls.
Start with the task and the loss from error
An assistant retrieving a leave policy has a different risk profile from a system supporting diagnosis, credit decisions, or incident response. For each use case, specify permitted sources, freshness expectations, intended users, foreseeable consequences of error, and the condition for human escalation. An answer can be grammatically correct while being unacceptable in the operational context.
Distinguish “no evidence” from a negative answer. If the corpus does not cover a scenario, the model should state that limitation rather than extrapolate from a neighboring policy. Higher-impact workflows need stronger abstention behavior, independent review, and limits on downstream action. Do not reduce that decision to one uncalibrated confidence score; evidence coverage and the cost of being wrong are more informative.
Define what success means for the user as well. A perfectly grounded paragraph may still be unusable if it omits a required exception or fails to identify the next accountable action.
Operate the corpus as a governed product
Every source needs an owner, audience, effective date, update rule, usage rights, and publication status. The index should not mix approved instructions with working notes unless authority is represented in metadata and ranking. When documents conflict, the system needs rules for version and precedence rather than asking the generator to choose the most convenient passage.
Preserve lineage from the original document through parsing, chunks, embeddings, and index versions. A product owner should be able to identify which source and model versions contributed to an answer. Index and trace retention must respect confidentiality. Copying an entire knowledge repository into an external service must be a reviewed architecture decision, not an accidental consequence of ingestion.
Freshness is more than a scheduled rebuild. Detect withdrawn documents, failed parsers, missing sections, delayed feeds, and records whose effective date has passed. Publish those conditions as product health signals.
Evaluate retrieval before judging the prose
Build a representative question set with expected sources or passages. Include real user phrasing, abbreviations, spelling errors, multi-part questions, unanswerable cases, and documents with similar names. Domain experts should approve the expected evidence, but the set must not contain only questions invented by the implementation team.
Useful retrieval measures include whether the required passage appears among the first results, how much irrelevant context is supplied, and whether the current authoritative version wins. Latency matters alongside relevance, not in place of it. Any change to chunking, embeddings, reranking, query rewriting, or metadata filters should rerun this evaluation. The original Retrieval-Augmented Generation paper emphasized updateable knowledge and provenance; a production implementation has to preserve those properties in practice.
Inspect failures by category. Missing evidence, wrong ranking, over-broad filters, and poor chunk boundaries require different fixes. Prompt tuning cannot repair evidence that never reached the model.
Score faithfulness and task utility separately
Evaluate the generator against both the supplied context and the user's task. Ask whether each material claim is supported, whether the citation identifies the correct passage, whether important qualifications remain intact, whether facts are separated from inference, and whether the answer follows the product instruction. RAGAS is an early primary reference for decomposing retrieval relevance, faithful use of context, and answer quality.
Model-based judges can scale regression testing, but they introduce their own failure modes. Calibrate them against a sample reviewed by domain experts and inspect disagreements. A binary pass/fail score can be too coarse: a wrong exception in an otherwise accurate procedure may be more consequential than several stylistic defects.
Maintain both stable and evolving evaluation sets. The stable set makes releases comparable; production failures and new content supply harder cases. Protect holdout questions from prompt and retrieval tuning to reduce overfitting.
Treat retrieved content as untrusted data
A document is context, not an instruction the model should obey. It may contain prompt injection asking the system to reveal secrets, ignore policy, or call an external resource. Limit available tools, separate system instructions from retrieved text, restrict source types, and test attacks embedded in documents.
Authorization must apply before retrieval. Asking the model not to quote a sensitive passage after it has received the passage is not access control. Enforce tenant, role, and classification filters in the index query and log their decisions. NIST AI 600-1 recommends lifecycle controls for generative AI including content provenance, pre-deployment testing, monitoring, and incident disclosure processes.
Red-team the full chain, not only the prompt. Test poisoned sources, malicious metadata, encoded instructions, cross-tenant identifiers, excessive context, and tool calls derived from retrieved text.
Gate releases and instrument the live system
Every release should identify the model, prompt, index, embedding model, reranker, and filter configuration. A release gate compares a stable regression suite and newly observed cases. A material regression in a critical category blocks the change or requires a documented risk acceptance from the appropriate owner.
Production traces should support analysis of the query, retrieved passage identifiers, citations, abstention, latency, and user feedback while minimizing retained sensitive data. Monitor empty result sets, topic-distribution shifts, stale sources, unusual access attempts, and escalation changes. A raw satisfaction score is not enough: users often reward fluent answers even when the evidence is weak.
Define rollback at each layer. The team may need to restore a prior index without changing the model, disable one collection, revert a reranker, or place the product in retrieval-only mode while an incident is investigated.
Assign ownership and prepare for RAG incidents
The corpus owner is accountable for source authority and freshness; the platform team for ingestion, retrieval, and isolation; the product owner for behavior and thresholds; domain experts for substantive criteria; and security and privacy teams for abuse and data controls. Responsibility can be distributed, but every control needs a named destination.
A RAG incident is not limited to data leakage. Index poisoning, systematic mis-citation, use of a withdrawn procedure, or persistent failure for one category of users also requires response. The runbook should support freezing ingestion, rolling back an index or model, identifying affected answers, notifying owners, and validating the fix before restart. Continuous governance, mapping, measurement, and management—consistent with the NIST AI RMF—is what separates a production knowledge system from a convincing demonstration.