Self-Hosted RAG and GraphRAG: Architecture for Private Enterprise Data

Self-hosted retrieval-augmented generation requires more than placing a language model beside a vector database. The complete pipeline must ingest approved sources, preserve their access rules, retrieve useful evidence, generate supported answers, and keep every derivative current. A graph-based pipeline adds entity resolution, relationship provenance, and summary maintenance to that responsibility.

The practical design question is which retrieval architecture serves the required queries within the private-data boundary. Start with a credible baseline, then introduce graph structures where relationships or corpus-wide synthesis resolve a demonstrated limitation. Evaluate the whole pipeline, including permissions, updates, and operating effort.

This guide uses an illustrative internal product-support library containing manuals, release notes, dependency records, and incident reports. The example is a proposed architecture, not a claimed client deployment or benchmark result. It focuses on the implementation decisions that make private retrieval maintainable.

Define the query and information boundary

List the questions the application needs to answer. A request for one policy exception differs from a request for dependencies across several products or recurring themes across incidents. Group queries by the evidence they require, then identify which sources contain that evidence and who may read it.

Record where each processing stage runs: document parsing, optical character recognition when needed, embedding, graph extraction, reranking, generation, and evaluation. A local generator does not make an externally processed document private. Include caches, logs, failed-job payloads, and diagnostic exports in the map.

Use the RAG vs. GraphRAG comparison to decide whether a graph addresses the actual query need. The Private AI Architecture guide places retrieval inside the wider stack. The private deployment checklist captures the constraints the pipeline must preserve.

State what the application should do when evidence is unavailable or inaccessible. It may ask for clarification, return a supported partial answer, or route a question to a source owner. It should not silently substitute an external search over private material or invent an answer because the local index has no match.

Build an ingestion contract before an index

Each source should have a stable identifier, version or change marker, owner, access scope, processing status, and useful location information. Preserve these fields through chunking and derivative creation. They are necessary for citations, corrections, deletion, and incident investigation.

Define how the system detects additions, changes, withdrawals, and permission updates. A scheduled full rebuild may suit a small stable library, while a changing repository may need more frequent updates. Choose the method according to the source and freshness requirement rather than assuming one schedule fits every collection.

Record failed processing explicitly. A malformed file, unsupported format, or extraction error should create an observable result with an owner. Otherwise, users may interpret an empty search result as evidence that the organization has no relevant information when the real problem is an ingestion gap.

Test document preparation on difficult sources. Tables, headings, footnotes, identifiers, and exception clauses can carry important meaning. Keep enough context with each retrieval unit to interpret it correctly. Splitting a rule from its exception can make a technically relevant passage misleading.

Establish a permission-aware retrieval baseline

Begin with retrieval appropriate to the corpus. Exact identifiers may need exact matching. Natural-language questions may benefit from semantic retrieval. Keyword and vector signals can be combined, and candidate results can be reranked before context assembly. Record the actual configuration used in evaluation.

The pgvector documentation describes exact and approximate vector search. It also explains that filtering after an approximate index scan can reduce the number of returned matches, with iterative scans among the available approaches. This is a practical reason to test retrieval quality under the application's access filters, not only against an unrestricted development index.

Authorization and retrieval quality are separate requirements. A restrictive filter can correctly prevent disclosure while leaving too little useful evidence for an answer. The response should reflect that lack of evidence. Increasing candidate search or changing index organization must preserve the same permission boundary.

Evaluate the actual application identity. PostgreSQL's row-security documentation identifies roles that bypass row security. A retrieval prototype using an administrator or table-owner role may not exercise the policy the team believes it is testing. Verify both allowed and denied cases with the deployed role design.

Choose isolation deliberately

Scroll horizontally to see all columns.

Avoid selecting an isolation pattern solely for convenience during ingestion. Consider revocation, backup restoration, evaluation access, operational inspection, and future source changes. An index that is easy to build can still be difficult to maintain under the organization's actual permission model.

For a shared internal library, access may vary by project, department, or document classification. Those scopes can overlap. Define whether permissions are inherited from the original repository, mapped into application roles, or enforced through another approved mechanism. Record how a change becomes effective in retrieval and derivatives.

Test with users whose access differs by only one sensitive source. Ask questions that could reveal the restricted fact through a summary or relationship, not only questions that name the document directly. This exercises the information boundary the user experiences.

Add a graph for a specific information need

A graph can represent entities and relationships that help answer connected questions. In the product-support example, useful entities might include products, versions, components, and incidents. Relationships might describe documented dependencies or references from an incident to an affected component. The schema should follow the query need.

Microsoft's GraphRAG indexing overview describes extracting structured information and producing graph-related artifacts and summaries. Its query overview distinguishes entity-oriented local search and broader search over community reports. These are implementation mechanisms to evaluate, not a guarantee that a graph improves every question.

Separate explicit source statements from inferred relationships. A document saying that two components failed during the same incident does not necessarily establish a causal dependency. Preserve the source passages and the status of any interpretation. The answer layer should not silently promote an extracted hypothesis into an authoritative business fact.

Define entity resolution before relying on graph traversal. Aliases, reused product names, and changing identifiers can merge unrelated records or split one entity into several. Keep evidence for resolution decisions and a correction process. A graph with incorrect identity links can produce convincing explanations built on the wrong connections.

Verify local model compatibility at each stage

An inference server exposing a familiar API shape does not guarantee that every retrieval framework feature will work correctly. Test structured outputs, message formatting, token limits, batching, retries, and error responses for the exact framework and model versions in use.

Microsoft's GraphRAG model-selection documentation calls out structured-output requirements and warns about malformed responses when using alternate model arrangements. That makes extraction and indexing quality a distinct evaluation task. A model that writes good conversational answers may still produce unreliable graph records or summaries.

Evaluate embedding and reranking models separately from the generator. Confirm that the supported languages, document types, and retrieval task match the corpus. Record the embedding configuration with the index version. When changing the embedding model or its representation, define the required reindexing and compatibility procedure instead of mixing vectors without an explicit design.

The open-weight model selection guide provides the broader candidate record. For private retrieval, add stage-specific acceptance examples so an apparently successful end-to-end answer does not hide unreliable intermediate artifacts.

Preserve provenance through summaries and answers

Every answer should be traceable to the evidence that supports its material claims. Store source identifiers and useful locations with retrieval units. When generating summaries or graph-derived reports, preserve the source scope and build version used to create them.

A summary can combine information from several documents with different permissions. Decide whether the combined artifact can be shared, must be generated within a compatible access scope, or should be excluded from certain query paths. Filtering original documents alone does not prove that a previously generated summary is appropriate for the current user.

Test citations for support, not only presence. A citation can point to a real document without supporting the associated conclusion. For relationship answers, inspect the evidence for each relevant edge or connection. For corpus-wide themes, define how reviewers assess coverage and material omissions.

Keep uncertainty visible where the sources do not settle a question. Conflicting versions may require a source owner to decide precedence. Missing records may limit the scope of a synthesis. The application should explain those conditions without implying that the retrieved subset is a complete account of the enterprise.

Design updates, revocation, and deletion together

Map each source to its derivatives: text units, embeddings, graph records, summaries, and cached answers. This dependency record supports updates and removal. Without it, a withdrawn document can disappear from the source list while its content remains available through an older summary.

Define the effect of a permission change separately from an ordinary content refresh. If access must stop promptly, the online path may need to deny use of stale artifacts before rebuilding them. Record the supported behavior and test it. Do not describe eventual refresh as immediate revocation unless the implementation actually enforces that boundary.

A correction can also affect relationships. If two product identifiers were merged incorrectly, determine which edges and summaries used that merge. Preserve enough build metadata to regenerate the affected artifacts or rebuild the relevant partition safely. The correction procedure should be part of operations, not an improvised database edit.

Include backups in the lifecycle. Restoring an older index can reintroduce withdrawn content or outdated access metadata. The recovery process should reconcile the restored artifacts with the current source and permission state before returning the application to normal service.

Worked lifecycle: a changing product library

The support application begins with approved manuals and incident records. Its first retrieval baseline handles exact product-version questions and returns source-backed drafts. The team then identifies a query class about shared dependencies across incidents and tests a graph-assisted path for that class.

During evaluation, two products use the same informal name. The graph extractor merges them, causing one dependency answer to combine unrelated evidence. The team corrects the entity-resolution rule, rebuilds affected artifacts, and adds a regression case. The finding changes the graph pipeline; it does not justify rewriting every answer prompt.

Next, a release note supersedes a manual section. The ingestion job records the new version, updates the relevant retrieval units, and refreshes dependent summaries. Evaluation verifies that the answer cites the current controlling evidence and handles the older document according to the source policy.

Finally, access to one incident report is restricted. A user who previously saw it repeats a related question. The test inspects the selected passages, graph-derived context, returned answer, and cache behavior. A correct final refusal alone would be insufficient if the restricted report still entered the model context.

This lifecycle exercise evaluates the system the team will maintain. It reveals whether the graph adds useful retrieval capability and whether the application can sustain accurate, permission-aware results as the corpus changes.

Evaluate quality, access, and operating cost

Build separate evidence for retrieval coverage, answer support, permission enforcement, and lifecycle behavior. Where reference passages can be labeled, inspect whether the correct evidence appears in the final context. Where questions require broad synthesis, define a review method appropriate to the source collection and intended conclusion.

Compare the baseline and graph candidate on the same corpus version, query set, access scope, and generation configuration where possible. Record differences deliberately introduced by the architecture. A candidate with a fresher corpus or a larger processing budget may be useful, but the comparison should show what changed.

Measure ingestion duration, refresh delay, storage, query latency, review effort, and maintenance work alongside answer quality. Graph construction can create work before a single user query arrives. The relevant business comparison includes the cost of keeping the evidence usable, not just the inference cost of the final response.

Use the AI Agent Evaluation guide to structure case outcomes and grading. Keep denied access and unsupported questions as explicit case types. A private retrieval system should be evaluated on what it correctly refuses to disclose as well as what it successfully answers.

Prepare the operating handoff

Assign owners for source intake, access mapping, entity resolution, model artifacts, index updates, answer quality, and incident response. Provide a way to inspect failed ingestion and stale derivatives without granting broad access to every source. The operator should know which component to investigate for each failure category.

Document how to rebuild the pipeline from approved artifacts and how to restore service without reviving withdrawn data. Rehearse a source correction and a permission change with the receiving team. Those exercises show whether the handoff covers the lifecycle users will actually encounter.

Where retrieval becomes shared enterprise infrastructure, AI Technology & Platform Strategy can connect the application requirements with the platform decision. Retain the workflow-specific queries and acceptance criteria so shared infrastructure remains grounded in useful work.

Migrate an index without losing the evidence baseline

A retrieval upgrade may change chunking, embeddings, index settings, entity resolution, or summary generation. Keep the old accepted configuration identifiable while building the candidate. Record the source snapshot and access rules used by both so the comparison does not confuse a pipeline change with a different corpus.

Run representative queries against both configurations in an authorized test environment. Compare selected evidence and final answers, including difficult identifiers, restricted sources, unsupported questions, and recently changed documents. A better aggregate relevance score should not conceal a permission regression or loss of a controlling policy exception.

For a graph upgrade, inspect the artifacts that changed materially. An entity merge can alter many downstream relationships. A revised summary prompt can change which facts survive into broad query context. Review those differences where they affect the application's important questions, then add meaningful regression cases rather than testing every output string for exact equality.

Define the cutover and rollback procedure before replacing the active index. The application should identify which index version served a case and avoid mixing incompatible artifacts during one request. If the candidate is rejected, returning to the prior configuration should preserve current access restrictions and source withdrawals. An older index is not automatically an eligible rollback target.

Document capacity during the migration. Building and retaining two configurations can temporarily increase processing and storage demand. Decide when old artifacts can be removed and which evidence must remain for diagnosis. This is part of the cost of maintaining a retrieval system, especially when the corpus is large or graph construction is expensive.

Assign the acceptance decision to the owners of answer quality, source handling, and operations. A migration is complete when the selected pipeline serves the intended queries under the current boundary and the receiving team can maintain it. Successful index construction alone does not establish that outcome.

Build private retrieval around supported evidence

Self-hosted RAG and GraphRAG succeed when evidence remains relevant, permission-aware, traceable, and maintainable throughout its lifecycle. A graph is justified by the questions it helps answer and the operating responsibility the organization can sustain.

Bring the source inventory, representative questions, current retrieval failures, and required data boundary to Private & Self-Hosted AI Solutions. Those inputs define a concrete implementation comparison and a useful path from an experimental index to an operating knowledge application.