SOLUTIONS · FOR AI PRODUCTS
Automatic metadata tagging for RAG, in your own categories
Tag every document for RAG with your own metadata (type, department, product and who may see it) using a model trained on files your team already filed.
Retrieval-augmented generation (RAG) works best when every document carries metadata: what it is, which team it belongs to, which product it's about, and who may see it. Then a question about "the 2026 refund policy" searches policies, not every PDF on the drive, while your retrieval system separately enforces the viewer’s actual permissions. Incomplete metadata makes that filtering harder.
StayCharted fills it in with a private model trained on documents your team has already filed, so every new document is tagged the same way, with a confidence.
Why your own metadata
A general AI can guess that a file is "a contract" or "a policy". Retrieval filters need your values:
- your document types: "MSA", "Order form", "DPA", not just "contract";
- your departments and product lines;
- your access groups, exactly as SharePoint or your DMS names them.
Use reviewed examples from your existing folders and libraries as training data. Check that folder names and existing tags still reflect your current rules.
What it reads, and what it answers
It reads what you have for each document:
| Field | Example |
|---|---|
| Title or file name | "Acme MSA v3 signed.pdf" |
| Opening text (first page, extracted) | "This Master Services Agreement…" |
| Folder or library (optional) | Legal/Customers/Acme |
It answers with your metadata, as one answer or several together:
| Metadata | Example |
|---|---|
| Document type | MSA |
| Department | Legal |
| Product line | Platform |
| Who may see it (a list) | Legal Owners; Sales Leadership |
The model returns its answer with a confidence. Load approved content tags as metadata and send uncertain tags for review. Access suggestions require a separate authorization policy, regardless of confidence.
Who may see it: permission-aware RAG
Tagging access groups is where a mistake costs the most. StayCharted can learn them from your SharePoint permission export: it reads the report as it is and learns which groups open which kinds of files. A new file gets suggested groups for your access-management workflow to validate. Your retriever must enforce authoritative permissions, including existing restrictions, for every result. A predicted tag must not grant access by itself. AI for SharePoint and file permissions →
How to set it up
1. Export documents that are already filed
One row per document: file name, extracted opening text, folder, and the metadata you want: type, department, product, groups. Your DMS export or SharePoint's file list usually has most of it; the folder often is the label.
2. Train one model per kind of metadata, or one with several parts
Upload and train an AI Classifier Model. The report shows, for documents it never saw, how often each value was right.
3. Tag before you index
In your ingestion pipeline, call the model through the API for each new or changed document, and write the answers into the chunk metadata of your vector store. For a backlog, fill a spreadsheet of all documents at once and load it.
4. Retrain as the library changes
New document types and new groups need examples. Retrain monthly, or when the structure changes.
Use the same field names and ordering as training. Put relevant opening text first and keep it concise: the AI Classifier reads a limited input window. API-based ingestion requires Essentials or Business.
What it doesn't do
- It doesn't extract text or chunk documents. Use your existing parser; send the extracted opening text.
- It doesn't store, index or search documents. It returns metadata for your pipeline to store.
- It doesn't enforce permissions. Your retriever and your identity system do. Enforce actual permissions for every retrieval, and keep uncertain access suggestions restricted until reviewed.
Evaluate before automating
Use reviewed examples that reflect your current rules. Check errors by category and sample confident answers as well as uncertain ones. Confidence is a model score, not a guarantee. Keep mandatory safety and access rules in your application. Best practices for training and review →.
Common questions
How do I add metadata to documents for RAG?
Train a model on documents your team has already filed with the metadata you need, then tag every new document in your ingestion pipeline before it's indexed.
What metadata helps RAG retrieval most?
Usually document type, department or owner, product, date and who may see it. Pick what your users' questions narrow down by.
Can it tag who is allowed to see a document?
It can suggest access groups learned from reviewed permission history, with confidence. Those suggestions are not an authorization decision. Validate them through your access-management process and enforce actual permissions in the retriever.
Does it work with my vector database?
Yes. It returns values through the API or in a spreadsheet; you write them to whichever vector store you use.