Knowledge bases and RAG

What is RAG?

Retrieval-Augmented Generation (RAG) gives your agents access to specific documents and data that aren't part of their pre-training. Instead of relying solely on the language model's general knowledge, RAG agents retrieve relevant information from your documents and use it to generate informed, grounded responses.

How RAG works

The key insight: LLMs are excellent at reasoning and language generation, but they don't know YOUR data. RAG bridges this gap by retrieving your data at query time and providing it as context.

Why use RAG?

When to use RAG

RAG is ideal for:

• HR policy assistants — Answer employee questions based on specific policy documents
• Compliance reviewers — Verify that proposals comply with internal guidelines and regulatory requirements
• Technical documentation Q&A — Help users find information in product documentation
• Customer support — Answer questions based on knowledge bases and help articles
• Contract analysis — Review contracts against your organization's standard terms and conditions
• Research assistants — Query large document collections to find relevant information

If your agent needs to know information specific to your organization, domain, or use case, you need RAG.

RAG pipeline architecture

MagOneAI implements a production-grade RAG pipeline with hybrid search, reranking, and advanced retrieval techniques.

Retrieval flow

Qdrant vector database

MagOneAI uses Qdrant as its vector database for semantic search and retrieval.

• Hybrid search — Dense vector similarity + BM25 keyword matching with RRF fusion
• Performance — Fast similarity search even over millions of vectors
• Filtering — Combine vector similarity with metadata filtering (e.g., filter by kb_id)
• Scalability — Handles large knowledge bases with consistent query latency

MagOneAI manages the Qdrant infrastructure for you. You simply upload documents and configure knowledge bases — the vector database operations happen automatically.

Creating and managing knowledge bases

Knowledge bases are collections of documents that agents can query. Each knowledge base has its own vector collection and can be attached to multiple agents.

Knowledge base types

When you create a knowledge base, you choose how its documents are made available to agents. The type is set per knowledge base.

Pick vector store when you have many documents and want precise retrieval; pick document when you have a few short documents that should always be present in full.

Tabular knowledge bases (spreadsheets answered with SQL)

When you add a CSV or Excel file to a knowledge base, MagOneAI keeps it relational instead of chunking and embedding it. The file becomes a dataset in the knowledge base's tabular catalog, and agents query it with real SQL over the underlying data.

This matters for numbers. Vector search is built for prose, and asking it to "sum the invoices" or "average the deal size" returns an approximation at best. A tabular knowledge base answers those questions with the exact value, because the aggregation (SUM, AVG, GROUP BY, top-N) runs as an actual query.

Large & scanned PDFs

Large PDFs are extracted page by page rather than loaded whole, so document size doesn't blow up memory. Extraction reports live progress as it works through the pages and can resume on failure — if ingestion is interrupted, it picks up from where it stopped instead of restarting the whole file.

A layout-aware router inspects each page and chooses the right extraction path:

• Pages that are clean digital text are sent down a fast text path.
• Pages containing tables, figures, or scanned/image content are routed to deeper extraction with OCR.

This means tables keep their structure and scanned or image-only pages are still captured accurately, instead of coming through as empty or garbled text.

Image understanding (vision enrichment)

Image understanding is an optional, per-knowledge-base setting. When enabled, a vision-capable model describes image uploads at ingestion time, and that description — not a generic placeholder — becomes the searchable text for the image. This lets agents retrieve images by what they actually depict.

• Off by default — enable it only on knowledge bases where image content matters.
• You choose the model — select which vision-capable model generates the descriptions.

Managing knowledge bases

Syncing from external sources (SharePoint)

In addition to manual uploads, a knowledge base can be kept automatically in sync with a SharePoint document library. Files in a chosen SharePoint site and folder are mirrored into the knowledge base — added, updated, and removed to match the source — so agents always retrieve from current documents without anyone re-uploading by hand.

Source types

Every knowledge base has a source type:

• Manual (default) — you upload and manage documents yourself.
• Web pages and crawls — add content directly from a URL. MagOneAI fetches the page (and optionally follows links to crawl a site), extracts the content, and ingests it like any other document. See Adding web pages below.
• SharePoint — documents are mirrored from a SharePoint folder. The source type flips to SharePoint automatically the first time an external sync writes to the knowledge base.

Manual uploads and synced documents are tracked independently, so external syncing never touches files you uploaded by hand.

Adding web pages and crawls

You can populate a vector store knowledge base straight from the web instead of downloading and uploading files by hand.

Connecting SharePoint

SharePoint is connected as a credential-based MCP connection, the same way other API-key integrations are configured (see External MCP servers). The connection is read-only and project-scoped — one SharePoint site per connection.

How sync works

The sync walks the SharePoint folder and diffs it against what's already in the knowledge base, then reconciles the difference:

Syncing is idempotent: each document is keyed by its stable SharePoint item ID, so renames and moves don't create duplicates, and re-running a sync that found no changes does nothing.

Source attribution

Synced documents carry attribution metadata from SharePoint — the source URL, document type, author and last-modified-by, and last-modified date. This metadata powers citations: when an agent answers from a SharePoint document, the citation chip links straight back to the file in SharePoint. Manually-uploaded documents have no source URL and render as a non-clickable citation.

Chunking strategies

The quality of your RAG system depends heavily on how documents are chunked. MagOneAI supports two chunking strategies.

Standard chunking

Section-aware recursive splitting with title prepend for better embedding quality.

How it works:

1. Documents are parsed into sections (headings, paragraphs)
2. Each section is recursively split using progressively finer separators: \n\n → \n → . →
3. Chunks receive a title/section prefix prepended to the embedding text (e.g., filename > Section Title: chunk text)
4. Overlapping windows ensure information at chunk boundaries isn't lost

Configuration:

• Chunk size — Max words per chunk (default: 400)
• Chunk overlap — Words of overlap between chunks (default: 40)

Best for: Most use cases. Works well with structured documents that have clear headings and sections.

Small2Big chunking

A parent-child chunking strategy that retrieves on small chunks but expands to larger parent chunks for context.

How it works:

1. Documents are first split into parent chunks (default: 400 words, no overlap)
2. Each parent chunk is sub-divided into smaller child chunks (default: 200 words)
3. Child chunks store a reference to their parent (parent_id and parent_text)
4. At retrieval time, search matches on precise small chunks, then expands to the full parent chunk for richer context
5. Results are deduplicated by parent ID — only the highest-scoring child per parent is kept

Best for: When you need precise retrieval (matching on specific phrases) but want to provide broader context to the agent.

Contextual chunking

An optional LLM-enriched step that generates situating context for each chunk before embedding. This dramatically improves retrieval quality.

How it works:

1. A document summary is generated using an LLM (2-3 paragraphs covering document type, main topics, key entities)
2. For each chunk, an LLM generates 1-2 sentences of situating context that describes how the chunk relates to the overall document
3. The situating context is prepended to the chunk text before embedding

Configuration:

contextual_chunking:
  enabled: true
  llm_config_id: "your-llm-config"  # Required - LLM used for context generation
  summary_prompt: "..."  # Customizable document summary prompt
  context_prompt: "..."  # Customizable chunk context prompt

The context prompt receives: the document summary, the previous chunk, the current chunk, and the next chunk — giving the LLM full context to write accurate situating context.

Best for: Knowledge bases where retrieval precision is critical. Adds processing time and cost during ingestion, but significantly improves retrieval quality.

Chunk overlap

Overlap ensures important information at chunk boundaries isn't lost:

Without overlap:

Chunk 1: [...employee must complete 90 days]
Chunk 2: [of employment before remote work eligibility...]

The connection between "90 days" and "remote work eligibility" is split across chunks.

With overlap:

Chunk 1: [...employee must complete 90 days of employment before]
Chunk 2: [complete 90 days of employment before remote work eligibility...]

Now both chunks contain the complete concept.

Recommended overlap: 10% of chunk size (e.g., 40 words for 400-word chunks)

KB retrieval modes

MagOneAI supports two retrieval modes that control how agents interact with knowledge bases.

Auto mode (default)

In auto mode (kb_retrieval_mode: "auto"), the system performs a single retrieval when the agent starts executing:

1. The agent's input is used as the search query
2. All attached knowledge bases are searched in parallel
3. Retrieved chunks are injected into the agent's system prompt as static context
4. The agent generates its response using this context

Best for: Simple Q&A, straightforward document lookup, and cases where the input query is a good search query.

Agentic mode

In agentic mode (kb_retrieval_mode: "agentic"), the agent can iteratively search knowledge bases during its reasoning:

1. A __kb_search tool is added to the agent's available tools
2. The agent decides when and what to search based on its reasoning
3. The agent can make multiple search calls with different queries
4. Each search returns formatted results with source metadata and relevance scores
5. The agent synthesizes information across multiple searches

Configuration on the agent:

capabilities:
  kb_retrieval_mode: "agentic"
  max_kb_searches: 10        # Max search calls per execution (1-50)
  hyde_enabled: true          # Enable HyDE query expansion
  hyde_llm_config_id: "..."   # LLM for HyDE passage generation
  kb_search_all_kbs: false    # Search all attached KBs as one pool (see below)

Searching across multiple knowledge bases

When an agent has more than one knowledge base attached, a "Search all knowledge bases together" toggle (kb_search_all_kbs, default off) controls how results are combined:

• Off (routed, default) — Each attached knowledge base is searched and keeps its own top results, grouped under per-KB section headers. The agent can also target a specific knowledge base by ID. Leave this off when you have a per-chat upload KB alongside a shared common KB and want each represented.
• On (merged) — Every attached knowledge base is merged into a single pool, reranked together, and trimmed to one global top set. Best when the KBs cover the same domain and you want the single most relevant results regardless of which KB they came from.

How agentic search works:

The agent receives a KB search tool with this schema:

• query (required) — The search query. The agent crafts specific queries based on its reasoning.
• top_k (optional) — Number of results (1-20, default 5)
• kb_id (optional) — Target a specific knowledge base by ID (only shown when multiple KBs are attached)

Results are returned with source metadata and relevance labels:

=== SOURCE: Remote Work Policy 2024.pdf > Section 3.2 [relevance: HIGH] ===
[Page 4] Employees must complete the initial 90-day probationary period before...
=== END ===

Best for: Complex queries that require multiple searches, research tasks, and cases where the initial input isn't a good search query on its own.

HyDE (Hypothetical Document Embeddings)

HyDE is an advanced query expansion technique that improves retrieval by generating a hypothetical answer before searching.

How HyDE works

Why HyDE helps

User queries often use different vocabulary than source documents. For example:

• User asks: "Can I work from home?"
• Document says: "Remote work eligibility requires completion of the probationary period"

The hypothetical passage bridges this vocabulary gap by generating text that's likely to use similar language to the source documents.

Enabling HyDE

HyDE is configured per agent in the capabilities section:

capabilities:
  hyde_enabled: true
  hyde_llm_config_id: "your-llm-config"  # LLM used for passage generation

HyDE works with both auto and agentic retrieval modes. In agentic mode, HyDE passages are generated automatically for each __kb_search tool call.

Hybrid search and reranking

MagOneAI uses a multi-stage retrieval pipeline for high-quality results.

Hybrid search

Every search query produces both:

• Dense vector — Captures semantic meaning (what the text means)
• Sparse BM25 vector — Captures keyword relevance (what words appear)

Qdrant runs both searches in parallel and merges results using Reciprocal Rank Fusion (RRF). This combines the strengths of both approaches:

• Semantic search finds conceptually similar content even with different vocabulary
• Keyword search finds exact term matches (names, acronyms, codes)

Reranking

After hybrid search returns ~50 candidates, a cross-encoder reranking model rescores each result:

• Model: BGE-reranker-v2-m3
• Input: (query, candidate_text) pairs
• Output: Relevance scores with labels
  • HIGH — Score > 0.5 (highly relevant)
  • MEDIUM — Score > -1.0 (moderately relevant)
  • LOW — Score ≤ -1.0 (marginally relevant)

Top-k results after reranking are returned to the agent.

How RAG works in agent execution

When a RAG agent executes within a workflow, the retrieval and generation process follows a precise sequence:

Detailed execution flow

{
  "question": "What is our policy on remote work for new employees?",
  "context": "employee_onboarding"
}

=== SOURCE: Remote Work Policy 2024.pdf > Section 3.2 [relevance: HIGH] ===
[Page 4] Employees must complete the initial 90-day probationary period
before becoming eligible for remote work arrangements...
=== END ===

=== SOURCE: Employee Handbook.pdf > Onboarding [relevance: MEDIUM] ===
[Page 12] New employees are assigned a buddy during their first 90 days...
=== END ===

Best practices

Keep documents focused and well-structured

Good document structure:

• Clear headings and sections
• Logical information hierarchy
• Consistent formatting
• One topic per document or section

Well-structured documents chunk better and retrieve more accurately. Section headings are used in the chunk title prefix, improving embedding quality.

Use descriptive file names

File names appear in source citations and are prepended to chunk embeddings:

Good file names:

• remote_work_policy_2024.pdf
• employee_onboarding_checklist.pdf
• gdpr_compliance_guidelines.pdf

Poor file names:

• document_final_v3.pdf
• policy.pdf
• untitled.pdf

Choose the right retrieval mode

Enable HyDE for vocabulary mismatch

If your users ask questions using different terminology than your source documents, enable HyDE. It's especially helpful for:

• Technical documentation with domain-specific jargon
• Policy documents with formal language
• Multi-language knowledge bases

Use contextual chunking for high-stakes use cases

Contextual chunking significantly improves retrieval quality at the cost of higher ingestion time and LLM usage. Enable it for:

• Compliance and regulatory documents
• Legal contracts and policies
• Medical or financial documents where precision matters

Test retrieval quality before production deployment

Before deploying RAG agents to production:

Troubleshooting RAG issues

Next steps