> ## Documentation Index
> Fetch the complete documentation index at: https://helpcenter.magure.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Data flow diagram

> How data moves through MagOneAI — from user input to AI-generated output, with every storage layer and external service mapped

## Overview

This page documents how data enters, moves through, is stored in, and exits the MagOneAI platform. Understanding these flows helps you assess data exposure, plan compliance, and make informed decisions about model selection and tool integrations.

***

## Platform architecture

MagOneAI is a layered system where all data flows through a secure API layer. No external service connects directly to any internal data store.

```mermaid theme={null}
graph TB
    subgraph clients["Clients"]
        studio["MagOneAI Studio\n(workflow builder)"]
        hub["MagOneAI Hub\n(chat interface)"]
        api_client["API / Webhooks\n(programmatic access)"]
    end

    subgraph platform["MagOneAI Platform"]
        direction TB
        api["API Gateway\nAuthentication & Authorization"]

        subgraph compute["Compute Layer"]
            direction LR
            worker["Workflow Engine\n(Temporal)"]
            activities["Activity Executors\n(Agent, Tool, Condition, Parallel)"]
        end

        subgraph storage["Secure Storage Layer"]
            direction LR
            db[("Relational\nDatabase")]
            vault[("Secrets\nVault")]
            object[("File\nStorage")]
            vector[("Vector\nStore")]
        end
    end

    subgraph external["External Services"]
        direction LR
        llm["LLM Providers\n(OpenAI, Anthropic,\nPrivate Models)"]
        google["Google APIs\n(Gmail, Calendar)"]
        microsoft["Microsoft APIs\n(Outlook, OneDrive)"]
        web["Web Search\nAPIs"]
    end

    studio --> api
    hub --> api
    api_client --> api
    api --> compute
    compute --> storage
    activities --> llm
    activities --> google
    activities --> microsoft
    activities --> web

    style clients fill:#f8f9fa,stroke:#adb5bd,color:#212529
    style platform fill:#e7f0ff,stroke:#3077F3,color:#212529
    style compute fill:#dce4f5,stroke:#6366F1,color:#212529
    style storage fill:#cfe2ff,stroke:#3B82F6,color:#212529
    style external fill:#fff3cd,stroke:#F59E0B,color:#212529
    style api fill:#3077F3,stroke:#2060D0,color:#ffffff
```

***

## Workflow execution flow

This is the primary data path — how user input travels through MagOneAI and becomes AI-generated output.

```mermaid theme={null}
sequenceDiagram
    participant U as User / API Client
    participant API as API Gateway
    participant DB as Database
    participant WF as Workflow Engine
    participant A as Activity Executor
    participant V as Secrets Vault
    participant LLM as LLM Provider
    participant MCP as MCP Tool Server

    U->>API: Execute workflow (input data)
    API->>API: Authenticate & authorize
    API->>DB: Create execution record
    API->>WF: Queue workflow
    API-->>U: Execution ID (async)

    loop For each activity in workflow
        WF->>A: Execute activity

        alt Agent Activity (LLM call)
            A->>V: Resolve API key
            V-->>A: API key
            A->>LLM: Send messages + tools
            LLM-->>A: AI response
            A->>DB: Log usage (tokens, cost)
        end

        alt Tool Activity (MCP call)
            A->>V: Resolve tool credentials
            V-->>A: Credentials
            A->>MCP: Call tool with parameters
            MCP-->>A: Tool result
        end

        alt Human Task Activity
            A->>DB: Create pending task
            WF->>WF: Pause workflow
            U->>API: Submit approval / answers
            API->>WF: Resume workflow
        end

        A->>DB: Store activity output
    end

    WF->>DB: Mark execution complete
    U->>API: Fetch result
    API->>DB: Read execution output
    API-->>U: Final output
```

***

## Data entering the platform

All data enters MagOneAI through authenticated API endpoints. There are no direct connections to internal stores from outside.

```mermaid theme={null}
flowchart LR
    subgraph inputs["Data Entry Points"]
        direction TB
        ui["Studio & Hub UI"]
        rest["REST API"]
        webhook["Webhooks"]
        schedule["Scheduled Triggers"]
        oauth_cb["OAuth Callbacks"]
    end

    auth["Authentication\n& Authorization"]

    subgraph processing["What Happens"]
        direction TB
        agent_config["Agent & workflow\nconfiguration saved"]
        execution["Workflow\nexecution triggered"]
        file_proc["Files uploaded\n& processed"]
        creds["Credentials stored\nsecurely in Vault"]
    end

    ui --> auth
    rest --> auth
    webhook --> auth
    schedule --> auth
    oauth_cb --> auth

    auth --> agent_config
    auth --> execution
    auth --> file_proc
    auth --> creds

    style inputs fill:#f8f9fa,stroke:#adb5bd,color:#212529
    style processing fill:#e7f0ff,stroke:#3077F3,color:#212529
    style auth fill:#3077F3,stroke:#2060D0,color:#ffffff
    style ui fill:#ffffff,stroke:#6c757d,color:#212529
    style rest fill:#ffffff,stroke:#6c757d,color:#212529
    style webhook fill:#ffffff,stroke:#6c757d,color:#212529
    style schedule fill:#ffffff,stroke:#6c757d,color:#212529
    style oauth_cb fill:#ffffff,stroke:#6c757d,color:#212529
    style agent_config fill:#ffffff,stroke:#3077F3,color:#212529
    style execution fill:#ffffff,stroke:#3077F3,color:#212529
    style file_proc fill:#ffffff,stroke:#3077F3,color:#212529
    style creds fill:#ffffff,stroke:#3077F3,color:#212529
```

| Entry Point            | Authentication                                                                                          | What Data                                                        |
| ---------------------- | ------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------- |
| **Studio / Hub UI**    | JWT session (HttpOnly cookies)                                                                          | Workflow definitions, agent configs, chat messages, file uploads |
| **REST API**           | JWT bearer token                                                                                        | Execution inputs, CRUD operations                                |
| **Webhooks**           | API key + HMAC-SHA256 request signature; the key secret is encrypted at rest, never stored in plaintext | Freeform JSON payload to trigger workflows                       |
| **Scheduled triggers** | Internal (no external entry)                                                                            | Pre-configured input for recurring workflows                     |
| **OAuth callbacks**    | State token verification (CSRF protection)                                                              | Authorization codes from Google/Microsoft                        |

***

## Data leaving the platform

Understanding what data exits your environment is critical for compliance. MagOneAI sends data externally only through two paths: **LLM calls** and **MCP tool calls**.

```mermaid theme={null}
flowchart LR
    subgraph platform["Your Environment"]
        agent["Agent\nActivity"]
        tool["Tool\nActivity"]
    end

    subgraph llm_ext["LLM Providers"]
        direction TB
        cloud_llm["Cloud LLMs\n(OpenAI, Anthropic)"]
        private_llm["Private LLMs\n(vLLM, Ollama, TGI)"]
    end

    subgraph tool_ext["External Tool APIs"]
        direction TB
        google["Google\n(Gmail, Calendar)"]
        microsoft["Microsoft\n(Outlook, OneDrive)"]
        websearch["Web Search"]
    end

    subgraph local_tools["Local Tools — no external call"]
        direction TB
        filetools["File Tools"]
        database["Database / SQL"]
        filesystem["Filesystem"]
    end

    agent -->|"prompts + context"| cloud_llm
    agent -->|"stays in your network"| private_llm
    tool -->|"parameters + credentials"| google
    tool -->|"parameters + credentials"| microsoft
    tool -->|"search queries"| websearch
    tool -.->|"no external call"| local_tools

    style platform fill:#e7f0ff,stroke:#3077F3,color:#212529
    style agent fill:#ffffff,stroke:#3077F3,color:#212529
    style tool fill:#ffffff,stroke:#3077F3,color:#212529

    style llm_ext fill:#fff3cd,stroke:#d4a017,color:#212529
    style cloud_llm fill:#ffffff,stroke:#d4a017,color:#212529
    style private_llm fill:#d4edda,stroke:#28a745,color:#212529

    style tool_ext fill:#fff3cd,stroke:#d4a017,color:#212529
    style google fill:#ffffff,stroke:#d4a017,color:#212529
    style microsoft fill:#ffffff,stroke:#d4a017,color:#212529
    style websearch fill:#ffffff,stroke:#d4a017,color:#212529

    style local_tools fill:#d4edda,stroke:#28a745,color:#212529
    style filetools fill:#ffffff,stroke:#28a745,color:#212529
    style database fill:#ffffff,stroke:#28a745,color:#212529
    style filesystem fill:#ffffff,stroke:#28a745,color:#212529
```

### What goes to LLM providers

| Data Sent                | Description                                                 |
| ------------------------ | ----------------------------------------------------------- |
| **System prompt**        | Agent persona, role, and instructions                       |
| **User input**           | The input data provided to the workflow or chat message     |
| **Conversation context** | Previous activity outputs flowing through the workflow      |
| **Tool schemas**         | Definitions of available tools (function names, parameters) |

<Warning>
  **When using cloud LLM providers**, all of the above data is sent to the provider's API. To keep everything within your environment, use privately hosted models via any OpenAI-compatible endpoint (vLLM, Ollama, LM Studio, TGI, etc.). MagOneAI treats private models identically to cloud models — no workflow changes needed.
</Warning>

### What goes to tool APIs

| Tool                   | Data Sent Externally                   |
| ---------------------- | -------------------------------------- |
| **Google Gmail**       | Email content, recipients, OAuth token |
| **Google Calendar**    | Event details, attendees, OAuth token  |
| **Microsoft Outlook**  | Email content, recipients, OAuth token |
| **Microsoft Calendar** | Event details, attendees, OAuth token  |
| **Web Search**         | Search query text                      |

### Tools that stay local

These MCP tools process data entirely within your environment:

| Tool                 | What It Does                                          | External Calls             |
| -------------------- | ----------------------------------------------------- | -------------------------- |
| **File Tools**       | Extract text from PDFs, Excel, CSV                    | None                       |
| **Database / Vanna** | Query your own databases with SQL or natural language | None (connects to your DB) |
| **Filesystem**       | Read/write local files                                | None                       |

***

## Secrets and credential management

MagOneAI separates sensitive credentials from application data. Credentials are never stored in the application database.

```mermaid theme={null}
flowchart TB
    subgraph app_db["Application Database"]
        config["Non-sensitive config\n(display names, scopes,\nconnection status)"]
        ref["Vault references\n(pointers, not secrets)"]
    end

    subgraph vault["Encrypted Secrets Vault"]
        direction TB
        llm_keys["LLM API Keys"]
        oauth_tokens["OAuth Tokens\n(access + refresh)"]
        tool_creds["Tool Credentials\n(API keys, passwords)"]
    end

    subgraph usage["Used At Runtime"]
        agent_call["Agent calls LLM"]
        tool_call["Tool calls external API"]
    end

    ref -->|"resolve at runtime"| vault
    llm_keys --> agent_call
    oauth_tokens --> tool_call
    tool_creds --> tool_call

    style app_db fill:#f8f9fa,stroke:#adb5bd,color:#212529
    style config fill:#ffffff,stroke:#adb5bd,color:#212529
    style ref fill:#ffffff,stroke:#adb5bd,color:#212529

    style vault fill:#cfe2ff,stroke:#3B82F6,color:#212529
    style llm_keys fill:#ffffff,stroke:#3B82F6,color:#212529
    style oauth_tokens fill:#ffffff,stroke:#3B82F6,color:#212529
    style tool_creds fill:#ffffff,stroke:#3B82F6,color:#212529

    style usage fill:#e7f0ff,stroke:#3077F3,color:#212529
    style agent_call fill:#ffffff,stroke:#3077F3,color:#212529
    style tool_call fill:#ffffff,stroke:#3077F3,color:#212529
```

### How credentials are resolved

When a workflow needs credentials (e.g., to call Google Calendar), MagOneAI uses a **scoped fallback chain**:

```mermaid theme={null}
flowchart TD
    start["Tool needs credentials"]
    user["User-level\ncredentials"]
    project["Project-level\ncredentials"]
    org["Organization-level\ncredentials"]
    found["Credentials found"]
    error["Error: no credentials\nconfigured"]

    start --> user
    user -->|"found"| found
    user -->|"not found"| project
    project -->|"found"| found
    project -->|"not found"| org
    org -->|"found"| found
    org -->|"not found"| error

    style start fill:#f8f9fa,stroke:#adb5bd,color:#212529
    style found fill:#d4edda,stroke:#28a745,color:#212529
    style error fill:#f8d7da,stroke:#dc3545,color:#212529
    style user fill:#ffffff,stroke:#3077F3,color:#212529
    style project fill:#ffffff,stroke:#3077F3,color:#212529
    style org fill:#ffffff,stroke:#3077F3,color:#212529
```

This allows flexible credential management:

* **User-level**: Individual team members connect their own Google/Microsoft accounts
* **Project-level**: Shared credentials for a team (e.g., a shared service account)
* **Organization-level**: Default credentials for the entire org

***

## OAuth integration flow

When connecting to Google or Microsoft services, MagOneAI uses standard OAuth 2.0 with PKCE for security.

```mermaid theme={null}
sequenceDiagram
    participant U as User
    participant M as MagOneAI
    participant V as Secrets Vault
    participant P as OAuth Provider<br/>(Google / Microsoft)

    U->>M: Click "Connect Google Calendar"
    M->>M: Generate PKCE challenge + state token
    M-->>U: Redirect to Google
    U->>P: Authorize MagOneAI
    P-->>M: Callback with authorization code
    M->>M: Verify state token (CSRF protection)
    M->>P: Exchange code for tokens
    P-->>M: Access token + refresh token
    M->>V: Store tokens in encrypted vault
    M-->>U: Connection successful

    Note over M,V: Tokens stored encrypted,<br/>never in application database

    rect rgb(240, 249, 255)
        Note over M,P: Automatic token refresh
        M->>V: Read token (expired?)
        M->>P: Refresh token request
        P-->>M: New access token
        M->>V: Update stored token
    end
```

***

## File processing flow

Files uploaded to MagOneAI are stored in private object storage and processed for use in workflows.

```mermaid theme={null}
flowchart LR
    upload["User uploads\nfile (max 100MB)"]
    validate["Validate type\n& size"]
    store["Store in private\nobject storage"]
    extract["Extract text\n(PDF, Excel, CSV)"]
    embed["Embed into\nvector store\n(if knowledge base)"]
    workflow["Available in\nworkflows"]

    upload --> validate --> store --> extract
    extract --> embed
    extract --> workflow

    style upload fill:#f8f9fa,stroke:#adb5bd,color:#212529
    style validate fill:#fff3cd,stroke:#d4a017,color:#212529
    style store fill:#cfe2ff,stroke:#3B82F6,color:#212529
    style extract fill:#e7f0ff,stroke:#3077F3,color:#212529
    style embed fill:#dce4f5,stroke:#6366F1,color:#212529
    style workflow fill:#d4edda,stroke:#28a745,color:#212529
```

* Files are streamed in chunks to prevent memory issues
* Original files and extracted text are stored separately
* Access is controlled via time-limited signed URLs (no public access)
* Files are scoped to the project — only project members can access them

***

## Human-in-the-loop flow

Workflows can pause for human approval or input, then resume automatically.

```mermaid theme={null}
flowchart LR
    running["Workflow\nrunning"]
    pause["Human Task\ncreated"]
    waiting["Workflow paused\n(no compute used)"]
    notify["Assigned user\nnotified"]
    respond["User approves\nor provides input"]
    resume["Workflow\nresumes"]
    complete["Workflow\ncompletes"]

    running --> pause --> waiting
    waiting --> notify
    notify --> respond
    respond --> resume --> complete

    style running fill:#e7f0ff,stroke:#3077F3,color:#212529
    style pause fill:#fff3cd,stroke:#d4a017,color:#212529
    style waiting fill:#f8f9fa,stroke:#adb5bd,color:#212529
    style notify fill:#fff3cd,stroke:#d4a017,color:#212529
    style respond fill:#d4edda,stroke:#28a745,color:#212529
    style resume fill:#e7f0ff,stroke:#3077F3,color:#212529
    style complete fill:#d4edda,stroke:#28a745,color:#212529
```

<Info>
  When a workflow is paused for human input, **no compute resources are consumed**. The workflow engine (Temporal) durably persists the state and resumes exactly where it left off — even if servers restart in the meantime.
</Info>

***

## Data protection summary

### Sensitive data handling

| Data Type              | How It's Protected                                                                          |
| ---------------------- | ------------------------------------------------------------------------------------------- |
| **User passwords**     | Bcrypt hashed — never stored in plaintext                                                   |
| **Session tokens**     | HttpOnly + Secure + SameSite cookies with short expiry                                      |
| **LLM API keys**       | Stored exclusively in encrypted Vault — never in application database                       |
| **OAuth tokens**       | Stored exclusively in encrypted Vault — auto-refreshed on expiry                            |
| **Tool credentials**   | Split storage: non-sensitive config in database, secrets in Vault                           |
| **Webhook / API keys** | Secret encrypted at rest (never plaintext); requests authenticated by HMAC-SHA256 signature |
| **Uploaded files**     | Private object storage — access via time-limited signed URLs only                           |

### Encryption

| Layer          | Protection                                                                                       |
| -------------- | ------------------------------------------------------------------------------------------------ |
| **In transit** | TLS/HTTPS for all external and client-facing communication                                       |
| **At rest**    | AES-256 encryption in Secrets Vault; database and storage encryption configurable per deployment |
| **Secrets**    | Vault seal mechanism with support for cloud KMS auto-unseal                                      |

### Access control

| Scope                 | Who Can Access                                                           |
| --------------------- | ------------------------------------------------------------------------ |
| **Organization**      | Members of that organization only                                        |
| **Project**           | Project members with appropriate role (Viewer, Operator, Builder, Admin) |
| **Execution data**    | Project members only — isolated per project                              |
| **User credentials**  | Only the user who created them                                           |
| **Org-level secrets** | Organization Owners (and platform Superadmins)                           |

### AI sovereignty

MagOneAI is designed for organizations that need complete control over their AI data:

* **Private LLM support**: Use any OpenAI-compatible model endpoint — your prompts and data never leave your network
* **Self-hosted deployment**: Run the entire platform on your infrastructure (Docker Compose or Kubernetes)
* **Local tools**: File processing, database queries, and filesystem access happen entirely within your environment
* **No telemetry**: MagOneAI does not phone home or send usage data externally

<Tip>
  For maximum data sovereignty, deploy MagOneAI with privately hosted LLMs and use only local tools (File Tools, Database, Filesystem). In this configuration, **zero data leaves your network boundary**.
</Tip>

***

## Next steps

<CardGroup cols={2}>
  <Card title="Secrets management" icon="key" href="/security/secrets-management">
    How Vault integration works and the credential lifecycle
  </Card>

  <Card title="RBAC" icon="users" href="/security/rbac">
    Role-based access control and permission scopes
  </Card>

  <Card title="Audit logging" icon="clipboard-list" href="/security/audit-logging">
    What gets logged and how to review audit trails
  </Card>

  <Card title="Infrastructure" icon="server" href="/platform/infrastructure">
    Deployment architecture and network configuration
  </Card>
</CardGroup>