> ## Documentation Index
> Fetch the complete documentation index at: https://helpcenter.magure.ai/llms.txt
> Use this file to discover all available pages before exploring further.
# Model configuration
> Configure LLM providers and models across the MagOneAI platform
## Model configuration hierarchy
MagOneAI provides fine-grained control over which models are available and who can use them. Configuration flows through four levels, each building on the previous:
**SuperAdmin** configures available LLM providers and models in the Admin Portal. This is the global catalog of models that could potentially be used anywhere on the platform.
At this level, you:
* Add LLM providers (OpenAI-compatible, Anthropic, Google Gemini, self-hosted models)
* Store API keys securely in HashiCorp Vault
* Configure provider-specific settings (endpoints, default parameters)
* Set per-token cost (input/output cost per 1k tokens) for usage analytics
**Organization admins** control which providers and models their organization can access. Not every org needs access to every model.
At this level, you:
* Enable/disable specific providers for your organization
* Set which models from each provider are available
* Apply token-based quotas (see [Usage and quotas](/platform/usage-and-quotas))
* Monitor model usage for your org
**Project settings** can further restrict which models are available within a specific project. This is useful when you want different model access for different use cases.
At this level, you:
* Select subset of org-available models for this project
* Set project-specific token quotas (see [Usage and quotas](/platform/usage-and-quotas))
* Configure default models for agents in this project
**Each agent** uses a specific model selected from the project's available models. This is where you choose the right model for each task.
At this level, you:
* Select the exact model this agent will use
* Configure model parameters (temperature, max tokens, etc.)
* Override defaults for specific agent needs
This hierarchy ensures:
* **Security:** Only authorized organizations can use specific models
* **Cost control:** Organizations can't accidentally use expensive models they didn't approve
* **Flexibility:** Different projects and agents can use different models based on their needs
### How a model is resolved at runtime
At request time, MagOneAI resolves which model to use through a clear precedence order:
**pinned node model → per-turn chat pick → activity's assigned model → workflow default → org default**
* **Pinned node model** — when an agent node sets `force_llm_config: true` together with an `llm_config_id`, that node is locked to its model and nothing overrides it (see below).
* **Per-turn chat pick** — a model chosen on the chat surface for this message, constrained to the project's permitted models.
* **Activity's assigned model** — the `llm_config_id` set on the agent node.
* **Workflow default** — the use case's default model.
* **Org default** — the one LLM config the organization has marked as default.
Admins control which models each project is allowed to use, so resolution always stays within the project's permitted set. Standalone runs (no use case) skip the workflow level and fall back directly to the org default.
The model picker in the Builder and Hub now defaults to **Auto (default assigned model)**. Auto sends no override, so each agent uses its own assigned model — choosing a specific model is an explicit, per-turn opt-in. See [Usage and quotas](/platform/usage-and-quotas) for how model selection interacts with token quotas.
#### Per-node model pinning
By default a per-turn chat pick wins over an agent's assigned model. To guarantee a specific node always runs on a specific model regardless of what the user selects, set both `llm_config_id` and `force_llm_config: true` on the agent node.
Use this to keep sensitive steps on a specific model — for example, a confidentiality or refusal gate that must stay on a self-hosted model so sensitive text never leaves your perimeter, or a per-role agent whose model is decided by the role rather than the user. Pinning is opt-in: without `force_llm_config` the per-turn pick still wins.
## Provider management in Admin Portal
SuperAdmins manage all LLM providers through the Admin Portal. This is the central configuration point for the entire platform.
### Adding a provider
In the Admin Portal, open the provider configuration and click **Add Provider**.
Choose from:
* **OpenAI-compatible** — covers OpenAI itself, Azure OpenAI, self-hosted vLLM, Ollama, and any endpoint that speaks the OpenAI API (this is also how you configure self-hosted models)
* **Anthropic** — native integration
* **Google Gemini** — native integration via a Google AI Studio API key
Provide:
* Provider name (for identification)
* API endpoint URL (for cloud providers, this is pre-filled)
* API key or authentication credentials
* Optional: default parameters, timeouts, retry settings
API keys are automatically stored in HashiCorp Vault. You'll see a reference like `vault:openai/api_key` instead of the actual key.
Click **Test Connection** to verify MagOneAI can reach the provider and authenticate successfully.
Choose which organizations have access to this provider. You can enable for all orgs or specific ones.
### Quotas
MagOneAI enforces **token-based quotas** (daily, weekly, and monthly) at the org, project, use-case, and user scope. Quotas are configured and managed on the dedicated quotas surface rather than here.
See [Usage and quotas](/platform/usage-and-quotas) for how to set and monitor token quotas.
### Usage monitoring
Track model usage across the platform:
* **Token usage:** Input and output tokens per provider, org, and project
* **Per-token cost:** When you set input/output cost per 1k tokens on a model, usage analytics surface token-based cost
* **Latency:** Response times by provider and model
* **Model popularity:** Identify which models are most/least used
Usage and cost analytics appear in the analytics dashboards — see the [Admin Portal](/platform/admin-portal).
## Model selection per agent
Each agent in MagOneAI uses a specific LLM. Choosing the right model for each task is crucial for balancing cost, speed, and capability.
### How to select a model
The model picker defaults to **Auto (default assigned model)** — Auto sends no override, so the agent uses its own assigned model. Picking a specific model is an explicit per-turn opt-in.
When creating or editing an agent:
1. **Choose from available models:** Dropdown shows models available to your project (with **Auto** as the default)
2. **Consider the task requirements:**
* Complex reasoning → large, capable models (GPT-4o, Claude Opus 4.6)
* Simple classification → small, fast models (GPT-4o-mini, Claude Haiku)
* Vision tasks → multimodal models (GPT-4o, Gemini 2.5 Pro)
3. **Configure model parameters:**
* **Temperature:** Creativity vs consistency (0.0 - 1.0)
* **Max tokens:** Maximum response length
* **Top-p:** Alternative sampling method
* **Frequency/presence penalty:** Control repetition
### Choosing the right model for the task
Different tasks require different model capabilities:
**Use:** GPT-4o-mini, Claude Haiku, Gemini Flash
**For:**
* Classification (spam detection, sentiment analysis)
* Routing (which agent handles this request?)
* Simple Q\&A
* Data extraction from structured formats
**Why:** 10x cheaper, 5x faster, good enough for simple tasks
**Use:** Claude Opus 4.6, o1, GPT-4o
**For:**
* Multi-step reasoning
* Complex analysis and synthesis
* Strategic decision-making
* Nuanced understanding of context
**Why:** Superior reasoning, worth the cost for complex tasks
**Use:** GPT-4o, Gemini 2.5 Pro, Qwen3-VL
**For:**
* Document OCR and analysis
* Image understanding
* Visual question answering
* Chart/diagram interpretation
**Why:** Native vision capabilities, no separate OCR step needed
**Use:** Different models for different agents in the same workflow
**For:**
* Routing agent (fast model) → Analysis agent (reasoning model)
* Vision agent (multimodal) → Summary agent (text model)
* Classifier (cheap model) → multiple specialized agents
**Why:** Optimize cost and performance for each step
### Model parameter tuning
**Temperature** controls randomness in responses.
* **0.0 - 0.3:** Deterministic, consistent — use for classification, extraction, structured output
* **0.4 - 0.7:** Balanced — default for most tasks
* **0.8 - 1.0:** Creative, varied — use for content generation, brainstorming
**Example:** A customer support classifier should use temperature 0.0 for consistent categorization. A marketing copy generator can use 0.8 for creative variety.
**Max tokens** sets the maximum response length.
* Short responses (100-500 tokens): classifications, summaries
* Medium responses (500-2000 tokens): explanations, analyses
* Long responses (2000-4000+ tokens): comprehensive reports, documents
**Tip:** Set max tokens appropriately for your use case. Unnecessarily high limits increase cost without benefit.
**Top-p** (nucleus sampling) is an alternative to temperature for controlling randomness.
* 0.1 - 0.5: Conservative, high-probability tokens only
* 0.5 - 0.9: Balanced
* 0.9 - 1.0: Diverse, includes lower-probability tokens
Generally use temperature OR top-p, not both. Top-p is often more predictable than temperature at extreme values.
**Frequency penalty:** Reduces likelihood of repeating tokens based on how often they've appeared.
**Presence penalty:** Reduces likelihood of repeating tokens that have appeared at all.
Both range from -2.0 to 2.0. Positive values discourage repetition, negative values encourage it.
**Use case:** Set frequency penalty to 0.5-1.0 for content generation to avoid repetitive phrasing.
### Where parameters live: extra\_parameters
Sampling parameters like `temperature`, `top_p`, `seed`, and `max_tokens` are not a fixed schema in MagOneAI. Each LLM config carries a free-form **`extra_parameters`** map (up to 50 keys) that an admin fills with the parameters they have verified work with that model. These merge into every request as a base layer; per-activity overrides on an agent node (`temperature`, `top_p`, `max_tokens`, or `extra_param_overrides`) layer on top and win. Because it is free-form JSON, providers can add or drop parameters without any schema migration on MagOneAI's side.
## LLM config capabilities
Beyond the model name, endpoint, and cost, each LLM config declares a few capability and behavior settings that shape how MagOneAI calls the model.
`context_window_tokens` is the model's maximum input size. For known **cloud** models it is auto-filled at create/update time from a community model map, and admins can override it; an unknown cloud model falls back to a **32K** default (safe middle ground, high enough for normal prompts, low enough not to overflow a smaller unknown model).
**Self-hosted models never use the default.** Their window is auto-detected from the endpoint when the server advertises one — vLLM exposes `max_model_len` on its `/v1/models` listing — and is otherwise required in the UI. The value drives context-budget file routing: small inputs are inlined into the prompt, large inputs are indexed and retrieved instead.
`supports_thinking` declares how a model exposes reasoning: `none`, `on_off` (a simple thinking toggle), or `effort` (low / medium / high depth). Activities turn it on per call with `thinking_enabled` and an optional `thinking_effort`.
The gateway translates one internal directive into each provider's wire format: OpenAI o-series → `reasoning_effort`, Anthropic → a thinking block, Gemini → `thinkingConfig`, and self-hosted Qwen3 models → `chat_template_kwargs.enable_thinking`. Configs left unverified pass through unchanged so existing workflows keep running.
Explicit prompt caching reuses a stable prompt prefix across calls to cut cost and latency. Set it per agent with `prompt_cache_ttl`: absent/None = default (5-minute cache, on), `1h` = one-hour cache, `off` = disabled.
Providers are handled where the feature exists: **Anthropic** honors the duration via `cache_control` breakpoints on the stable system/tools prefix, **OpenAI** receives a stable routing key for its automatic cache, and **self-hosted / Gemini** cache automatically with no request directive. A global kill-switch disables explicit caching platform-wide; provider-side automatic caching is independent of it.
`supports_vision` marks a config as able to accept images (and native documents such as whole PDFs). It also carries vision-specific limits — a maximum image size, a default image detail level, and a longer request timeout, since vision calls take more time than text-only calls.
## Function calling support
For agents to use tools, the underlying model must support **function calling** (also called tool use).
### What is function calling?
Function calling allows models to:
1. Receive a list of available tools with their parameter schemas
2. Decide during reasoning when to call a tool
3. Generate a tool call with appropriate parameters
4. Receive the tool's response and continue reasoning
Without function calling, agents can't use MCP tools for actions like sending emails, querying databases, or calling APIs.
### Models with function calling support
**Full support:**
* OpenAI: GPT-4, GPT-4o, GPT-4o-mini, o1, o3-mini
* Anthropic: All Claude models (function calling via "tool use")
* Google: Gemini 2.0 Flash, Gemini 2.5 Pro
**No function calling:**
* Base LLaMA models (unless fine-tuned)
* Some older or smaller open-source models
### How MagOneAI converts tool definitions
When you attach tools to an agent, MagOneAI converts MCP tool definitions to the provider's function calling format:
**MCP tool definition:**
```json theme={null}
{
"name": "send_email",
"description": "Send an email to recipients",
"parameters": {
"type": "object",
"properties": {
"to": {"type": "array", "items": {"type": "string"}},
"subject": {"type": "string"},
"body": {"type": "string"}
},
"required": ["to", "subject", "body"]
}
}
```
**Converted to OpenAI function calling format:**
```json theme={null}
{
"type": "function",
"function": {
"name": "send_email",
"description": "Send an email to recipients",
"parameters": {
"type": "object",
"properties": {
"to": {"type": "array", "items": {"type": "string"}},
"subject": {"type": "string"},
"body": {"type": "string"}
},
"required": ["to", "subject", "body"]
}
}
}
```
This conversion happens automatically. You define tools once in MCP format, and they work across all providers that support function calling.
### Agent types and function calling
MagOneAI agents have different function calling requirements depending on their configuration:
* **Basic agents:** No tools, no function calling needed
* **Tool agents:** Require function calling — agent decides when to use tools
* **Full agents:** Require function calling — agent uses tools during complex reasoning
If you select a model without function calling for a Tool or Full agent, you'll see a warning during configuration.
**Use the smallest model that reliably handles the task.** A fast model for initial classification, a powerful model for deep analysis, and a vision model for documents — all in the same workflow — optimizes both cost and performance.
## Cost optimization strategies
Model selection has significant cost implications. Use these strategies to minimize spending while maintaining quality:
### 1. Tier your agents by model size
* **Tier 1 (entry points):** GPT-4o-mini or Claude Haiku for routing and classification
* **Tier 2 (specialists):** GPT-4o or Claude Sonnet for most tasks
* **Tier 3 (experts):** Claude Opus 4.6 or o1 only for complex reasoning
Route requests through Tier 1, escalate to higher tiers only when needed.
### 2. Compress prompts and context
* Remove unnecessary whitespace and verbose instructions
* Summarize long documents before including in context
* Use structured formats (JSON, tables) instead of verbose prose
* Prune irrelevant context from multi-turn conversations
### 3. Monitor usage regularly
Review usage analytics to identify optimization opportunities:
* Agents using expensive models more than necessary
* High-volume tasks that could run on a cheaper model
* Token-based cost trends across orgs and projects
See [Usage and quotas](/platform/usage-and-quotas) for token quotas and the [Admin Portal](/platform/admin-portal) for usage analytics.
### 4. Use private models for high-volume tasks
If you have consistent, high-volume workloads, deploying private open-source models can be significantly cheaper than cloud APIs:
* High upfront cost (GPU infrastructure)
* Near-zero marginal cost per request
* Breakeven typically at 10M+ tokens per month
See [Private model deployment](/models/private-models) for details.
## Next steps
Configure OpenAI, Anthropic, and Google
Deploy open-source models on your infrastructure
Learn how to configure agents with models
Monitor token usage and set quotas