Skip to main content

​
Overview

Web search tools allow your agents to access real-time information from the internet. While agents have extensive knowledge from their training data, that knowledge has a cutoff date. Web search bridges the gap, enabling agents to:
  • Find current information about companies, people, and events
  • Retrieve the latest news and trends
  • Verify facts against up-to-date sources
  • Research topics not covered in training data
Web search is most effective when combined with agent reasoning. The agent decides what to search for, interprets results, and synthesizes information from multiple sources.

​
Capabilities

MagOneAI’s web search integration provides agents with several search-related capabilities:

​
Search the web

Execute web searches and retrieve ranked results with titles, snippets, and URLs. Parameters:
  • query — search query string
  • num_results (optional) — number of results to return (default: 10)
  • region (optional) — regional search preference (e.g., “us”, “uk”, “de”)
  • language (optional) — language code for results (e.g., “en”, “es”, “fr”)
  • safe_search (optional) — content filtering level
Returns: List of search results with title, URL, snippet, and ranking. Example use case: “Find the latest news about artificial intelligence regulation in the EU.”

​
Retrieve web page content

Fetch and parse the full content of a web page. Parameters:
  • url — web page URL to retrieve
  • format (optional) — return format (text, markdown, html)
Returns: Page content with title, body text, and metadata. Example use case: After finding relevant URLs through search, retrieve full article content for deeper analysis.

​
Extract structured data

Parse web pages to extract specific information like contact details, product specifications, or article metadata. Parameters:
  • url — web page URL to parse
  • schema — expected data structure (JSON Schema format)
Returns: Extracted data matching the specified schema. Example use case: “Extract the company name, headquarters location, CEO name, and employee count from this company’s website.”

​
Configuration

Enable and configure web search tools in your project settings.
1

Enable web search in project settings

Navigate to Project Settings → Tools → Web Search and enable the integration.
2

Configure search parameters

Set default parameters for your project:
  • Default result count: How many results to return per search
  • Default region: Geographic preference for search results
  • Default language: Language for search results
  • Safe search level: Content filtering strictness
3

Set rate limits

Configure rate limits to prevent excessive API usage:
  • Searches per minute per agent
  • Searches per day per project
  • Page retrievals per minute
4

Attach to agents

In agent configuration, enable web search tools. Agents can now use search during reasoning.
Web search consumes API quota. Monitor usage in the analytics dashboard to ensure you stay within rate limits and budget.

​
Use cases

Web search unlocks powerful capabilities across many domains. Here are common patterns and use cases.

​
Sales intelligence

Research companies and contacts before meetings. Example workflow: 
Agent system prompt:
You are a sales research assistant. Before meetings, you research
the prospect's company, recent news, and key contacts.

For each company, find:
1. Recent press releases or news articles
2. Company size, funding, and growth trajectory
3. Key decision makers in the target department
4. Recent product launches or initiatives
5. Potential pain points we can address

Synthesize this into a pre-meeting brief.
Agent actions:
  1. Search for company news: “Acme Corp press releases 2024”
  2. Search for funding information: “Acme Corp Series B funding”
  3. Search for decision makers: “Acme Corp VP Engineering”
  4. Retrieve and parse relevant articles
  5. Synthesize findings into a structured brief

​
Market analysis

Monitor market trends, competitors, and industry developments. Example workflow:
  • Agent searches for industry trend reports
  • Retrieves competitor product pages and pricing
  • Finds analyst opinions and market forecasts
  • Synthesizes information into a market intelligence report

​
Competitive intelligence

Track competitor activities and announcements. Example workflow:
  • Agent monitors competitor websites for changes
  • Searches for competitor mentions in news
  • Extracts product features from competitor pages
  • Compares against your product offerings
  • Generates competitive analysis report

​
Fact checking

Verify claims against current, authoritative sources. Example workflow:
  • Agent receives a claim to verify
  • Searches for authoritative sources on the topic
  • Retrieves full articles from credible publications
  • Cross-references information across multiple sources
  • Returns verification result with source citations

​
Combining with other tools

Web search becomes more powerful when combined with other MagOneAI capabilities.

​
Web search + RAG

Use web search to supplement internal knowledge bases. Pattern:
  1. Agent searches internal documents via RAG (Retrieval Augmented Generation)
  2. If information is missing or outdated, agent searches the web
  3. Agent cross-references external information with internal policies
  4. Agent synthesizes both sources into a comprehensive answer
Example: “What’s our company policy on AI tool usage?” → search internal docs → policy found but mentions specific tools → search web for current tool capabilities → compare with policy → provide informed answer.

​
Web search + Email

Research topics and send summary reports. Pattern:
  1. Agent receives a research request via email or workflow trigger
  2. Agent searches the web for relevant information
  3. Agent retrieves and analyzes source pages
  4. Agent composes email summary with key findings and citations
  5. Agent sends report to stakeholders
Example: Daily competitor monitoring workflow that emails a digest of competitor news each morning.

​
Web search + Database

Enrich database records with external information. Pattern:
  1. Agent queries database for records needing enrichment
  2. For each record, agent searches the web for additional data
  3. Agent extracts structured information from web sources
  4. Agent updates database with enriched data
Example: CRM enrichment workflow that finds company size, industry, and key contacts for new leads.

​
Best practices

Generic instructions like “search the web” lead to poor results. Be specific:✅ Good: “Search for recent press releases from [Company] about product launches in the last 3 months”❌ Bad: “Find information about [Company]”Specificity in agent instructions leads to more relevant search queries and better results.
Search results are raw data. Agents need instructions on how to use them:
When you receive search results:
1. Identify the most relevant and recent results
2. Prioritize authoritative sources (company websites, major news outlets)
3. Retrieve full content from top 3-5 results
4. Cross-reference information across sources
5. Synthesize findings, noting any conflicting information
Not all web information is accurate. Teach agents to validate:
  • Check source credibility
  • Look for multiple corroborating sources
  • Note publication dates (prefer recent information)
  • Flag conflicting information for human review
  • Cite sources in outputs for transparency
Better queries yield better results. Agent should construct queries that:
  • Use specific keywords and phrases
  • Include date ranges when recency matters
  • Use site-specific searches when appropriate (e.g., “site:company.com”)
  • Employ Boolean operators for complex queries
  • Avoid overly broad or ambiguous terms
Web search APIs have rate limits. Design workflows that:
  • Batch research tasks instead of searching in real-time for every query
  • Cache search results for a reasonable time period
  • Use search judiciously — only when internal data is insufficient
  • Monitor usage in analytics to avoid hitting limits
Search can fail (rate limits, network issues, no results). Agents should:
  • Retry with alternative query phrasing if no results found
  • Degrade gracefully to internal knowledge if search unavailable
  • Inform users when searches fail and why
  • Log failures for monitoring and debugging
Web search is most effective when combined with specific agent instructions. Tell the agent exactly what to search for, how to interpret results, and how to synthesize information from multiple sources.

​
Privacy and compliance

Web search accesses public internet data. Consider these privacy and compliance aspects:

​
Data handling

  • Search queries are sent to the search API provider
  • Retrieved content is processed by the agent (LLM provider sees this content)
  • Search results are not permanently stored by MagOneAI (unless you explicitly log them)

​
Sensitive information

Never include sensitive information in search queries. Search queries may be logged by search API providers. Avoid searching for:
  • Personally identifiable information (PII)
  • Confidential business data
  • Authentication credentials
  • Internal system details

​
Compliance considerations

  • GDPR: Web search retrieves publicly available data, but be mindful of how you use personal information found online
  • Copyright: Retrieved content is subject to copyright. Don’t republish scraped content without permission
  • Terms of Service: Respect the terms of service of websites you scrape or retrieve content from

​
Rate limiting and robots.txt

MagOneAI’s web retrieval respects:
  • Standard rate limits to avoid overwhelming target websites
  • robots.txt directives where appropriate
  • HTTP status codes and retry-after headers

​
Troubleshooting

Symptoms: Search returns empty results or very few results.Solutions:
  • Query may be too specific or contain typos
  • Try broader search terms or alternative phrasings
  • Check if safe search settings are too restrictive
  • Verify regional/language settings match content availability
Symptoms: Results don’t match the agent’s intent or user’s needs.Solutions:
  • Refine agent instructions to construct more specific queries
  • Use Boolean operators (AND, OR, NOT) in queries
  • Add date constraints for time-sensitive information
  • Implement result filtering logic in agent reasoning
Symptoms: URLs from search results can’t be retrieved.Solutions:
  • Some sites block automated access
  • Page may require JavaScript rendering (not supported by basic retrieval)
  • Retry after a delay (temporary issue)
  • Some sites require authentication (not accessible via web search)
Symptoms: Tools return “rate limit exceeded” errors.Solutions:
  • Reduce search frequency in workflows
  • Implement caching for repeated searches
  • Increase rate limit quota in project settings (if available)
  • Batch search tasks instead of real-time searches

​
Next steps

Database tools

Combine web search with internal data queries

Custom MCP tools

Build custom search integrations for specialized sources

Agent configuration

Learn how to attach tools to agents

Workflow builder

Design workflows that leverage web search