Mistral’s Agentic Search gives RAG permission to keep looking

2026-08-22

Maestro’s take: Basic one-shot RAG systems get one pile of search results and are told to answer from it. Mistral’s new Agentic Search lets the model search again, open the document, move to the right page, read the evidence, and admit the first result was not enough. Revolutionary? No. Sensible? Extremely.

TL;DR

Traditional RAG usually searches once, retrieves a few chunks, and asks the model to answer from them.

Mistral Agentic Search adds a loop. The model can use five tools: search, open, navigate, read, and grep. It can inspect a result, refine the query, follow a reference, and verify the answer before responding.

In Mistral’s vendor-run, out-of-the-box Search Toolkit tests using GLM-5.2, the full navigation loop raised FinanceBench correctness from 26.7% for one-shot RAG to 86%. On OfficeQA Pro, the same tested GLM-5.2 setup rose from 6.3% to 51.9%. Mistral also reports lower token use and tail latency after adding targeted navigation.

These are vendor-run benchmarks. They are not a promise that your chatbot will become eight times better.

The practical lesson is smaller: use one-shot retrieval for simple lookups. Use an investigation loop when the answer may be buried across pages, tables, footnotes, or several documents.

The diagram explains the retrieval loop. It is not a performance chart.

One search is sometimes enough

Suppose a user asks, “What is our vacation policy?”

If the answer lives in one clean handbook section, normal retrieval may work perfectly. Search the index. Return the relevant passage. Answer with a citation.

Adding an autonomous search loop here creates more latency, cost, and ways to wander.

Now suppose a user asks, “Why did cloud spending rise even though customer traffic fell?”

The answer may require a billing export, an incident report, and a deployment log. Assume those sources are already indexed and the user is authorized to read them. Agentic Search cannot investigate data it cannot access.

The first search result is only a lead. That is where an agentic loop helps.

What Mistral added

The existing search index still does the first useful job: finding likely documents.

Then the model can:

1. Search for likely sources.
2. Open a promising document.
3. Navigate to a page, section, or table.
4. Read the exact material.
5. Grep for a name, number, or phrase.
6. Search again if the evidence is incomplete.

This is closer to how a careful developer researches a bug. You do not stop because one file contains the right keyword. You open it, follow the call, inspect the log, and check whether the explanation survives contact with evidence.

The benchmark is encouraging, not magical

Mistral tested its system on long financial filings and scanned Treasury bulletins. These are hard retrieval problems with tables and exact numeric answers.

The largest gain came from allowing repeated search. Navigation added another improvement and, in Mistral’s tests, reduced wasted tokens compared with repeated broad searches.

That does not prove agentic search is always cheaper. Easy questions may take longer because the system has more tools and more possible steps. Mistral’s setup, documents, models, judge, and defaults may not match yours.

Treat the published numbers as a reason to test the pattern—not as your expected production result.

Why Maestro users should care

Coding agents already investigate repositories this way. They search, open files, follow references, run tests, and revise their theory.

Document assistants should not become less careful merely because the source is a PDF.

Maestro’s inference: the same pattern belongs in internal knowledge tools. Give the agent a bounded way to keep looking, but require it to show where the answer came from. The goal is not more searching. The goal is enough evidence.

Do this

• Preserve the retrieval layer. Then verify that the index stores ordered chunks and source offsets needed for navigation. Mistral’s implementation requires a NavigableIndex.
• Add an investigation loop only for questions that need it.
• Require page, section, file, or record references in the answer.
• Limit steps, time, and allowed data sources.
• Test with questions your current system gets wrong.
• Measure correct answers first. Then compare tokens and time.

Do not do this

• Do not replace every simple lookup with a wandering agent.
• Do not let the model browse data the user cannot access.
• Do not accept a citation merely because it exists. Check that it supports the claim.
• Do not judge success by how many searches the agent performed.
• Do not call a vendor benchmark your production result.

One thing to try

Collect ten questions your document bot answered incorrectly.

For each one, allow a second search after the model reads the first result. Give it one open tool and require an exact source location.

Then compare:

• Was the final answer correct?
• Did the cited passage actually support it?
• How many tokens and seconds did it take?

If the hard questions improve, keep the loop for that class of work. If easy questions only become slower, leave them alone.

That is the simple rule: direct lookup for known facts; investigation for questions that require a trail.

Sources considered

• Mistral: Introducing Agentic Search — product design, five tools, availability, benchmarks, token use, latency, and stated limitations.
• Mistral Agentic Search documentation — navigable-index requirement, MCP interface, tool behavior, deployment details, and loop limits.

Published August 22, 2026. Mistral’s performance figures are vendor-run. Maestro’s opinions and summaries are AI-generated. A separate AI editor reviewed this article for evidence, scope, and reader value.