SmartGateSmartGate

Agentic Search vs RAG: A Tool Call or an Index You Own

An agent's search tool and a RAG index are both retrieval, and they differ in four places that decide everything downstream: what they read (the live web against a corpus you ingested), who owns the ranking (an engine you do not operate against a pipeline you configured), how fresh the answer can be, and what a call costs.

Short answer: An agent's search tool and a RAG index are both retrieval, and they differ in four places that decide everything downstream: what they read (the live web against a corpus you ingested), who owns the ranking (an engine you do not operate against a pipeline you configured), how fresh the answer can be, and what a call costs. A search tool trades control and predictability for reach and freshness; an index trades reach for a ranking you can measure. Systems that work at scale run both and route per question rather than choosing once.

Key takeaways

  • The generation step is identical. Both paths end in retrieved text placed in a prompt. Every difference is upstream of the model, which is why the choice is an engineering decision.
  • Ranking ownership is the sharpest split. A search tool borrows someone else's relevance model; an index is only as good as the chunking and embedding you chose.
  • The cost curves are opposites. An index is a fixed cost at ingest and almost free per query; a search call is free to build and billed per invocation.
  • Freshness is what the variable cost buys. "What changed this week" is a search question; "what our policy says" is an index question, because only the index can cite an exact revision.
  • Route by question, not by preference. Freshness, privacy and repeat rate are three signals you can evaluate before generation, which makes routing implementable rather than aspirational.
  • Do this next: take twenty real questions, mark each as freshness-dependent or corpus-bound, and count the split. That ratio is the only input the routing decision needs.

The generation step is the same; the retrieval path is not

Both arrangements end the same way: retrieved text is placed in a prompt and the model generates from it, which is what the original RAG paper formalised. Everything that differs happens before the prompt is assembled. A search tool hands the model documents — a title, a URL, a snippet and often a date — ranked by a search engine whose ranking function you cannot read and whose index you cannot inspect. An index hands the model passages: fragments of your own documents with similarity scores produced by an embedding model you chose over a chunking scheme you chose. The first is a window on a corpus nobody owns. The second is a window on a corpus you built and are therefore responsible for.

That both are called retrieval is not an accident of vocabulary; it is why the two are usually separate tools in a working system rather than two settings on one component. The research literature makes the same split when it separates a fixed retriever from a model that decides when to search, and when it measures how many searches an answer took (learned search use). The failure modes, the cost curve and the evaluation method all differ, and the difference starts with the request itself.

A search request is small enough to be serialised, logged, diffed and replayed. The function below writes only the values that differ from a default, and returns early when one field makes the rest irrelevant:

# lib/dashboard/logs-url.ts — source lines 31–45 (serializeLogsSearchParams)
function serializeLogsSearchParams(state: LogsViewState): URLSearchParams {
  const qs = new URLSearchParams();
  if (state.primaryMode === "task") {
    qs.set("mode", "task");
    return qs;
  }
  if (state.present === "table") {
    qs.set("present", "table");
    return qs;
  }
  if (state.density === "terminal") {
    qs.set("density", "terminal");
  }
  return qs;
}

Read it as a property of the request rather than as a UI helper. A query that can be expressed as a handful of parameters is a query you can put in a URL, paste into a bug report, replay against a different day, and cache by its own serialisation. An embedding is none of those things: it is an opaque vector, you cannot hand-edit it, and two queries that mean the same thing in different words produce unrelated vectors. That asymmetry is the practical reason retrieval tuning looks so different on the two paths — on the search side you tune the query text and the parameters beside it, and on the index side you tune the pipeline that produced the vectors, because the query is generated rather than written.

Retrieval-augmented generation over a corpus you own

Owning the corpus is the index path's whole proposition, so it is worth being precise about what ownership buys and what it costs. It buys recall you can measure on labelled questions, a ranking you can change by editing chunk size, overlap, metadata filters or a reranker, and determinism: the same query returns the same passages until somebody reindexes. It also buys a cost structure — the expensive embedding happens at ingest, so queries are cheap — and privacy, because the corpus never has to leave your boundary to be searched.

What it costs is freshness and reach. An index can only answer from what was ingested, so its knowledge has a cadence rather than a date, and a question about last week is a question it cannot answer no matter how good the retrieval is. Long documents have to be split to be retrievable, which means the unit of evidence is a fragment whose surrounding context may live in a neighbour that was not retrieved — the failure mode the retrieval-side surveys spend most of their length on (RAG for large language models).

Every knob in that pipeline is a number you chose and can defend, and the shape of a defensible choice is the same everywhere: a named constant with a default, a band it must sit inside, and a refusal to start with a value outside the band. The snippet below is not a retrieval parameter itself — it is a billing blend ratio — but it is the shape of every tuning knob on the side of the stack you own:

# config/billing-savings-share.ts — source lines 7–14 (parseSearchKappa)
function parseSearchKappa(): number {
  const raw = process.env.BILLING_SEARCH_KAPPA;
  const n = raw == null || raw === "" ? 0.3 : Number(raw);
  if (!Number.isFinite(n) || n < 0 || n > 1) {
    throw new Error(`Invalid BILLING_SEARCH_KAPPA: ${raw}`);
  }
  return n;
}

The contrast is with the knobs you do not own. A search tool has a recency filter, a result count and a provider's defaults, and behind them sit a ranking function, a crawl schedule and a spam policy you can neither read nor change. When a search tool starts ranking a competitor's page above the page you wanted, there is no constant to edit. When the blend ratio above is invalid, the process refuses to boot, which is the honest behaviour for a parameter that silently changes every number a customer sees. That is the difference between the two paths in one pair of functions: on one side a value in your configuration with a validation boundary, on the other a black box whose behaviour changes on somebody else's release schedule. Which parameters exist at all is a design decision rather than an accident, and the index's configuration surface is where a retrieval stack lists them.

LangChain's retriever and search tool: two interfaces, one graph

Frameworks make the distinction mechanical, and that is their useful contribution here. A retriever is an object your code calls: typed input, documents out, invoked exactly once by the chain that needs evidence, testable in isolation, and able to fail loudly without confusing anything. A search tool is an action the model chooses: the model writes the arguments as text, the runtime validates them against a schema the model was shown, and the result has to be rendered back into the context as text the model can read. In LangChain, as in the other agent frameworks, both can live in the same graph, and wrapping a retriever as a tool is the standard bridge between them.

Three consequences follow, and none of them is cosmetic.

  • The retriever can be evaluated offline; the tool cannot. Retrieval quality is a property of the index and the query string, so it is measurable on a labelled set. Tool quality includes the model's ability to write a good query, which only shows up in end-to-end traces.
  • The retriever's call count is fixed; the tool's is not. One request, one retrieval, by construction. A tool may be called zero times on a question that needs no evidence and six times on one that does, which is the entire cost story of an agentic system in one sentence.
  • The retriever's parameters are invisible to the model. Top-k, filters and thresholds live in your code. Every parameter you add to a tool's schema is prompt surface: tokens in every request, and one more argument the model can get wrong.

The learned variants sit between the two: a model trained to decide when to retrieve and to critique what came back (retrieve, generate, critique) behaves like a retriever with judgement attached, while a reasoning-acting loop keeps the decision in the prompt (reasoning and acting) — that loop is how an agentic loop works in the cluster's own words. Which of the three you build depends on how often the right answer is "do not retrieve", and that frequency is measurable in your own traffic before you commit to an architecture.

What an agentic search call costs, and how to count it

The two paths have opposite cost curves, and the difference is large enough that it should be visible in the accounting before it is visible in a bill. An index is expensive once — embedding the corpus, and re-embedding it when the model or the chunking changes — and then nearly free per query, because a similarity lookup is the cheapest operation in the stack. A search call is free to build and billed every time it runs, sometimes twice, because a good search tool also fetches the pages it decided were worth reading. Freshness is what that variable cost buys, and it is the only thing a fixed index cannot buy at any price.

Counting it honestly means one record per call, with a correlation identifier so that calls can be folded back into the answer they belong to. The builder below writes exactly that kind of entry:

# dashboard-calibration/dashboard_calibration/entry_builder.py — source lines 43–56 (build_search_entry)
def build_search_entry(
    *,
    tokens: int,
    route: str,
    ts_day: str,
) -> tuple[str, dict[str, Any], int]:
    request_id = str(uuid.uuid4())
    params = {
        "route": route,
        "correlation_id": f"cal-{ts_day}-{request_id[:8]}",
        "agent_platform": AGENT_PLATFORM,
        "query": "calibration mock search",
    }
    return request_id, params, tokens

Three fields in that structure are the whole accounting argument. The request identifier makes the row unique, so a retried call is two rows rather than one and a lost result is visible. The correlation identifier ties the row to a day and an id prefix, which is what turns "calls per answer" from an estimate into a query. And the token count is carried on the row rather than inferred from the query length, because a search result is only free until you put it in a prompt — at which point the retrieved pages become the dominant term in the bill, which is where fitting evidence into context stops being a comfort argument and becomes arithmetic.

Latency deserves the same treatment. An index query's p95 is a property of your own infrastructure, so it is tunable, cacheable and measurable on a graph you control. A live search's p95 is somebody else's rate limiter and somebody else's crawl, and no amount of caching fixes a tail that is decided outside your boundary. Cost per completed answer is the single number that makes the comparison fair, because the cheap path that needs three attempts is not the cheap path.

Implementation: choosing the retrieval path per question

The choice is not architectural in the sense of being permanent; it is a routing rule that can be written down and tested. Five signals decide it, and all five are knowable before generation.

  1. Freshness. If the true answer changed this week and the change matters to the reader, the live search is the only path that can be correct. If the answer is a policy that changes twice a year, freshness is not a reason to pay per call.
  2. Ownership of the truth. If the answer lives in documents you control — product documentation, tickets, contracts, runbooks — index them and retrieve from the index, because only there can you guarantee that the clause you cite is the clause that is currently in force.
  3. Privacy and licence. Private data must not be sent to a third-party engine, and licensed material must not be copied into an index. Those two constraints point in opposite directions, and a system that ignores either one fails an audit rather than a test.
  4. Repeat rate. A question asked a thousand times should be answered from an index or a cache once the corpus covers it. Paying a third party per call for the same question is a choice you can only justify while the corpus is thin.
  5. Precision of citation. A reference answer wants a passage with a stable identifier. A snippet from a page that may be rewritten tomorrow is evidence that expires with the page.

The hybrid is the pattern most teams converge on, and it is worth naming because it grows the index rather than replacing it: search once for pages that look authoritative, fetch them, store them, and retrieve from the store afterwards. The first question pays the search cost and every subsequent one pays the index cost. What the pattern does not do is remove the corpus's obligations — once you are storing fetched pages you own a freshness cadence and a recall problem again, and the rules of the ingestion and retrieval pipeline apply to pages you scraped exactly as they apply to documents your team wrote.

Tool definition: the contract a search tool exposes

A search tool is defined by its schema, and the schema is prompt surface, so it is worth keeping small and explicit. A defensible one declares the query as natural language, the number of results, an optional recency window, an optional domain restriction, and an output shape that carries the title, URL, snippet and publication date of each result. The date matters more than it looks: it is the only field that lets the model reason about whether the page is current, and it is the field most tools quietly omit. A timeout, a cost annotation and a structured failure response belong in the contract too, because a loop that cannot tell a rate limit from an empty result will retry the wrong one. How a tool is declared is the reference for the mechanics of that schema.

The contrast with a retriever's interface is instructive in both directions. A retriever has no schema the model sees: its signature is fixed in your code, changing it means a code review rather than a prompt change, and adding a filter costs nothing at inference time. A tool's signature is read by the model on every request, so every parameter is tokens and every optional field is a chance to be misused. That is the reason good search tools look austere: one query, one count, and perhaps one filter for freshness — with the rest of the tuning moved into the application, where it costs nothing to run and can be tested.

Architecture: freshness, ranking ownership and cost per call

Laid out side by side, the two paths differ on almost every operational dimension, which is why "search or RAG" is usually the wrong question and "which of these two do I need for this question" is the right one.

Dimension A search tool (agentic search) A retrieval index (RAG)
What is read The live web, through a provider's index Fragments of a corpus you ingested
Who owns the ranking A search engine you do not operate Your chunking, embedding model and filters
Freshness Minutes to days, decided outside your boundary Your ingestion cadence, decided by you
Cost shape Per call, plus per page fetched Fixed at ingest, near zero per query
Cacheability Weak — the same query can return different pages Strong — the same query returns the same passages until a reindex
Failure mode Rate limits, empty result sets, pages that changed under you Missing evidence, stale fragments, recall that nobody measured
Right for Current events, public facts, open-ended discovery Policies, product knowledge, private documents, exact citations

The table also names the two failure modes that surprise teams most. A search-backed answer fails by being untraceable: the page moved, or the snippet lost the clause that mattered, and the transcript no longer explains the answer. An index-backed answer fails by being confidently narrow: the fragment that came back is real, current and insufficient, and nothing in the pipeline notices. Both failures are cheaper to detect with instrumentation than with prose, which is the argument for measurement on the same page as the choice, the pipeline-versus-loop decision being the workload-level version of it and the survey of retrieval loops naming the pieces each path shares.

That framing is the workload-level half of a larger split: rag vs agentic ai is where the vocabulary itself is separated, so a choice made here is not mistaken for a choice about the architecture above it.

How SmartGate compares

The question this page answers is where the retrieval path, its limits and its record live, and whether a single endpoint can carry both paths without the accounting splitting in two.

What it decides What that means for a mixed workload
A vector store and a client library What the index returns Cheap per query, blind to anything that happened after the last ingest
A provider SDK called directly What the search provider returns Fresh by construction, billed per call, with the ranking and the rate limit outside your control
A bespoke proxy in front of both Whatever it counts Two accounting paths that drift apart exactly when a cost problem appears
SmartGate When each tool is called, with per-key limits, a spend guard and one audit row per call A search call and a retrieval call produce the same shape of record, so cost per completed answer is a query rather than a reconciliation

The free tier covers 2,000,000 tokens a month and all seven tools; Pro starts at 18 dollars a month and Teams at 55 dollars a month. Those limits sit on the call path itself rather than in a dashboard, which is what makes them comparable across the two retrieval paths this page is about — where tool calls are metered is the longer version of that argument, and what a run remembers is what keeps a long search-heavy run from re-reading the same pages.

How to get started

  1. Classify twenty real questions. Freshness-dependent or corpus-bound, and write the ratio down. Everything else on this page is downstream of that number.
  2. Measure the index path first, because it is cheap to measure. Recall at k on a labelled set, plus the ingestion date of the fragments that were retrieved. An index without a freshness field is an index nobody can route around.
  3. Instrument the search calls before you add a second one. One row per call with a correlation identifier and a token count. Without the record, the comparison between the paths is a preference.
  4. Write the routing rule as code, not as a guideline. A cheap classifier for freshness and ownership in front of the retrieval decision, with both paths behind the same tool surface.
  5. Re-measure when the corpus changes. A workload that needed live search last quarter is often answerable from the index once the pages it kept fetching have been ingested, and the repeat rate is what tells you when.

Start on the free tier — 2,000,000 tokens a month and all seven tools — with start free; the plan limits are on the pricing page, the tool parameters are in the product docs, and contract traffic starts at the contact form.

Frequently Asked Questions

Limitations and what this does not do

  • Three of the seven planned sections carry code. The LangChain comparison, the routing rules, the tool contract and the architecture comparison matched no unique symbol in this codebase and are written from the published research with no quoted implementation, as the abstain rule requires.
  • The quoted code is the plumbing around a search call, not a search engine. The snippets show how a request is serialised, how a configuration constant is validated and how one call is recorded for calibration. None of them ranks, fetches or summarises results, and the page does not claim otherwise.
  • The comparison table is a shape, not a benchmark. It names the dimensions on which the two paths differ so that you can measure your own; it reports no numbers, because the latency and cost that matter are the ones on your deployment against your providers.
  • The routing rules are a skeleton. Freshness, ownership, privacy, repeat rate and citation precision are the signals worth evaluating; the thresholds that turn them into a classifier are properties of your traffic and your corpus, and this page does not invent them.
  • This page is not a build guide for either path. The loop that calls a tool and the stages inside an index each have their own page in this cluster, and neither is reproduced here.

Sources

Method note

The code in this article is not transcribed. Each block was cut directly out of the slice body returned by the SmartGate slice API and re-asserted byte-for-byte as a substring of that body before publication; the first line inside every fence records the file and the exact source lines. Symbols were pinned with whole-name containment (rule A level 2) and confirmed by the service's slot-proof endpoint — 3 of the 7 planned sections matched; the other four are written from the published research with no code. The three pinned slices are the parts of a search call that a retrieval decision actually depends on: the request as a set of serialisable parameters, a tuning constant with a validated range, and the per-call record that makes cost per answer a query. They are evidence about what each path leaves you in control of, not an implementation of a search tool.

Slice provenance

# SERP keyword Symbol File Source lines How it was pinned sha256(12)
1 agentic retrieval augmented generation a survey on agentic rag serializeLogsSearchParams lib/dashboard/logs-url.ts 31–45 rule A L2 → slot-proof 09b90cf6849f
2 agentic retrieval-augmented generation: a survey on agentic rag parseSearchKappa config/billing-savings-share.ts 7–14 rule A L2 → slot-proof 741541c43d1a
3 rag vs agentic build_search_entry dashboard-calibration/dashboard_calibration/entry_builder.py 43–56 rule A L2 → slot-proof 34911a2425fc

Every fenced block above was cut from the slice body and re-asserted against it byte-for-byte before publication. 3 of 7 sections pinned, 0 abstentions, and 4 section(s) written from external sources because no unique symbol matched.