Web search providers

Web search and page reading in DataMind Oda run through a provider: the service that answers a search or fetches a page. This page covers the provider layer for deployments and developers: where its settings live, how Oda chooses a provider, the keys it uses, and what its failures mean. The tools themselves are on Web search and page reading, and the everyday side is on Chat with Oda.

Where the web search settings live

• The Web search section. The provider's settings live under Settings > Plugins > Plugin configuration > Web search, which renders the Endpoint field. A failed search points you here, and the recovery text ends Only the user should choose or change the endpoint.
• The plugin entry. The provider's plugin entry is titled Web Search Provider.
• No reload needed. Saving the endpoint serves the next search under the same registration. Keys work the same way: they resolve for each search, so a stored or rotated key applies without a restart.
• Separate from chat. The search endpoint is deliberately independent of the endpoint your chat models use; changing one does not move the other. The endpoint can also come from the environment Oda starts in.
• Per-target settings. A search target that runs a model call for each search carries settings for its base address and model, its API version, a token limit and a search-use limit (4,096 and 5 by default). Targets can also carry a result count and, on some, a recency filter. Limit fields take whole numbers of at least 1.

How Oda chooses a provider

• One provider per capability. Search and page reading each resolve one provider from the registered ones. Two ship with the layer: the platform provider (id datamind), which searches and reads pages through your company's DataMind server, and the anonymous fetcher (id http), which reads public pages on this computer without credentials. A deployment can register others.
• A pin wins. A provider pinned in the provider settings (one for search, one for page reading) is used whenever it is registered and available.
• With no pin, exactly one usable provider must be registered. None usable, or several, fails the call with a selection error; the order in which providers were registered never changes the outcome.
• Availability is a local check. Asking whether a provider is usable makes no network call; whether it truly works appears when a call runs.
• Tools stay put. The web tools stay visible when their provider is missing or misconfigured; the failure arrives with the call, not in the tool list.

The selection errors read:

Keys and signing in

• Where a key can come from. A non-empty key set directly in the target's configuration wins. Otherwise Oda looks in its stored credentials, where a key saved on Settings > Models lives; see Set up models and providers and Stored credentials. Last, a key can come from the environment Oda starts in. The missing-key message points at these three places and never prints a key.
• Keyless fixes need no restart. A key resolves for each search, so a key you store or replace applies to the next one. A value set in the launch environment is read when Oda starts.
• Signed out. While you are signed out, web search and page reading on the platform path fail with a key error before any network request is made.
• Signed in. The platform path follows your session: Oda refreshes an expired session once and retries, and when the server needs a newer app than you run it raises the Update DataMind Oda notice.
• No redirects on credentialed requests. A request that carries your credentials never follows an HTTP redirect; the redirect's target is never contacted.
• Proxies are honoured. Web search and page reads respect the proxy settings of the environment Oda starts in.

What a search and a page read return

• Sources. A search returns the sources it found: an address and a title for each, with a snippet and a date when the target supplies them. Entries without an address are skipped, and duplicate addresses are kept once, first one wins. Search results never carry page content; open the page to read it.
• A written answer, on some targets. Depending on the target, a search can also carry a short written answer beside its sources. One kind of target runs a full model call for every search, which spends time and tokens; others answer from their own index.
• Request shape. A search asks the target with the query; a page read hands the target a single address. A result count sent along is only a cost and latency optimisation, never something the model controls.
• Page reads. A read returns the page's readable text, its address, and the status the page answered with. Content is cut at the target's character cap (100,000 characters by default) and marked as truncated when it is cut.
• A 404 can be a success. A page that answers 404 to the platform was still read, and the result reports status 404. A 404 from the platform itself is a provider failure.

Fix a failed search

1. Read the message. The platform path reports its failures with the texts below; the selection errors above cover provider problems.
2. Open Settings > Plugins > Plugin configuration > Web search and check the Endpoint field; only you should choose or change the endpoint.
3. If the failure is about a key, store it (the Models settings page writes keys into Oda's credentials), set it in the environment Oda starts in, or set it directly in the target's configuration.
4. The change applies to the next search; no restart is needed. If the failure repeats, keep the message and contact your administrator; the section's collected messages are on Troubleshooting.

Questions, answered

How do I change the web search endpoint?

Open Settings > Plugins > Plugin configuration > Web search and edit the Endpoint field. Only you should choose or change it. The change serves the next search, with no restart; a failed search points you to the same place.

Do I need to restart Oda after changing the endpoint or a key?

No. The endpoint is read afresh for each search under the same registration, and keys resolve for each search, so a stored or replaced key applies to the next one. Keys and endpoints that come from the environment Oda starts in are the one exception: that environment is read when Oda starts.

Why does web search fail with a key error?

No usable key reached the provider. If you are signed out, the platform path fails this way before making any network request. Otherwise store the key (see Set up models and providers), set it in the environment Oda starts in, or set it directly in the target's configuration. The next search can then succeed with no restart. The message points at these places and never prints the key.

Which provider does Oda use when several are registered?

A pinned provider wins whenever it is registered and available. With nothing pinned, exactly one usable provider must be registered for the capability; if several are usable, the call fails and asks you to pin one. The order in which providers registered never changes the outcome.

Why does a search fail saying several web providers are registered?

Nothing is pinned and more than one usable provider is registered for search, so Oda cannot choose. Pin one in the provider settings. The message reads multiple usable web providers are registered (<ids>); configure one explicitly.

Why is the web tool still available when no provider is configured?

The tools follow the product's configuration, not provider availability. A missing or misconfigured provider does not remove the tool; the failure appears when a search or a read runs, with one of the messages above.

Is the web search endpoint the same as my chat model's endpoint?

No. The search endpoint is deliberately independent of the endpoint your chat models use. An error about web search therefore points at its own settings path, and changing one endpoint does not move the other.

Why did a page read succeed with status 404?

Because the page itself answered 404. The read completed and reports the page's status, including 404. Only a 404 from the platform itself is a provider failure.

Where do I store the key for web search?

In Oda's stored credentials: save it on Settings > Models and it serves web search too. You can also set it in the environment Oda starts in, or directly in the target's configuration, where a non-empty value wins. Oda never shows a stored key back.

Does web search work while I am signed out?

On the platform path, no: it fails with a key error before any network request. Sign in, or configure a key for a search target that accepts one. Page reading through the anonymous fetcher carries no credentials and does not depend on your DataMind sign-in.

What does each provider return, and what does it cost?

The platform provider returns sources with titles, addresses, snippets and dates, and its page reads return the page's text; its work runs on your company's server. The anonymous fetcher returns page text only, read on this computer, and sends no credentials (Direct web fetching covers its rules). A deployment can also mount other search targets; one kind runs a full model call for each search, which costs time and tokens, while an index-backed target answers directly.