Build an ANY API connector

An ANY API connector reads an API that no ready-made source covers: you describe the API as a recipe, and DataMind runs it. DataMind's AI, Kartlos, builds that recipe with you, in the panel beside the connector's recipe editor, from the first draft to a confirmed recipe whose streams return real rows. This page covers how the assistant works; the connector screens are on Connect any API and The recipe editor.

The assistant beside the editor

• On a new connector, the empty recipe offers Describe your API to the AI to start a build. The panel itself, and how to open it, are on The AI assistant.
• It works from the same three things you see in the editor: manifest.yaml, the recipe file; config, the connection's stored settings; and streams, the data sets the recipe can return.
• Before it changes anything, it reads the connector: the saved recipe, the stored settings and the streams already discovered.
• Its edits land in manifest.yaml as a new version. If they meet unsaved edits of yours, the editor merges them or asks you to choose which version to keep; see The recipe editor.

How a build runs

1. It reads the connector first, so it knows what already exists.
2. If your request leaves open points, it asks once: which API, which kind of sign-in, which data sets, and what each stream syncs on. It skips the questions when the task is explicit, and it never asks for a secret value. You can attach screenshots of API documentation or responses, and it reads them.
3. It researches the API and probes it with real requests. When the documentation and the real response disagree, the response wins.
4. It drafts the recipe and writes it into the editor. On an unfamiliar API it starts with one stream and grows the recipe as the picture clears.
5. It tells you which secrets to enter under Connection, and continues once each shows as set.
6. It validates the recipe and reads every issue back: a fatal issue blocks the save, an error blocks confirmation. A host the recipe calls stays an issue until you approve it; see Hosts, streams and data access.
7. It checks the connection the way Test Connection does: the recipe's check runs with the stored secrets, and nothing is loaded.
8. It samples every stream the recipe declares, and one discovered instance of each template, so you see real rows and columns.
9. It confirms the recipe and reports in a short list: the streams, the sign-in kind, the date column each stream syncs on, and what you do next.

Checking, saving and syncing

• Test Connection runs the recipe's check with your stored settings and loads nothing. It reads Connection test successful or Connection failed.
• Save keeps your changes; Save and sync saves, validates the recipe, discovers the streams and infers their columns. The form states it: "Save keeps your changes. Save and sync also validates the recipe and discovers streams and columns. No rows are loaded until you load a table."
• In the Lake tree, the same sync appears as Sync Schema, and a run in progress can be stopped with Stop Sync.
• The recipe is Confirmed while the saved file matches its last validated version. Any change you save leaves it Unconfirmed until it is validated again, and only a confirmed recipe can discover or sync streams.
• A sampled stream still reads Not loaded yet until its table is loaded: "Data lands only when this table is loaded - manually from Lake or by a workflow." The assistant samples and edits; it never loads a table itself.
• Every change made through the assistant is saved as a version. In the editor's History you can Roll back to an earlier version or Restore a rolled back one; see Roll back and restore connector versions.

What the assistant will not do

• It edits the recipe only. Recipes are declarative, and executable components are refused: no custom code enters a recipe.
• Its requests only read. It never sends requests that change data in your API.
• If your API needs custom code, it says so and points you to a Custom connector: Connect Anything (custom connector).
• It never invents a secret, approves a host or grants access to table rows: those stay human decisions. See Connector hosts, secrets and data access.

Limits it works to

• One check or sample runs at a time on a connector; a second waits for the first.
• The same error three times makes it stop and ask you, and five failed validation passes in a row do the same. A change that was refused is never retried unchanged.
• A stream that returns no rows without an error is a complete result: it reports 0 rows and asks once whether that is expected before widening the read.
• Requests time out between 5 and 900 seconds, and at most 16 run at once. Parallel page fetching is used only for offset or page-number APIs with a stable sort.
• Text that is not valid UTF-8 is replaced rather than failing the run, and the run notes it in the log.
• Its reports use five words with one meaning each: done, in progress, waiting for you, blocked and failed.

Questions, answered

How do I start a build with the assistant?

Add an ANY API connector in Lake and open its recipe editor. While the recipe is empty, Describe your API to the AI starts a build; you can also ask the assistant to change a recipe that already exists. The panel itself is The AI assistant.

What does the assistant do before it writes the recipe?

It reads the connector first: the saved recipe, the settings and the streams. Then it researches your API and probes it with real requests. When the documentation and the real response disagree, the response wins.

Which sign-in methods can a recipe use?

The usual kinds: an API key in a header, a bearer token, basic auth, OAuth with a client credential or a pasted refresh token, a JSON web token, or a username-and-password login that returns a session token. If your API needs bespoke signing, the assistant says so and points you to a Custom connector.

When is a build finished?

When the recipe is Confirmed and every stream it declares has been sampled with real rows, or with a 0-row result you accepted. Sampling proves the shape of the data; loading a table stays a separate step, and the assistant never claims rows landed.

What if a stream returns no rows?

It is not a failure. A stream that returns no rows without an error is a complete result; the assistant reports 0 rows and asks once whether that is expected, and only widens the read when you say so.

What happens if the same error keeps coming back?

It stops and asks. After the same error three times, or after five failed validation passes in a row, the assistant stops working the problem and waits for your decision; a refused change is never retried unchanged.

Can a recipe carry custom code?

No. Recipes are declarative, and executable components are refused. If your API needs custom code, use a Custom connector instead.

How many requests can the connector run at once?

At most 16, with a request timeout between 5 and 900 seconds. Parallel page fetching is allowed only for offset or page-number APIs with a stable sort.