Hosts, streams and data access

These panels control what a Connect Any API connector may call and which data it may read: the hosts its recipe can reach, the streams discovery finds, and the data access grants that let the recipe read rows from a Lake or Warehouse object. You meet them in the connector wizard and in the recipe editor. For the connector end to end, see Connect any API; for the recipe itself, see The recipe editor.

Approve the hosts a recipe can call

A recipe can call only the hosts you approve. You approve them under Hosts, on the Connection step of the wizard and on the config node of the recipe editor.

  1. Find each host in the list. It carries Approved, Pending or manual; the assistant adds the API's host when it writes the recipe.
  2. Click Approve on a pending host, or Approve all when two or more are waiting.
  3. Click Revoke to stop a host being called.
  4. To approve a host that is not listed yet, type it under Add host and press Enter. A hostname or IP address works, with an optional port; anything else shows "Enter a hostname or IP, optionally with :port."

While the list is empty it reads "No hosts yet. The assistant adds the API's host when it writes the recipe; you can pre-approve one here."

A pending host keeps the Connection step open with "Approve the pending hosts below - the recipe can only call approved hosts.", and the recipe cannot move on until every host is approved or revoked. Every change confirms with "Host approved" or "Host revoked", or fails with "Failed to approve host" or "Failed to revoke host".

What approving a host does to existing grants

If the connector already has data access grants, approving a host asks first: Also send granted data to these hosts? The dialog explains "This connector has data access grants. Approving the hosts pins every active or broken grant to them as well." and lists each affected grant, for example "Grant {name}: values of {columns} from {object} will also be sent to {hosts}". A broken grant gets its own line: "Grant {name} (broken): once it is restored, values of {columns} from {object} will also be sent to {hosts}".

Click Approve and send to approve the hosts and pin the grants. Revoked grants are not pinned.

Discover and shape streams

Streams are the data sets the recipe can return. Discovery fills the Streams step of the wizard and the streams node of the recipe editor.

  1. Start or refresh discovery. The wizard has Discover streams; the recipe editor's panel has Refresh streams and Open in Lake picker. While discovery runs, the panel shows "Discovering streams from the API…" with Stop, and the step notes "Saving, validating and discovering streams - this can take a minute…".
  2. Read the status line: "Streams have not been discovered yet.", "The streams are out of date with the recipe or its settings." or "Streams discovered {ago} ago." A failed run shows Discovery failed with its reason.
  3. Check the table:
ColumnWhat it shows
StreamThe data set. Type in Filter streams… to narrow the list.
OriginDeclared when the recipe names the stream, Discovered when discovery found it.
ColumnsThe number of columns found; they are inferred on save and sync from a sample.
LakeImported once the stream is a Lake table, or Not imported.
LoadNot loaded yet, Loading, Loaded (with the row count and time) or Failed (with the error).

Before anything is discovered, the panel reads "No streams yet - Save and sync to validate the recipe and discover streams." With no manifest it reads "No streams - the recipe has no manifest yet.", and if discovery finds none while the recipe declares none, it reads "The recipe declares no streams and discovery returned none."

  1. Before importing, click Schema to shape the stream's columns: change Name, Type, Nullable and PK, remove a column, add one, or click Reset to inferred to return to the detected set. Save schema confirms with "Schema saved for {stream}".
  2. Click Import to make the stream a Lake table, confirmed with "{name} imported to Lake". A stream whose cursor is not a datetime cursor imports as Full and shows "Integer cursor - imports as Full (phase 2)"; for what that means for the table's syncs, see Choose a sync type.

Add a data access grant

A grant lets the recipe drive its requests from the rows of one Lake or Warehouse object; the recipe reads them as the stream grant_{name}. Only the admin role sees this section; see Who can do what in the Lake. For the grant's lifecycle and limits, see Data access grants.

  1. In the recipe editor, open the config node and click Add data access.
  2. Enter a Grant name. It starts with a lowercase letter, then lowercase letters, digits or underscores, up to 48 characters, and must be unique for this connector; if it is taken, the form answers "…" is already used.
  3. Choose the Source, Lake or WH, then pick the Object. Search with Search objects…; only objects with a PII admin view are listed, and a search with no match reads "No objects match. Only objects with a PII admin view can be granted."
  4. Choose the Columns with Select all and Clear. A token badge marks each PII column, and "PII columns are sent as tokens, never as raw values."
  5. Optionally set Date column (optional) and Row cap (optional). The date column keeps incremental loads to the rows inside the run window; "Without one, every run reads the whole object." A run that would read more rows than the cap "fails before the first API request".
  6. Click Continue. If something is missing, the form answers now: "Pick an object.", "Pick at least one column." or "Enter a whole number greater than 0."
  7. In Confirm data access, check the Columns and Hosts lists, type the object name under Type the object name to confirm, and click Grant access. The dialog states "Each time the recipe runs, the values of these columns are sent to every host below." If no host is approved yet, it warns "No approved hosts yet. The grant is pinned to the hosts approved later; each new host needs an admin confirmation."

Manage the grants on a connector

The Data access section lists every grant with its state, Active, Broken or Revoked, and a summary such as its row count, date column or cap. While it is empty it reads "No data access yet. A grant lets the recipe drive its requests from the rows of one Lake or WH object. PII columns are sent as tokens."

To stop a grant, click Revoke and confirm Revoke data access "{name}"?: "Loads that still reference the stream grant_{name} fail from the next run. The grant stays in the audit log." The button is hidden on an already revoked grant. Actions confirm with "Data access "{name}" granted" or "Data access revoked", or fail with messages such as "Could not grant the data access" or "Could not revoke the data access".

Questions, answered

Why did approving a host ask about data access grants?

Because the connector has live grants. Approving the hosts pins every active or broken grant to them, so granted columns will also be sent to the new hosts. Review the listed grants and click Approve and send only when that is what you want. See Data access grants.

Are revoked grants pinned when I approve a host?

No. Only Active and Broken grants are listed and pinned; revoked grants are left as they are.

Why is the Data access section missing?

Only users with the admin role see it. Ask your administrator to give you that role; see Who can do what in the Lake.

Why is Grant access greyed out?

You must type the object's exact name under Type the object name to confirm first. The box shows the name as its placeholder, and the input resets every time the dialog opens. Grant access becomes available when the typed name matches exactly.

Can I approve a host before the recipe names one?

Yes. Type it under Add host and press Enter, with an optional port. The recipe can call only approved hosts, so a host approved now is ready when the recipe needs it. If the connector has grants, approving still asks the pinning question first.

Why does my stream say Not loaded yet?

The stream is imported, but its table has never been loaded. Data lands only when the table is loaded, either by hand from the Lake or by a workflow; the badge's tooltip says so. See Run, stop and troubleshoot loads.

Why can't I change an imported stream's columns here?

After a stream is imported, its columns belong to the Lake table. The Schema button is disabled and says "This stream is imported - edit its columns in the Lake table." Open the table from the Imported link and edit its columns there.

The streams are out of date. What should I do?

The recipe or its settings changed since the last discovery. Click Refresh streams in the recipe editor, or Save and sync, which saves, validates and rediscovers. The rows update when discovery finishes; nothing is loaded into the Lake by this.

What does the token badge mean?

The badge marks each PII column of the object. Grants send PII columns as tokens, never as raw values, so the API never receives the original value.

Why is Save schema disabled?

A column name is blank, or a save is already running. Fill every name, or wait for the current save to finish, and the button becomes available.

What happens if the hosts change while I confirm a grant?

The dialog notices. It reads "The approved hosts changed while this dialog was open. Review the new host list below, then confirm again.", re-lists the hosts, and asks you to confirm on the new list.