Staging tables: conditions, sync and deploy

A staging table is a Lake table that holds the data loaded from a source. This page is the reference for the sync side of a staging table's popup: the filter that shapes each load, the actions that run, stop, clear and deploy the table, and the settings behind them. To set a table up, see Configure a table. For Warehouse tables, deploy, run and truncate tasks are on Deploy, run, truncate and monitor tables.

Filter the rows a sync loads

The filter lives on Table > Conditions. Its editor is labelled WHERE clause for sync, with the help text "Standard SQL applied as a WHERE clause during synchronization." Each synchronization loads only the rows that match the clause.

The step shows two examples:

Save the step to store the clause. A Conditions saved successfully toast confirms the save. If it fails, the toast title is Error and it shows the reason from the product, or Failed to save conditions when there is none.

Synchronize and abort

Synchronize starts one load for the date you pick. Its progress and result appear in the Sync Log, which refreshes on its own while a run is in progress. Abort stops a synchronization that is still running, from its row in the log.

The full flow, including the date picker and each log column, is on Run, stop and troubleshoot loads.

Truncate and delete the table

Truncating clears the table's data; the table itself stays and can load again.

Choose what to clear with the target: main clears the table's current data, history clears its stored history, and all clears both. The default target is main.

On a partitioned table you also pick the partitions to clear. A partition value must match the value in the partitions list exactly; a value that does not match is refused. If the partitions list is empty, the truncate clears the whole table. On a table that does not support partitions, the truncate still runs and clears the whole table, and the product shows the reason.

When a partition's row count is not known, the confirmation hides the row total instead of showing one that understates the removal. Truncation cannot be undone, so read the summary before you confirm.

To remove the table from the Lake, delete it; DataMind checks what depends on the table first.

Deploy the table

Deploying creates the table with its current columns. A staging table must be deployed before it can synchronize or truncate, and a deploy needs at least one primary key. For the deploy flow and its banners, see Configure a table.

The deploy result reports each table it covers with Success, Failed or Partially Success. A deploy can finish successfully overall while some of the tables fail, and a failed table carries its error message.

Settings that shape a load

On Table > General you also see the table's fill type (ETL, Manual or AI), its logical model, permission group, source systems and retention frequency. The permission group choices follow the table's roles.

A few settings change how much a load reads and what the product keeps for you:

Questions, answered

What does the WHERE clause for sync do?

Each synchronization loads only the rows that match it. The clause is standard SQL, applied as a WHERE clause during synchronization, for example status = 'active' or status = 'active' AND created_at > '2024-01-01'.

Why did my conditions save fail?

The Error toast shows the reason from the product, or Failed to save conditions when there is none. Check the clause, correct it and save again.

Can I stop a synchronization that is already running?

Yes. Use Abort on the run's row in the Sync Log while the run is in progress.

Does the Sync Log update while a run is going?

Yes. While a run is in progress the log refreshes on its own, so you do not need to refresh it yourself.

What is the difference between main, history and all?

main clears the table's current data, history clears its stored history, and all clears both. The default target is main.

Why is the row total missing from the truncate confirmation?

Because at least one partition has no known row count. The product hides the total rather than showing one that understates how much the truncate removes.

What happens when I truncate a table that does not support partitions?

The truncate still runs and clears the whole table, and the product shows the reason.

What do the deploy statuses mean?

Success means the deploy went through. Partially Success means the deploy finished, but some of the tables it covered failed; each failed table shows its error message. Failed means the table did not deploy.

Are earlier versions of my query settings kept?

Yes. Every save keeps a revision, and each revision records the query settings, whether an order by was applied, whether rows were made distinct, and who changed it.

Do the refresh offset and its unit go together?

Yes. They are a pair: set both, or clear both.