Sync a table with the database

Deploying a table creates it in the database from its definition. After that, DataMind keeps the two in step: when you save a change to a deployed table, a sync compares the definition with the stored table and applies the difference. This page describes what a sync changes, the order it works in, when it drops and recreates a table instead, and where to read what it did. For the day-to-day actions, see Deploy, run, truncate and monitor tables.

What a sync changes

A sync works through the parts of a table in a fixed order and changes only what no longer matches the definition. Each part is skipped when nothing changed for it.

  1. The history table of a Dimension. When a Dimension stores history, its History table is built before any column change, because the column changes are mirrored onto it. See Dimension history.
  2. Columns. Added, changed and removed columns are applied to the stored table. Renames are applied after the other column changes. See Define table columns.
  3. Dictionary structures. For a Dictionary, the structures it reads through are rebuilt after its columns change. See Dictionaries.
  4. Indexes. Indexes to drop are removed before new ones are added. See Table indexes.
  5. Access and personal data. The views behind the table's permission groups are rebuilt, the protected values kept for personal-data columns are filled in or dropped, the views that keep those columns out of general reads are rebuilt, and Mind access to those columns is reconciled. Access is taken away before it is granted, and only the columns that changed are touched. See Warehouse permissions and row-level security.
  6. Refresh settings. Refresh settings are updated for refreshing and appending views and for dictionaries. See Views.

When a table is dropped and recreated

Most changes are applied to the stored table in place. Some are not: when a change alters how the stored table must be built, DataMind drops it and creates it again from the definition. The run records why it did so, and you can read the reasons on the run in the DDL log:

These reasons are recorded by the sync engine, which recreates Lake tables this way. Changes that lead to a recreation are confirmed before they run; the Lake dialogs are described on Deploy a table. Turning Store History off on a deployed Dimension has its own confirmation and also recreates the table; see Dimension and fact general settings.

A recreation is destructive: the recreated table starts empty, so it holds no data until its next load. On a Warehouse table, a primary key change while the table is deployed is refused instead; drop the table first to change the key, as Define table columns describes.

What a sync leaves alone

Where to read what a sync did

Every sync is recorded. Open Observability, open the ClickHouse tab and click DDL log, then find the run for your table: its Kind is sync, and the User column shows who the run is recorded for. Open the run to read the statements it executed, its Schema changes, and, when the table was recreated, its Recreation reasons. A failed sync records its error as well. Table DDL logs describes the log in full; reading it needs View Table DDLs.

Limits

Questions, answered

Do I run a sync myself?

No. Deploying creates the stored table, and a change you save to a deployed table runs a sync straight away. There is nothing to start by hand.

Which changes drop and recreate a table?

A change that alters how the stored table must be built: moving a table to a load pattern in a different family, changing its key columns, or changing its partition column. Turning Store History off on a deployed Dimension also recreates the table. Each of these asks for confirmation before it runs; the dialogs are described on Deploy a table and Dimension and fact general settings.

Does a sync change the rows in my table?

No. A sync changes the table's structure, not its rows, and it never loads or refreshes data. Two details: removing a column removes its stored values, and a recreated table starts empty. When a column's personal-data setting changes, the protected values kept for it are updated too.

My change is not in the stored table yet. What happened?

Check that the table is deployed first: a sync runs for deployed tables only, so a table that still shows Pending needs its deploy (Deploy, run, truncate and monitor tables). On a deployed table, the sync runs with the save itself, so once the save succeeds the change has reached the stored table.

Can I see the SQL a sync ran?

Yes. Open the run in the Table DDL logs: its Statements list holds the SQL that ran, in order, and Copy SQL copies a statement's text.

What is the difference between Create and Sync in the DDL log?

create is the first build of a table, which happens when you deploy it. sync keeps an already-deployed table in step with its definition after a change.

Do dictionaries sync like tables?

Yes. A Dictionary is kept in step the same way: its structures are rebuilt when its columns change, and its refresh settings are updated in the same sync. See Dictionaries.

Is a sync the same as a run?

No. A sync keeps the stored table's structure in step with its definition. A run loads and refreshes the table's rows, and appears under Transformation Runs and in the Run Log; see Deploy, run, truncate and monitor tables.