How Dimension and Fact runs write data

Transformations load Dimension and Fact tables. This page follows what one run writes: where its result appears, what each pattern changes, and what the loaded rows carry. To build and test a transformation, see Build, test and finalize a transformation; to run one and watch it, see Deploy, run, truncate and monitor tables.

Follow a run from its results

When a test run finishes, click Show Results (see Build, test and finalize a transformation for the full test and finalize flow). The results open on Source Table, the rows your query read. Click Next: Target Output to review Data Changes, what the run would write to the table.

A production run reports on the transformation's Execution tab. The Run Log lists the run and its result, and when a run stops, the message it failed with is what you read there.

How a run moves data

A run does not write straight into your table. It works in stages:

  1. Source copy. The query result lands in a staging copy. This is what Source Table shows.
  2. Marked copy. For a Dimension, the platform compares the result with the rows already in the table for that source system, and marks each row with a DML Type in a second staging copy. This is what Target Output shows.
  3. Final write. A production Dimension run inserts the marked rows into the Dimension table. A production Fact run does not insert row by row: it replaces the Fact data for the days it loaded.

A test run stops after the staging copies. It writes nothing to the Dimension or Fact table, and no PII tokenisation runs.

What the DML Type means

The run marks each row as it compares the result with the table. The DML Type column carries the mark and stays empty until the pattern fills it in:

DML TypeWhat it means
InsertThe row is in the result and not yet in the table for this source system.
UpdateThe row is already in the table and one of its mapped values changed.
DeleteThe row is in the table and no longer in the result.
UnchangedThe row did not change. It is left alone and not written.

What each pattern writes

For a Dimension, the Pattern decides which rows reach the table:

PatternWhat the run writes
Full ReloadEvery changed row: inserts, updates and deletes.
Diff UpsertInserts and updates.
Diff InsertInserts only.
Diff UpdateUpdates only.
Diff DeleteDeletes only.
Note

A Dimension never loses rows physically. A delete sets the Delete Flag column to true on the row instead of removing it, as Transformation patterns and load settings notes.

Facts write in snapshot-day windows. A periodic Fact run replaces the data of the days it covers, and Diff Insert (Periodic) adds only rows that are not loaded for its source system yet.

Check for duplicate keys

Before writing, a run checks its staging copy for duplicate keys. When the check fails, the run stops before writing to your table.

Fix the problem at its source: change the query or the mapping so each key comes back once.

What a loaded row carries

Target rows are keyed by Source ID plus Source System ID. A Fact also keys its rows by snapshot day. Because the key includes the source system, the same Source ID loaded by two source systems stays two separate rows.

The row also carries the columns the platform maintains while loading:

A foreign-key column points at another table's row, and its value stays stable across runs and source systems.

When the first run looks for the earliest date

When a periodic Fact run loads full history into an empty partition and no Full Reload Start Date is set, it searches the source for the earliest date to start from. When the search cannot find what it needs, the run stops and the message names the missing piece:

Set Full Reload Start Date so the first run does not depend on the search (Transformation patterns and load settings), or make sure the transformation's Mapping step maps the Fact's snapshot column (Build, test and finalize a transformation).

The history step

A Dimension with Store History has an extra history step in its workflow: each history run writes a new version for records that changed since the last run. See Dimension history for the versions and their windows.

Questions, answered

What does a test run write?

Nothing in your table. A test run fills the staging copies and stops: it writes no rows to the Dimension or Fact table and runs no PII tokenisation. Use it to review Source Table and Target Output before you finalize.

Where do I see what a production run changed?

On the transformation's Execution tab, where the Run Log follows the run and its result. The table's Settings tab lists the run under Transformation Runs. Test runs are not listed there.

Why does my run say "Output Contains Duplicates in Source ID"?

The staging copy holds more than one row for the same Source ID, so the run cannot tell which row is the right one and stops before writing. Change the query or the mapping so each key comes back once.

Does a Fact check duplicates the same way?

Yes, per snapshot day and Source ID. Its message is Output Contains Duplicates in Source ID Per Partition. Find the duplicated key inside the day it names and fix the query or the mapping.

What does the Delete Flag column mean?

It marks a row whose record no longer exists in the source. A Dimension run that deletes sets it to true and keeps the row, so nothing is removed physically.

What are Insert Process ID and Update Process IDs?

They name the runs that wrote the row. Insert Process ID names the run that first inserted it, and Update Process IDs lists every run that has updated it, growing with each update.

Which key does a target row use?

Source ID plus Source System ID. A Fact also keys by its snapshot day. The same Source ID loaded by two source systems stays two separate rows.

Do foreign keys stay the same between runs?

Yes. A foreign-key column builds the key from the source value, so the reference stays stable across runs and source systems.

Why did the first run fail to find the earliest source date?

The search did not find what it needed. The message names the missing piece: No Snapshot column found for transformation ... or No mapping found for Snapshot column in transformation .... Set Full Reload Start Date so the first run does not depend on the search, or map the Fact's snapshot column on the Mapping step.

Does a load also update the history table?

Not by itself. A Dimension with Store History gets its history from a separate history step in its workflow, which writes new versions for records that changed. See Dimension history.