Full-history backfill and the performance ladder

The Model Builder starts a semantic model on a small sample of your data, then loads the full history and re-measures every chart on it. This page covers that load, the questions it asks about tables that existed before the design, and the performance ladder that makes slow charts faster. To start and follow a build itself, see Model Builder.

The sample load

The first load is a sample:

The full history load

The full history becomes available once the Model AI has finished its first pass. From there:

Shared-table questions

The build changes only the tables the design created or changed. Anything older is touched only with your word, through a question on the session card, headed The Model Builder needs a change on a shared table. The question reads:

The Model Builder wants to action on "object", a table that existed before this design and that other models or loads may use. It changes only the tables this design created or changed. Will you make this change yourself in the Warehouse Builder chat, or should the Model Builder continue without it?

The line under the question explains why: "This table existed before this design, so the Model Builder changes it only with your word."

Answer it in one of three ways:

  1. Choose I will make it in the Warehouse Builder chat to keep the change in your hands. The build skips it.
  2. Choose Continue without it to let the build carry on without the change.
  3. Write your own answer under Answer in your own words.

Once answered, the row shows answered by name, and the build carries on.

Verify rounds and slow charts

The performance ladder

When a chart misses its time budget, the build works down a ladder of fixes, measuring the charts again after every step:

  1. Add or adjust a skip index and materialise it.
  2. Turn a Standard view into a Refreshing view.
  3. Move a view into a loaded table.
  4. Denormalise hot attributes.
  5. Add an aggregate fact.

Open items on the done card

Questions, answered

Do the tests cover the current month?

No. Tests and reconciliation run only on the closed months of the sample. The month to date is loaded, but it is left out of them.

Why is the full history load waiting in line?

One backfill runs at a time for the whole platform, and a sample load holds the same slot. The session card shows your place, for example Waiting for the warehouse: 2nd in line.

When can the full history load start?

When the structure has settled and the Model AI has no open change requests, and after the tables that existed before the design are settled with their questions. Until then, the request waits. The depth it loads to is the one you chose on the semantic model design card.

Can a full history load time out?

No. There is no timeout on a load. If one gets stuck, a person can abort it in the Warehouse UI.

Why did the full history skip some tables?

Because they existed before the design. The build changes only the tables the design created or changed, so it asks you first, and if the answer is to continue without the change, that table keeps the history it has. The card lists it as Kept the history they have: names (they existed before this design).

Can the build speed up a slow chart itself?

Partly. It applies the first 2 ladder steps itself: it adds or adjusts a skip index (up to 6 per session, on tables that already hold rows) and turns a Standard view into a Refreshing view (refreshed at most once a day). Each step is followed by a fresh verify round. The later steps it relays to you, for a person to carry out.

Does a slow chart block the model being ready?

No. The bands are advisory, so a slow chart does not hold the build back. Only a hard failure, a chart over 30 seconds or one that runs out of memory, changes the ending: the build still finishes, and the done card reads Ready, with charts that failed, with that chart listed first.

Where do I follow the full history load?

On the session card's Full backfill section: it shows each table's state (Queued, Loading, Loaded, Failed or Cancelled), its rows and its time. When every load has finished, the Model Builder is woken and checks each table's health. For the card's parts, see Model Builder cards.