The Warehouse Builder

The Warehouse Builder is a Mind that builds and maintains Warehouse objects for you: facts, dimensions, static tables, dictionaries and views, together with the transformations that load them. It does the work a Warehouse administrator does by hand, in the same order and with the same checks, and it stops for your decision wherever a step needs one.

What it can build

• Tables and views. Facts, dimensions, static tables and dictionaries, and views, including a refreshing view that keeps itself up to date on an interval.
• Transformations. One per source system for each fact or dimension, with the query and the column mapping that fill the table.
• The structure around them. A semantic source, its source systems and a logical model, when the objects you asked for do not have one yet. See Semantic sources, source systems and logical models.
• A DataPrism semantic model, designed with you in the chat before anything is created. That flow is on Design a model with the Warehouse Builder.
• The source data it reads must already be in the Lake or built in the Warehouse.

Its answers can also carry KPI cards, charts, diagrams and links to the objects it touched, covered on Links, KPIs and diagrams in the Warehouse Builder.

Before it builds, it asks once

• It reads a little first, so its questions are concrete, and then states the task back on one confirmation card with clickable choices. You can also type your own answer.
• The choices depend on what it is confirming. For a single object: Correct, build it, Different grain or Different source. For a plan with several objects: Build as planned, Change something first or Dimensions only for now.
• It asks what it cannot work out safely: which semantic source and source system to build from, whether a table is a fact or a dimension, the business date, the key that makes a row unique, and the load pattern. A wrong grain is not repaired by editing a query later; it means rebuilding the table.
• It never asks about parallel sub-agents. Whether the build runs as one agent or as several working side by side is its own decision, taken from the task.
• In a scheduled run, where no one can answer, it states the assumptions it works from and carries on. It never removes anything on an assumption.

The build order

The order is fixed, and nothing moves on until the step before it is checked:

1. Plan. For work with several steps it publishes a plan and keeps it updated while it builds. The plan strip is described on How a Mind answers.
2. Prerequisites. It creates or reuses the semantic source, the source systems and the logical model the objects belong to. Dimensions are built before the facts that point at them; views, dictionaries and static tables come after.
3. The table. It creates the table, saves its columns and keys, and deploys it. A table must be deployed before anything can be tested against it.
4. The transformation. It writes the query, validates it, saves it, and maps every target column. See Build, test and finalize a transformation for what each of those steps does in the screens.
5. The test. It runs a test over a small window, waits for it, reviews the output and finalises only a test it reviewed and that finished.
6. The full load. It restores the window, backfills the history and checks the table's health.
7. The hand-over. It reports what was built and leaves the schedules for you.

How the work is proven

• Every number in its report comes from something it read back. It does not estimate totals and it does not present arithmetic of its own.
• The test runs on a few days, not the whole history, so mistakes surface quickly. After the run finishes it reads the aggregated output, not the rows, and looks for: rows that arrived, no column you defined that arrived entirely empty, an identifier that is unique where it must be, and no date that fell back to January 1970 because it could not be read.
• A key column that holds a single value means every row missed its join. That is a defect to fix, not a result to keep.
• A fill that repeats an identifier stops with a duplicates message instead of loading the rows. Output Contains Duplicates in Source ID means the identifier is not unique across the result; Output Contains Duplicates in Source ID Per Partition means the same day and identifier appear twice in one part of a fact. Both are grain problems, and the builder reworks the query rather than loading the rows.
• A test that failed, or that was never reviewed, is never finalised. When the full load finishes, the health check reads row counts, each column's contents and the newest data, and the report says what was found, including zero rows as zero rows.

Load patterns and history

• A transformation's pattern is chosen when it is created and locks then. A different pattern means a new transformation, not an edit of the old one. The patterns themselves are on Transformation patterns and load settings.
• For a fact, the everyday choice is the periodic reload: each run reloads a window of days, so corrections inside the window are picked up and older ones wait for a wider reload. It starts Synchronize Days at 60 and asks when it cannot judge how far back your data corrects itself.
• A one-off load can be a full reload, data that only grows can use a diff insert, and a window too heavy for one run can be walked in a loop.
• Facts are partitioned by day. A monthly fact keeps that shape and stores the month's start as its date.
• A dimension starts from a full reload, so the table matches the source. Store History stays off unless you ask for it; with it on, every version is kept in the table's separate History table.
• A refreshing view gets a refresh interval, set no tighter than the rhythm of the sources it reads.

Runs, failures and retries

• When loads belong to a Warehouse workflow, the builder can start a run, follow it to the end and work on what failed. It reads the failure in full first and changes one thing at a time, then tests and runs again.
• A restart continues the same run: steps that already finished stay finished. A run that finished is never restarted; a fresh attempt is a new run. See Monitor and fix a run.
• When the same error survives three attempts, it stops and comes back to you with what it found.
• It never retries a change you rejected, and a step blocked while a workflow run holds the same objects waits for that run to end. A step that breaks a Warehouse rule is refused before anything is saved.
• Workflow steps use your permissions. Reading or starting a run needs View Warehouse Workflow Objects or Execute Warehouse Workflow Objects; without them the step is refused and names what it needs, and an administrator can grant it. The full list is on Workflow permissions.

What is left for you

• The schedules. Every transformation and workflow it builds arrives with its schedule off. The builder names what to switch on, once, and a person does it in the Warehouse or the Workflows screens; the builder never enables a schedule itself. See Schedules and when a workflow starts and Warehouse Builder and Lake Builder.
• Removals. A step that removes rows or objects waits for your approval, asked for on its own after the builder reads what would be lost and what depends on the object. See Change approval in a builder chat.
• The report. It closes with a short report: what was built, the state of each object, what it found and what remains for you.

Questions, answered

How do I start a build with the Warehouse Builder?

Open a chat with the Warehouse Builder Mind and describe what you want built or fixed. It reads a little first, asks its questions in one batch and starts building after your confirmation. How a builder Mind is created is on Warehouse Builder and Lake Builder.

What do I do on the confirmation card it shows first?

It is confirming that it read your request correctly before anything is built. Choose Correct, build it to let it start, or the correction that fits, such as Different grain or Different source; a multi-object plan carries the plan's choices, such as Build as planned, Change something first or Dimensions only for now. Typing your own answer also works.

Which load pattern will it choose for my table?

For a fact, the periodic reload with a 60-day window, unless your data or your answers point elsewhere; a dimension starts from a full reload. One-off loads can be full reloads, growing data can use a diff insert, and heavy windows can run in a loop. The pattern locks when the transformation is created, so say at the start if you need something specific. See Transformation patterns and load settings.

Why does it ask about the grain, the key and the date?

Because it cannot work those out safely, and they decide the table's shape and its loads. A wrong grain is not fixed by editing the query later; it means rebuilding the table. So it asks in one batch before it builds instead of guessing.

Why did it test with only a few days of data?

To keep the test fast and safe. It shrinks the load window to a few days and tests against a date that has source rows. The full history is loaded afterwards, when the window is restored and the backfill runs; the test is not meant to cover it.

Why did the backfill cover only a few days?

A window that was not restored first becomes the backfill's window. The builder restores the window before backfilling, so a short backfill means the window still pointed at the test's few days; ask it to set the window back to the depth you want and backfill again.

Why does a fill report "Output Contains Duplicates in Source ID"?

The fill wrote the same identifier twice. For a dimension, Output Contains Duplicates in Source ID means the identifier is not unique across the result; for a fact, Output Contains Duplicates in Source ID Per Partition means the same day and identifier appear twice in one part of the table. The builder treats both as grain problems and reworks the query before loading anything. If the repetition is genuinely in your data, tell it, because the key or the date needs a different definition.

Why did it stop after the same error three times?

That is its retry limit. When the same error survives three attempts, it stops and reports what it found instead of looping. Read what it says and answer with what should change, and it continues from there.

Can it design a DataPrism semantic model as well?

Yes. A semantic model can be designed in a Warehouse Builder chat: one batch of questions, one design card to approve, and the model is created with your choices. Its objects and measures are then filled by the model's own AI. See Design a model with the Warehouse Builder.

What is left for me to do after a build?

Two things. Switch on the schedules you want, in the Warehouse or the Workflows screens, because the builder never switches one on itself. And read its closing report for what was built, what it found and anything still open. Nothing else waits on you unless it asked a question.