Warehouse

The Warehouse holds the modelled tables of DataMind OS: dimensions, facts, dictionaries, static tables and views. Read this page to choose the right kind of object for a job and to see how data moves from the Lake into a loaded table.

Find the Warehouse

  1. Click Warehouse in the sidebar. It sits in the data group with Lake, WriteBack and Lineage.
  2. Browse objects in the tree on the left. The header on the right has four views: Dashboard, Relationship Diagram, Workflow Diagram and Add New. The Warehouse workspace tour explains the screen, the Warehouse Dashboard explains the card view, and Saved views and configurations shows how to build your own view of the tree.
Note

The Warehouse item appears with View Warehouse Objects or Add/Modify Warehouse Objects, but the tree loads only with View Warehouse Objects. Warehouse permissions and row-level security lists what each permission allows.

Choose the right object

Click Add New in the header to see every object type with a description and a Use when note. Create Warehouse objects lists every entry point, including the New menu and the right-click menus.

ObjectUse it when
Semantic sourceYou onboard a new data source, such as a database, an Excel file or an API, and want the rest of the platform to understand it contextually.
Logical modelYou need the official business representation of an entity, such as what a customer means across the organisation, before you build transformations or reports.
TransformationRaw Lake data needs cleaning, joining or enriching before analysts or the data mart can use it.
DimensionYou are modelling a star or snowflake schema and need a context table that facts join to.
FactYou store a business event, such as a sale, a login or a click, that you slice by dimensions for analysis.
ViewYou want to expose a shaped version of Warehouse data to analysts or BI without creating a physical copy.
DictionaryYou need a lookup list that SQL queries on Lake tables keep up to date.
StaticYou keep reference data that no live source provides, such as country codes or manual corrections.
Note

These are the Warehouse's own table kinds. Lake and WriteBack objects live in their own areas: imported source tables are in Lake, and WriteBack sends data to other systems.

How data reaches the Warehouse

Lake tables are the input for most objects.

Build your first table

  1. Click Add New in the header, or New in the tree toolbar, and create a Semantic source. Open it and click Add system to add a source system.
  2. Create a Logical model for the business entity your table describes.
  3. Create a Dimension or Fact table. A Dimension or Fact needs a name, at least one logical model and at least one source system.
  4. Click Deploy to create the table in the Warehouse from its current definition. Until you deploy it, the table shows Pending.
  5. Add a transformation for the table. You cannot test it until the target table is deployed.
  6. Run a test. When it finishes, review Source Table and Target Output; reviewing both is required before you finalize.
  7. Confirm Finalize. The transformation can now run, and Execute Now appears on its Execution tab. Keep Schedule on so the Warehouse workflow runs it.
Warning

The Finalize Transformation dialog says that finalizing cannot be undone. Check the query and the test results before you confirm.

For a guided run-through, follow Model your data in the Warehouse. For the full setup of sources, systems and models, see Semantic sources, source systems and logical models.

After the first load, you can speed up queries with table indexes, sync a table with the database after a definition change, and delete objects you no longer need.

Work in popups

Each object opens in a popup window with Refresh, Minimize, Maximize or Restore, and Close controls, and a Copy title button.

For dragging and resizing, the popup bar and its filters, and the audit details on a popup, see Work with popups.

Permissions

Warehouse actions use four permissions in the Data Layer category. Administrators set them on a role in Roles and permissions.

PermissionWhat it lets you do
View Warehouse ObjectsLoad the tree and read Warehouse objects.
Add/Modify Warehouse ObjectsCreate and change objects, and deploy tables.
Execute/Abort Warehouse ObjectsTest, run and abort transformations.
Remove Warehouse ObjectsDelete objects.

If an action is refused, you see a message that starts with "Access denied". Each table also has permission groups that decide who can read it, and optional row-level security that limits the rows each group sees. The presets and the details are on Warehouse permissions and row-level security.

Limits and edge cases

Questions, answered

Which Warehouse table types can I build from Lake data?

Dimensions, facts, dictionaries and views read tables that are already loaded in the Lake, so load those first. Static tables are the exception: you fill them by hand, from an Excel file or with AI.

Which table types can a transformation load?

Only Dimension and Fact tables. The table picker in a transformation lists no other types.

Do I need a semantic source before I can create a Dimension?

Yes. A Dimension or Fact needs at least one source system, and you add source systems to a Semantic source. Create the semantic source first.

When should I use a View instead of a table?

Use a View when you want a shaped version of Warehouse data without a physical copy. Standard and Parametrized views store no data. Refreshing and Appending views store data and refresh on a schedule.

Where do I keep a list that no source provides, such as country codes?

In a Static table. Type the rows, upload an Excel file (which replaces all data), or ask AI to fill or add rows.

How does a transformation run automatically?

Finalize it and keep Schedule on. The Warehouse workflow then runs it on its schedule. See Schedules and when a workflow starts.

Can I see what uses a Warehouse table before I change it?

Yes. Open the table and choose the Dependants tab. It lists the objects that use the table.

Why can't I open another Warehouse popup?

You already have 20 Warehouse popups open or minimised. Close some first, and remember that minimised popups count too. Open Popups in the sidebar helps you find them. Popups in other areas, such as Lake, have their own limit of 20.

Can AI build Warehouse tables for me?

Yes. The Warehouse Builder Mind creates tables and transformations for you. See Warehouse Builder and Lake Builder.