Loading the documentation…

ClickHouse storage and maintenance

Use the Storage view of the ClickHouse tab to see how much space your tables take and to empty the platform's staging tables, and use the Merges view to merge fragmented parts that slow queries down. Some actions appear only if your role holds the matching permission.

Read how much space each table uses

  1. Open Observability > ClickHouse > Storage.
  2. Read the table list. It opens sorted by Active Parts, highest first. To find the largest tables, sort by Size On Disk.
  3. To narrow the list, type part of a database or table name in the search box, or turn on Hide system tables to leave out the database's own tables.

Each row shows Database, Table, Engine, Kind, Active Parts, Partitions, Rows, Size On Disk, Compressed, Uncompressed, PK In Memory, Index Granularity In Memory, Oldest Part and Newest Part. Kind shows System for the database's own tables; user tables leave that cell empty. You can filter Kind by System or User.

The table shows 50 rows at a time. You can switch to 100, 500 or 1000 rows. Changing the search or the Hide system tables switch returns the list to its first page. An empty list shows No tables with data parts.

SettingWhat it doesDefault
Hide system tablesHides the database's own tables from the list.Off

The Storage toolbar also holds the Excel export and the Empty staging tables action. The export downloads the current list as an Excel file named clickhouse-table-storage-YYYY-MM-DD.xlsx.

See the temporary ETL tables

Warehouse Fact loads and Lake loads collect their rows in temporary tables and publish them when they end. The Temporary ETL tables panel at the top of Storage shows them. It is read-only and refreshes every 30 seconds; the panel's header carries a Read-only chip, and with nothing to list it reads No temporary ETL tables right now.

The panel has one tile per kind, each with its number of tables, rows, size and the age of the oldest one:

TileWhat it holds
ETL stagingThe temporary tables of loads in progress, and of loads that ended without cleaning up.
Retired tablesPrevious versions of tables, kept for a short time after a load published a new one.
Lake load tablesA Lake table's load table while it loads, or the one a failed load left behind until that table's next load.

Click Show tables (n) to list them, and Hide tables to close the list. The list shows Kind, Table, Token (the internal number of the load that created the table), Holder (the run that owns it, with a link to its job, or No holder), Rows, Size and Age. An ETL staging or Retired tables table older than 6 hours shows its age in red.

The tables are removed for you:

When an ETL staging or Retired tables table is older than 6 hours, a warning reads <n> temporary ETL table(s) are older than 6 hours. The sweeper removes finished ones automatically; an old one usually belongs to a long or stuck load. Check the holder's run in ETL locks or in its workflow.

In the storage list, ETL staging and Retired tables tables carry the badge ETL temporary (read-only), with the tooltip Removed by the ETL sweeper; never edit or drop it by hand. Merge parts into partitions and Empty staging tables leave them alone, and the storage dashboards do not count them.

Lake load tables carry no badge. They are among the tables that Empty staging tables empties, and Merge parts into partitions can merge their parts. The storage dashboards do not count them either.

Merge parts into partitions

Many active parts in one partition can slow queries down. Merge parts into partitions runs OPTIMIZE on the tables whose partitions hold more than one active part. OPTIMIZE merges parts only within a partition, so every partition keeps at least one part. The action sits on the Merges toolbar; ClickHouse activity describes the view itself.

  1. Open Observability > ClickHouse > Merges.
  2. Click Merge parts into partitions.
  3. Read the preview. While it loads, it reads Checking which tables have parts to merge.... It then says either OPTIMIZE runs in the background on N tables and can reduce up to N parts across N partitions. or Every partition already holds a single active part.
  4. Click Start OPTIMIZE.

A Success toast then reads OPTIMIZE started on N tables.

Each OPTIMIZE statement shares its table with other work while it runs; see ETL locks.

Note

Start OPTIMIZE is disabled while the preview loads, and when every partition already holds a single active part. Only one OPTIMIZE run can be in progress at a time. If you start another, the Failed to start OPTIMIZE toast shows An OPTIMIZE run is still in progress. Wait for it to finish before starting another. If the preview cannot load, a Failed to load OPTIMIZE preview toast appears.

Empty staging tables

Staging tables hold intermediate data while the platform loads or transforms data. Use Empty staging tables on the Storage toolbar to clear that leftover data; the button's tooltip reads Remove every row from the ClickHouse staging tables, keeping the tables.

The dialog lists three kinds:

  1. Open Observability > ClickHouse > Storage.
  2. Click Empty staging tables.
  3. Check the kinds you want to empty. All three are selected by default, and all three are selected again when you reopen the dialog. Each one shows its number of tables, parts, rows and size.
  4. Click Empty tables.
Important

The dialog says: "Removes every row from the selected ClickHouse staging tables. The tables themselves are kept." It lists only staging tables, so your own Lake and Warehouse tables are not emptied.

After the run, the Staging tables emptied toast reads Truncated N tables. If any table fails, the toast uses a warning style, names up to three failed tables with their errors, and says and N more for the rest. The other tables are still emptied.

If workflow jobs are running, the dialog shows Running workflows: <name> (#<id>) - try again when they finish. and Empty tables is disabled; it is also disabled while the counts load and when no kind is selected.

If a workflow starts after you open the dialog, the server refuses the run, and the Failed to empty staging tables toast shows Staging tables cannot be truncated while workflow jobs are running: <jobs>. Wait for them to finish or abort them, then try again. See Monitor and fix a run for how to stop a run.

The server also refuses while any ETL lock is held or still releasing, with Staging tables cannot be truncated while ETL locks are held or releasing (<n> holder(s): <holders>). Wait for them to finish or abort them, then try again. The holders are listed in ETL locks. Each staging table is then emptied while its owner is locked, so a table whose transformation or Lake table is in use at that moment is listed as failed in the toast, and the others are still emptied.

If the dialog cannot load its counts, the toast reads Failed to load staging tables. The Observability troubleshooting page lists what to do for each message.

Find out what changed a table

The DDL log view has its own page: Table DDL logs. Open it to follow every table change the platform made, read a run's statements, schema changes and errors, and export the log.

Permissions

PermissionWhat it unlocks
View Active QueriesThe Storage and Merges views.
Manage ClickHouse MaintenanceMerge parts into partitions and Empty staging tables. Without it, both buttons are hidden.

The ClickHouse Maintenance Manager preset includes View Active Queries and Manage ClickHouse Maintenance. Set these in Roles and permissions. The full list is in the Permission reference.

Questions, answered

When should I merge parts?

When a table has many active parts in one partition and its queries feel slow. Open Merges and read the preview. It shows how many parts can be reduced before you start.

Why is Start OPTIMIZE disabled?

Either the preview is still loading, or every partition already holds a single active part, so there is nothing to merge.

Can I start a second OPTIMIZE while one is running?

No. The platform refuses a second run and shows the message in the Failed to start OPTIMIZE toast. Wait for the current run to finish, then start again.

Does emptying staging tables delete my data?

No. It removes rows from the platform's staging tables only, and keeps the tables. Your own Lake and Warehouse tables are not in the list, so they are not emptied.

Why can't I empty staging tables?

Workflow jobs are running, or loads still hold or release ETL locks. Wait for them to finish or abort them, then try again. The dialog names the running workflows and keeps Empty tables disabled; the locks are listed in ETL locks.

Why was a table skipped by OPTIMIZE?

Something held the whole table when its statement was due: a schema change, a Lake or Writeback load, or a Fact load whose pattern is not periodic. The table keeps its parts. Start Merge parts into partitions again when the table is free; ETL locks shows what holds it.

What are the tables marked ETL temporary (read-only)?

The ETL staging and Retired tables tables: temporary tables that a Fact or Lake load uses while it runs, and previous versions kept briefly after a publish. They are removed automatically, so never edit or drop them by hand. The Temporary ETL tables panel at the top of Storage lists them with the run that owns each one. A Lake table's load table is listed there too, under Lake load tables, but carries no badge.

How do I find the biggest tables?

Open Storage and sort by Size On Disk. Turn on Hide system tables to leave out the database's own tables.

What can I export from Storage?

Excel only. The download is named clickhouse-table-storage-YYYY-MM-DD.xlsx and reflects the current search and Hide system tables setting.

Why does the Storage list jump back to the first page?

The list returns to its first page when you change the search or turn Hide system tables on or off.

Where can I find the DDL log?

It has its own page: Table DDL logs. It lists the changes the platform made to your tables, with each run's statements, schema changes and errors.

Why don't I see the maintenance buttons?

The buttons need Manage ClickHouse Maintenance. Ask an administrator to add it to one of your roles. See Roles and permissions.