Nodes in a run

Each run is made of nodes, one for each step. Use this page to read a run's node list and find out why a node was skipped or is still waiting.

Open the nodes of a run

Open Workflows in the sidebar and find the run in the runs table. See Job runs.
Double-click the run, or choose View in its row menu. The node window opens, titled with the workflow name.

Node types

The Node Type badge sits next to each node name.

Node type	What it does
Lake Table	Loads one Lake table.
Write-back	Pushes one Writeback table.
Transformation	Runs one Warehouse transformation.
History	Writes history for one Warehouse table and source system.
Gate	Waits for a table that another workflow fills. See Cross-workflow gates.
File Prep	Fetches and prepares files from an SFTP or FTP server.
Meltano	Pulls fresh data from a connector source.
API Call	Runs one API step of a connector load.
Initialize	Marks the start of a Warehouse run. It is already Finished and does no work.

A node from an older run may show no badge. If a node reports a kind that is not in this list, its badge shows the value as reported.

Two kinds of preparation node run before the tables of a source:

Meltano Load, followed by the source code in brackets, runs at the start of every run for a connector source. The table nodes of that source wait for it. If it fails, they are skipped with Skipped due to previous node error.
File Preparation, followed by the source code in brackets, appears only for files fetched from an SFTP or FTP server. Uploaded files do not get one. See Load files.

Node statuses

Status	Meaning
Pending	Waiting for the nodes it depends on.
Running	Working now. A gate that is waiting also shows Running.
Finished	Completed. A gate that has passed also shows Finished.
Error	Failed. Click the status to read the full error stack.
Skipped	Not run. The Note column gives the reason.
Aborted	Stopped because its run was aborted while it was running or pending.

A failed step reads Error; a failed run reads Failed. Each status carries an icon: a check for Finished, an alert for Error and Failed, a ban for Aborted, a skip marker for Skipped, a spinner while Running or Running Error, and a dot for anything else.

Why nodes are skipped

A node starts when the nodes it depends on have all finished, failed or been skipped. What happens next depends on how they ended:

If a node it depends on ended in Error or Aborted, or was skipped because of an earlier failure, the node is skipped with the note Skipped due to previous node error.
If a node it depends on was skipped because it is not eligible, the node still runs. For example, a transformation whose schedule is off does not stop the transformations after it.
If file preparation for a source is cancelled, its File Preparation node is skipped with a note that starts Preparation cancelled:. The tables of that source are then skipped with Skipped due to previous node error.

A skipped table load writes its reason into the table's Sync Log, in the Note. Some reasons start with Skip Reason:, for example Skip Reason: One Time Table Already ran for a one-time load that already ran. Other reasons mention a full reload that starts in the future, a sync type the engine cannot run, or a table with no incremental column to replace days by. See Run, stop and troubleshoot loads. The step messages and skip reasons are explained in Run logs and step messages.

Cross-workflow gates

A gate appears in a Warehouse run when a transformation reads a table that another workflow fills. Transformation dependencies are set on the transformation itself, see Build, test and finalize a transformation. The gate stops the transformation from reading that table before it is ready for the run's Current Date.

While it waits, the gate shows Running and checks the other workflow every 15 seconds.
It passes when the other workflow's run for the same Current Date has finished the table. The gate then shows Finished, and its Note says which run it waited for. If that workflow has not run yet for the date, the gate keeps waiting. If its run finished without that table, the gate passes and the Note records it.
It passes at once if the other workflow's schedule is off. The transformation then uses the last loaded data, and the Note records this.
It fails at once if the node that fills the table in the other run failed, or was skipped because of an earlier failure.
It fails after 60 minutes if the table is still not ready.

When the wait ends without the table, the gate ends in one of three ways (the default is Fail):

Fail: the gate fails, the nodes after it are skipped, the run ends Failed, and its failure alerts go out.
Skip: the gate is skipped with a note that starts Gate not satisfied:, and the nodes that wait on it are skipped too. The run does not fail for this reason.
Proceed: the run continues on the data that is already loaded. The Note reads Proceeded on possibly stale data:.

A failed gate shows Error with the message Cross-workflow dependency not satisfied:, followed by the table, its source system and the reason. Fix the failure in the other workflow, run it again if needed, then restart this run. See Monitor and fix a run.

Settings and defaults

Setting	What it does	Default
Show skipped nodes	Shows the nodes with status Skipped. The switch appears only when the run has skipped nodes.	Off
Show system nodes	Shows the control nodes that have no table behind them and processed no rows. The switch appears only when the run has such nodes.	Off
Rows per page	Sets 20, 50, 100 or 500 rows per page.	50

The table shows Node ID, Job ID, Node Name, Node Type, Status, Start Date, Duration, Current Date, Processed Rows and Error Stack. End Date, Extension, Note and Process ID are hidden until you show them. The table remembers your column layout in the browser, with no expiry.

Use the filter on a column header to narrow the rows: Node Name and Status show pick lists, the date columns show a date filter, and Duration shows a number range. Dates show as day, month, year, then time, for example 15/10/26 · 14:30:05. Duration counts up while a step runs. Use the copy button on Node ID, Job ID or a node name to copy the value; a check appears for a moment.

Find the step that failed

Open the nodes of the run.
If the run has skipped nodes, turn on Show skipped nodes and show the Note column to read why.
Filter the Status column to Error to find the failing node. See Filter, sort and search.
Click the Error status to read the full error stack.

Permissions

Any one of these permissions lets you read the nodes of a run: View Lake Workflow Objects, View Warehouse Workflow Objects or View WriteBack Workflow Objects. Without one of them, you cannot open the nodes. See Workflow permissions.

Limits and edge cases

A run keeps the plan it started with, so tables added to the workflow later wait for the next run.
The number next to the workflow name in the node window counts only the nodes you can see.
A skipped node and a system node do not open when you click their name. Other nodes open their table or transformation. If the table or transformation was deleted or renamed, clicking the name reads Table "<node name>" not found or Transformation "<node name>" not found.
The node window loads up to 10,000 steps of a run.
When no step is visible, the table shows No nodes found. If the list cannot load, a Failed to load nodes message appears; close the window and open the run again.
A test run stops after the staging copies, so its steps do not change the table. See How Dimension and Fact runs write data.
Processed Rows shows whole numbers, and shows 0 for a node that reports no row count. A 0 therefore does not prove that no rows were handled.

Questions, answered

Why does the node count differ from the number of tables I expect?

Skipped nodes and system nodes are hidden by default. Turn on Show skipped nodes and Show system nodes to see them. The number next to the workflow name counts only what is visible.

Half my tables were skipped after one failure. What do I do?

The tables that depend on the failed node are skipped with Skipped due to previous node error. Find the first Error node, fix its cause and restart the run. See Monitor and fix a run.

One transformation is paused. Will everything after it stop too?

No. A node skipped because it is not eligible does not block the nodes after it.

What is the Meltano node at the start of my run?

It is the Meltano Load step for a connector source. It pulls fresh data from the source before the tables load, on every run. If it fails, the tables of that source are skipped.

Why is there an Initialize node at the start of my Warehouse run?

Every Warehouse run carries it as a marker of the run's start. It does no work and is already Finished, so it never blocks a node after it.

My Warehouse run has been waiting on a gate for a long time. Why?

The gate waits until the other workflow finishes the table for the same Current Date. Open that workflow's run to see where it stands. If the table is still not ready after 60 minutes, the gate fails. The product has no setting to change this wait.

A gate failed with "Cross-workflow dependency not satisfied". What now?

The reason after the message says what the gate found: the other run did not produce the table, or the wait ran out first. Fix the other workflow, run it again if needed, then restart this run.

Will a gate wait forever if the other workflow is switched off?

No. When the other workflow's schedule is off, the gate passes at once and the transformation uses the last loaded data. The Note records this. See Schedules and when a workflow starts.

Why can't I open some node names?

A node opens only when a table or transformation sits behind it. Skipped nodes and system control nodes have no table behind them, so their names cannot be opened. Other node names open the table or transformation the step ran.

Why does a File Preparation node say "Preparation cancelled"?

Someone stopped the file fetch for that source before it finished, so no files were prepared. The tables of that source are skipped with Skipped due to previous node error. Run the workflow again when the files are ready.

Why does Processed Rows show 0 for some nodes?

The column does not separate "no count" from zero. A node that reports no row count shows 0.