Keep it healthy

Keeping DataMind OS healthy is a routine: give colleagues the access they need, catch failed runs before anyone asks, and warn people when a query finds something they should see. This tutorial does each of these once, then shows you how to check what depends on a table before you change it.

Before you start

Your administrator gives you these permissions through your roles:

• Add User adds users. The Users Administrator permission includes it.
• View Lake Workflow Objects shows Lake runs, and Execute Lake Workflow Objects restarts them. The Warehouse and WriteBack versions work the same way.
• Permission to add workflows, such as Add Lake Workflows, shows the failure alert bell on a workflow card.
• Create Alerts lets you make query alerts. Notify Others With Alerts lets you send them to roles and other people.
• View Lake Objects, View Warehouse Objects or View WriteBack Objects lets you open Lineage.

Add a user with a role

1. Open Admin > Profile Management > Users.
2. Click Add User. The Add a User form opens.
3. Under User Details, enter a User Name and an Email.
4. Under Security, enter a Password and repeat it in Confirm Password. Meet the requirements listed under the password field.
5. Under Access, pick the Roles the user needs. Type in Search roles... to filter the list.
6. Click Create.

The form closes, a Success notice reads "User registered successfully", and the new user's page opens. The new user must change the password at first sign-in, as described in Sign in and sign out. To choose the right roles, see Roles and permissions. To edit, deactivate or reset a user later, see Users.

Watch workflow runs for failures

1. Open Workflows.
2. Look at the status dot on each workflow card. Its colour shows whether the latest run finished, is running, or failed or ended in an error.
3. For a workflow whose latest run failed, look at the runs panel on the right and find the run with a Failed status.
4. Click the Failed status to open Error Stack Trace and read the message. Double-click the run to open its nodes and find the step that failed.
5. Fix the cause. Then right-click the run and choose Restart.

Restart starts the run again, and its status updates in the list. It is greyed out while a run is still running. For the full run view, see Monitor and fix a run.

A run shows one of these statuses:

• In the node window, a failed step shows the status Error. Click it to read its Error Stack Trace, and click Copy to copy the text.
• Restart is offered for Failed and Aborted runs. Stop is offered for Running and Running Error runs. Each asks you to confirm, in Restart Job or Stop Job.
• The runs table hides the Error Stack and Note columns by default. Click Toggle column visibility to show them next to every run.

Set up failure alerts

A failure alert tells you when a workflow run fails, so you do not have to check the list yourself.

1. On the workflow card, click the bell. Its tooltip reads Set up failure alerts.
2. In the popup, on the Alert tab, click each channel you want. A channel is an Email, Slack, Discord or Google Chat connection that an administrator has set up.
3. For an email channel, click its recipients button, pick who gets the email, and click Done.
4. Click Create alert. Next time, the button reads Save.

The bell's tooltip now reads Alerts on, followed by the number of channels. For more, see Failure alerts.

Create a query alert

A query alert runs a query on a schedule and emails its recipients when the condition is met. It needs an email channel that an administrator has set up in Notifications. Without one, the alert cannot send.

1. Open Editor and write a query that returns a row only when something needs attention. Run it to check the result.
2. Click the alert button in the editor header. Its tooltip reads Create an alert from this query. The button stays greyed out until the editor holds some SQL. The New alert form opens with your query in it. You can also start from Observability > Alerts with New alert, and write the query in the form.
3. Enter a Name. Pick the role in Runs as. The query runs with that role's access, so the alert keeps working after you leave.
4. Under Conditions, keep Query based, which triggers whenever the query returns a row. Choose Rule based to trigger only when the rules you add match.
5. Under Schedule, set Runs and Time. The time follows the server's clock, which the form shows under the time picker.
6. Under Recipients, pick Roles, People, or both.
7. Click Test, or press Ctrl+Enter in the query. The result reads Would trigger or Would not trigger, with the number of rows returned. View rows shows the rows, and Preview email shows the email as it would be sent. Testing sends nothing.
8. Click Create alert. The notice reads Alert created.

The alert now appears in Observability > Alerts, with its Status, Schedule and Next Run. A new alert shows Pending until its first run. After that it shows OK, Triggered or Error, or Paused if you pause it. When a run fails, Last Error gives the reason.

Roles appears only if you have Notify Others With Alerts. Without it, you can pick only yourself. To pause or edit an alert later, see Manage alerts.

Check what depends on a table

Check this before you rename or remove a table, so you know which reports and Minds would be affected.

1. Open Lineage.
2. Click Focus in the view switch. The Pick what to focus on card appears.
3. In the search box, type the table's name and select it. The map now shows only that table's upstream and downstream lineage.
4. Find Downstream in the toolbar. It starts at All, so every item that depends on the table appears. Choose 1 or 2 to narrow it.
5. Click Back to leave Focus and return to the previous view.

Lineage shows only the tables you may see, and Minds, decks and DataPrism models only if you can read them. See Explore and trace lineage.

Look further in Observability

Besides Alerts, Observability has tabs for platform health, the SQL queries that ran, database activity, logs, Mind usage and live metrics. They are mainly for administrators, and you see only the tabs your permissions allow. See Observability.

Questions, answered

The new user did not get a welcome email. What do I do?

DataMind OS sends the welcome email only when an email channel is active and set as the default under Admin > Notifications. Without one, no email goes out, and the failed attempt is recorded under Observability > Logs > Notifications, if you can read notification logs. Share the password with the new user yourself, through a channel you trust, and make sure they change it at first sign-in.

The bell says "Set up notification channels in Admin first". What now?

Ask an administrator to add an Email, Slack, Discord or Google Chat channel under Admin > Notifications. Until a channel exists, the bell is disabled. See Notifications.

How do I test an alert without emailing everyone?

Click Test in the alert editor. DataMind OS runs the query with the role in Runs as and shows Would trigger or Would not trigger, with the first rows the query returns. It sends no email, and nothing is saved until you click Create alert.

Will I get an email every time the alert runs?

Not by default. When it changes is the default. It emails you when the condition starts being met and again when it stops. Every match emails you on every run where the condition is met.

How do I find what depends on a table before I drop it?

Open Lineage, click Focus, search for the table, and check Downstream. You see what the table feeds, and items you cannot read do not appear.