From a legacy installation

The previous DataMind OS installation ran its services under systemd — PostgreSQL, ClickHouse, carte, Lurien, the API and frontend, and a handful of others — with an NFS share between some of them. The containerised deployment replaces all of it, but it will not retire the old units for you: two PostgreSQL engines on one data directory, or an old container holding a port the new stack needs, corrupts or blocks the install. This page runs that one-off host migration.

The migration is a shell script, migrate-host.sh, shipped with the DataMind Installer. It acts only on a fixed allowlist of units, waits for each stop to finish, and tees its whole run to a timestamped log. Run it before you deploy the DataMind Installer stack, on the host itself.

Before you start

Confirm the following on the host. The script enforces the first two itself and refuses to run without them.

• You are root (or using sudo) — otherwise it exits with ERROR: run as root (sudo).
• The host runs systemd (systemctl is present) — otherwise it exits with ERROR: not a systemd host.
• Docker is installed and its service is active. If docker.service is not active the script prints a warning and refuses, because once the bare-metal platform is gone Docker must run the stack next. You can override the refusal with --force, but only if you intend to bring Docker up immediately after.
• The old platform's data directories are already readable by the user the new stack will use. The script does not move or chown data; data-dir ownership and Lurien's Atlas baseline are handled by the containerised stack itself.

Run the dry run

Always run the dry run first. It echoes every command it would run (+ <command>) without changing anything, and it never prompts.

sudo ./migrate-host.sh --dry-run

Read the output and check it against what you expect on this host. In dry-run mode nothing is removed, no unit is masked, /etc/exports is not edited, and no containers are removed.

Keep the terminal output: it is the same sequence the real run performs, in order.

Run the migration

When the dry run shows only units you expect, run it for real. Without --yes the script prompts Proceed? [y/N] and aborts on anything but y, Y, yes or YES.

sudo ./migrate-host.sh --yes

The script is intended to be run first and standalone, before the DataMind Installer stack is deployed. The intended order is:

1. sudo ./migrate-host.sh
2. Deploy the DataMind Installer.
3. Set the environment variables (.env) in the DataMind Installer.
4. Start DataMind OS (docker compose up).

Flags

Unknown options print Unknown option: <arg> (try --help) and exit with code 2.

Environment variables

Set these inline before the command if you need different values, for example sudo STOP_TIMEOUT=120 ./migrate-host.sh --yes.

Run it from the DataMind Installer (optional)

Once the DataMind Installer stack is up, you can start the same script from its API instead of an SSH session. Set the VM user and the platform URL first — the API refuses the run without them:

# Set the SSH/OS user on the VM, used to substitute <USER> in generated .env values.
curl -X POST https://<host>:<port>/api/migrate/vm-user \
  -H 'Content-Type: application/json' \
  -d '{"vmUser":"ubuntu"}'

# Start the migration; the response carries a jobId.
curl -X POST https://<host>:<port>/api/migrate/run-host

Both endpoints require the admin role. If the VM user is unset you get VM user must be set before running host migration; if PLATFORM_URL is unset you get PLATFORM_URL must be set before running host migration.

Stream the run with Server-Sent Events, or follow it in the UI:

GET /api/deployment/jobs/<jobId>/logs

How the Installer executes it: it copies its bundled migrate-host.sh to the host path MIGRATE_HOST_SCRIPT_PATH (default /opt/delamain/migrate-host.sh), then starts a short-lived, fully privileged container from the DataMind Installer image and runs the script inside the host's namespaces:

docker run --rm --privileged --pid=host \
  unistream.azurecr.io/delamain:<version> \
  nsenter -t 1 -a -- bash /opt/delamain/migrate-host.sh --yes

nsenter -t 1 -a enters the namespaces of PID 1 on the host, which is what makes systemctl stop inside the container act on the host's units rather than on the container's own. The helper passes --yes, so there is no prompt. The migration job is treated as hung — and aborted — if it produces no output for 15 minutes (No output for 15 minutes — aborted as hung).

What it stops and removes

The platform units (allowlist)

The script neutralises exactly these units and nothing else. For each, it stops the unit (waiting for it to actually stop), disables it, then masks it — or, for a unit that lives in /etc/systemd/system, moves the unit file aside into BACKUP_DIR because mask cannot cover it there.

Anything not on this list it never touches — in particular Docker, the operating system itself, and Jenkins, which is deliberately left running so the old environment's values stay readable for import.

A unit that is not present or already masked is reported and skipped, so re-running the script is safe.

The NFS share

It stops nfs-server.service, unexports the share, removes the matching export line(s) from /etc/exports (after backing the file up to BACKUP_DIR/exports.bak.<timestamp>), re-exports, and then neutralises the NFS units:

Leftover container stacks

The old bare-metal Redash ran as its own Docker Compose project (default name redash), so the systemd steps above do not touch it. Its PostgreSQL mounts the same volume the new stack adopts, so leaving it up would mean two engines on one data directory. The script removes the redash project's containers but keeps its volumes for the new stack to adopt.

It also removes standalone leftover containers whose names collide with the new stack's — by default qdrant, an old container squatting on port 6333 (the new stack's own container is unistream-qdrant, so a bare qdrant is always a leftover). Extend the set with LEFTOVER_CONTAINERS="qdrant foo bar".

Any other non-DataMind Compose project is only reported, never removed — the script lists it under a note asking you to review or retire it manually.

Finish

The run ends with systemctl daemon-reload and systemctl reset-failed to clear any failed-unit state left behind.

The log it writes

From the point the real work begins, the script redirects stdout and stderr through tee, so the console output and the log file are identical. The log is:

/var/log/unistream-migrate-host-<YYYYMMDD-HHMMSS>.log

Override the location with LOG=/path/to/file.log. Because every command is echoed before it runs (+ systemctl stop …) and every stop's result is printed, the log reads like a manual session and is the artifact to attach if the migration goes wrong. The last line of the console output is Full log: <path>.

What it deliberately leaves behind

• Jenkins, left running so the old environment's values stay readable for import. Do not run the old Jenkins pipelines after the migration.
• The OS and Docker itself.
• All Docker volumes, including the leftover Redash volumes — the new stack adopts them.
• All data directories. Data-dir configuration and ownership, and Lurien's Atlas baseline, are handled by the containerised stack on first start, not here.
• Any Compose project not named redash (or listed in LEFTOVER_CONTAINERS) — these are reported for manual review only.

Verify

After the run, confirm no platform unit is set to auto-start. The script prints this check itself; run it again at any time:

systemctl list-unit-files --state=enabled \
  | grep -iE 'carte|clickhouse|pentaho|kettle|meltano|lurien|minio|postgres|unistream|reporting|jdbc|nfs|nginx'

Expected output is (none — good); any unit listed means it is still enabled and you should stop and disable it.

Also confirm no leftover container is holding a port the new stack needs:

docker ps --format '{{.Names}}\t{{.Ports}}'

Then continue with the deployment: deploy the DataMind Installer, set .env, and start DataMind OS.

Rollback

There is no automated rollback. The script stops, disables and masks the legacy units permanently; it does not restore them, and it makes no attempt to reverse itself if you change your mind.

What can be recovered manually, if you must:

• Unit files that were moved aside (units that lived in /etc/systemd/system) are in BACKUP_DIR (default /root/disabled-units). The script prints the list at the end. Restore one with mv back to /etc/systemd/system/ followed by systemctl daemon-reload.
• Units that were masked were not moved; unmask them with systemctl unmask <unit> and re-enable if needed.
• An edited /etc/exports was backed up to BACKUP_DIR/exports.bak.<timestamp> before the line was removed.

Because reversing all of that by hand is error-prone, treat the decision to run the real migration as final. If you are unsure of the outcome, stop after the dry run and move to new hardware instead.