Backup and restore

A deployment has two backups. Take both on a schedule and before every update.

Part	What it holds	Size	Section
The DataMind Installer	Its database, its secrets and the deployment files it writes	Small, a few files	Back up and Restore
DataMind OS data	Your lake, warehouse and dashboards, in the deployment's Docker volumes	Large, grows with your data	Back up DataMind OS data

Know what you are backing up

The DataMind Installer keeps its state in two Docker volumes and one directory:

What	Where	What is in it
Database	`delamain_pgdata`	Every deployment setting, the user accounts and their sessions, the audit trail and the job history
Secrets	`delamain_delamain_secrets`	`jwt.secret`, `encr.key`, `pg.pass`, `curato.token`
Files	`/opt/delamain`	`docker-compose.yml`, the optional `.env`, and the `deployment` directory

Compose prefixes each volume with the project name, which is the install directory's name. The commands below use the names for /opt/delamain. If you installed elsewhere, list the real names first:

bash

docker volume ls --filter name=delamain_

Check it worked. The output lists both volumes:

text

DRIVER    VOLUME NAME
local     delamain_delamain_secrets
local     delamain_pgdata

Back up encr.key with the database dump it belongs to. What each secret file does is in What lives on the host.

Warning

Keep encr.key with its dump. It encrypts the Azure client secret, every configuration value marked secret and every value the DataMind Installer generated, including the DataMind OS database passwords. Without it, those values read back as empty. The next Start, Apply changes or Update writes them empty into deployment/.env, an Update also generates new passwords, and the DataMind OS services can no longer open their data. Before any of these actions, re-enter each value in Configuration, copying it from the deployment/.env in your files archive, and re-enter the Azure client secret under System secrets with the client_secret DataMind issued with your licence.

Back up

You do not need to stop the DataMind Installer. Start when Status shows no running job.

Run every step as root, in the same shell: later steps use the variables set here. umask 077 makes the backup directory readable by root only. Keep the archives in it, and protect them the same way wherever you copy them.

bash

cd /opt/delamain
umask 077
mkdir -p /root/datamind-backup
BACKUP=/root/datamind-backup
DATE=$(date +%F)

1. Dump the database

The command reads the database user and name from the container, so a changed POSTGRES_USER or POSTGRES_DATABASE_NAME still works.

bash

docker exec delamain-postgres sh -c 'pg_dump -U "$POSTGRES_USER" -d "$POSTGRES_DB"' \
  > "$BACKUP/datamind-installer-db-$DATE.sql"

Check it worked.

bash

head -n 3 "$BACKUP/datamind-installer-db-$DATE.sql"

text

--
-- PostgreSQL database dump
--

2. Archive the secrets volume

The command uses the PostgreSQL image the DataMind Installer already runs, so it needs no extra download.

bash

docker run --rm \
  -v delamain_delamain_secrets:/data:ro \
  -v "$BACKUP":/backup \
  unistream.azurecr.io/docker-hub/library/postgres:16 \
  tar czf "/backup/datamind-installer-secrets-$DATE.tgz" -C /data .

Check it worked.

bash

tar tzf "$BACKUP/datamind-installer-secrets-$DATE.tgz"

text

./
./pg.pass
./jwt.secret
./encr.key
./curato.token

The order can differ. All four names must be there.

3. Archive the files

The command adds .env only when it exists.

bash

FILES="docker-compose.yml deployment"
[ -f .env ] && FILES="$FILES .env"
tar czf "$BACKUP/datamind-installer-files-$DATE.tgz" $FILES

Check it worked.

bash

tar tzf "$BACKUP/datamind-installer-files-$DATE.tgz" | grep -E '^(docker-compose.yml|\.env|deployment/\.env)$'

The output shows docker-compose.yml, deployment/.env, and .env when you have one.

4. Record the build and copy the backup off the host

The file records the exact DataMind Installer build. Give it to DataMind if a restore fails.

bash

docker image inspect -f '{{.RepoTags}} {{index .RepoDigests 0}}' \
  "$(docker inspect -f '{{.Image}}' delamain-backend)" \
  > "$BACKUP/datamind-installer-version-$DATE.txt"

Check it worked. ls /root/datamind-backup lists the .sql, the two .tgz files and the .txt file with today's date.

Copy these four files to storage outside this host. deployment/.env holds every DataMind OS password, and the secrets archive holds encr.key.

Restore

Use this procedure on the same host or on a new one. The database dump loads into an empty database volume. You never copy the old volume.

Important

Restore the secrets volume before any docker compose up. A first start with an empty secrets volume generates new keys, and the encrypted settings in the dump can no longer be read.

Run every step as root, in the same shell. Step 1 is for a new host only. Step 2 is for the same host only.

Copy the backup to /root/datamind-backup on this host, then set the backup location:

bash

BACKUP=/root/datamind-backup
DATE=2026-10-01   # replace with the date in your backup file names

1. New host only: prepare the host

Install Docker, as in Install on a fresh host. Then create the shared network and sign in to the registry with the client_id from DataMind. Enter the client_secret when prompted for the password.

bash

docker network inspect unistream >/dev/null 2>&1 || docker network create unistream
docker login -u <client_id> unistream.azurecr.io

Check it worked. docker login prints Login Succeeded.

2. Same host only: empty the old volumes

The command first checks that both archives can be read. Then it stops the DataMind Installer and removes both volumes: a dump loaded into a database that already has tables fails part-way.

bash

tar tzf "$BACKUP/datamind-installer-secrets-$DATE.tgz" >/dev/null \
  && tar tzf "$BACKUP/datamind-installer-files-$DATE.tgz" >/dev/null \
  && cd /opt/delamain \
  && docker compose down \
  && docker volume rm delamain_pgdata delamain_delamain_secrets

Check it worked. This lists no volumes:

bash

docker volume ls --filter name=delamain_

3. Put the files back

bash

mkdir -p /opt/delamain && cd /opt/delamain
tar xzf "$BACKUP/datamind-installer-files-$DATE.tgz"

Check it worked. ls -a in the install directory shows docker-compose.yml and deployment.

4. Restore the secrets volume

bash

docker volume create delamain_delamain_secrets
docker run --rm \
  -v delamain_delamain_secrets:/data \
  -v "$BACKUP":/backup:ro \
  unistream.azurecr.io/docker-hub/library/postgres:16 \
  tar xzf "/backup/datamind-installer-secrets-$DATE.tgz" -C /data

Check it worked.

bash

docker run --rm -v delamain_delamain_secrets:/data:ro \
  unistream.azurecr.io/docker-hub/library/postgres:16 ls /data

text

curato.token
encr.key
jwt.secret
pg.pass

Compose later warns that this volume was not created by Docker Compose. The warning is harmless, here and for every volume you create by hand.

5. Start the database alone and load the dump

Start only PostgreSQL. It creates an empty database volume and the database named by POSTGRES_DATABASE_NAME in the Installer's .env, or the default.

bash

cd /opt/delamain
docker compose pull
docker compose up -d postgres

Wait until docker compose ps postgres shows (healthy). Then load the dump. ON_ERROR_STOP=1 stops at the first error. Do not run CREATE DATABASE: the container already created it.

bash

docker exec -i delamain-postgres sh -c \
  'psql -v ON_ERROR_STOP=1 -U "$POSTGRES_USER" -d "$POSTGRES_DB"' \
  < "$BACKUP/datamind-installer-db-$DATE.sql"
echo "exit code: $?"

Check it worked. The last line reads exit code: 0, and no line starts with ERROR:. On the same host, an error means the volume was not empty: repeat step 2. On a new host, send the first ERROR: line to DataMind.

6. Start everything

bash

docker compose up -d

The one-shot migration container brings the restored schema up to the running build. Then the backend container starts.

Check it worked.

bash

docker compose ps
curl -sS http://localhost:8000/api/health

The backend container shows (healthy), and the health check answers:

json

{"status":"ok","db":"up"}

Roll back only the DataMind Installer database

On the same host, to return the DataMind Installer to the dump you took before an update, without touching the files or the secrets:

bash

cd /opt/delamain
docker compose down
docker volume rm delamain_pgdata

Then run steps 5 and 6.

Back up DataMind OS data

Your lake, warehouse and dashboards live in the Docker volumes of the DataMind OS deployment. This procedure archives every one of them. Run it as root, in the same shell, in a maintenance window: the deployment is stopped while the archives are written.

1. List the deployment's volumes

Read the Compose project name, then list every named volume its containers mount. Filtering by the project label includes external volumes too.

bash

cd /opt/delamain/deployment
PROJECT=$(docker compose config | awk '/^name:/{print $2; exit}')
echo "project: $PROJECT"
VOLUMES=$(docker ps -aq --filter "label=com.docker.compose.project=$PROJECT" \
  | xargs docker inspect -f '{{range .Mounts}}{{if eq .Type "volume"}}{{println .Name}}{{end}}{{end}}' \
  | grep -v '^$' | sort -u)
printf '%s\n' $VOLUMES

Check it worked. The first line names the project, and one volume name follows per line.

Check the disk space. The free space on /root must exceed the total of the volume sizes listed under Local Volumes space usage:

bash

docker system df -v
df -h /root

2. Choose the host directories

Some services mount host directories instead of volumes. List them:

bash

docker ps -aq --filter "label=com.docker.compose.project=$PROJECT" \
  | xargs docker inspect -f '{{range .Mounts}}{{if eq .Type "bind"}}{{println .Source}}{{end}}{{end}}' \
  | grep -v '^$' | sort -u

Put the data directories from that list in HOSTDIRS. Leave out /var/run/docker.sock and /: they are host infrastructure, not data.

bash

HOSTDIRS="/srv/example-data"   # replace with your data directories, separated by spaces

3. Stop the deployment

A file-level copy of a database is consistent only while the database is stopped. Click Stop on Status, or run on the host:

bash

cd /opt/delamain/deployment
docker compose stop

Check it worked. docker compose ps lists no running services.

4. Archive every volume and host directory

bash

DATE=$(date +%F)
OSBACKUP=/root/datamind-backup/os-$DATE
umask 077
mkdir -p "$OSBACKUP"
printf '%s\n' $VOLUMES > "$OSBACKUP/volumes.txt"
for v in $VOLUMES; do
  docker run --rm \
    -v "$v":/data:ro \
    -v "$OSBACKUP":/backup \
    unistream.azurecr.io/docker-hub/library/postgres:16 \
    tar czf "/backup/$v.tgz" -C /data .
done
printf '%s\n' $HOSTDIRS > "$OSBACKUP/hostdirs.txt"
for d in $HOSTDIRS; do
  tar czf "$OSBACKUP/host$(echo "$d" | tr / _).tgz" -C / "${d#/}"
done

Check it worked. One archive exists per volume, and both counts match:

bash

ls -lh "$OSBACKUP"
wc -l < "$OSBACKUP/volumes.txt"
ls "$OSBACKUP"/*.tgz | grep -v '/host_' | wc -l

5. Start the deployment again

Click Start on Status, or run on the host:

bash

cd /opt/delamain/deployment
docker compose start

Copy $OSBACKUP off the host with the DataMind Installer backup.

Restore DataMind OS data

On a new host, restore the volumes before the deployment starts for the first time. A service that starts on an empty volume fills it with new files, and the archive would then be unpacked on top of them.

Run every step as root, in the same shell. Step 2 is for the same host only.

1. Set the backup directory

bash

OSBACKUP=/root/datamind-backup/os-2026-10-01   # replace with your backup directory

2. Same host only: remove the old volumes

Remove the containers and each volume you restore. docker compose down keeps the volumes. The loop removes a volume only after tar tzf has read its archive.

bash

cd /opt/delamain/deployment
docker compose down
for v in $(cat "$OSBACKUP/volumes.txt"); do
  tar tzf "$OSBACKUP/$v.tgz" > /dev/null && docker volume rm "$v"
done

3. Create and fill each volume

bash

for v in $(cat "$OSBACKUP/volumes.txt"); do
  docker volume create "$v"
  docker run --rm \
    -v "$v":/data \
    -v "$OSBACKUP":/backup:ro \
    unistream.azurecr.io/docker-hub/library/postgres:16 \
    tar xzf "/backup/$v.tgz" -C /data
done

Check it worked. docker volume ls lists every name in volumes.txt.

4. Put the host directories back

Each directory returns to its original path.

bash

for d in $(cat "$OSBACKUP/hostdirs.txt"); do
  tar xzf "$OSBACKUP/host$(echo "$d" | tr / _).tgz" -C /
done

5. Start the deployment

Same host: click Start on Status.
New host: follow Start the deployment.

Check it worked. docker compose ps in deployment shows every service Up, and your users see their data.

Questions, answered

Does the DataMind Installer back up DataMind OS data?

No. Back it up yourself with Back up DataMind OS data. It archives every volume of the deployment in one loop.

What has to be backed up together?

The database dump and the secrets archive. See the encr.key warning.

Do I have to stop anything to take a backup?

Not for the DataMind Installer backup: wait for any running job to finish, then run it. Stop the DataMind OS deployment while its volumes are archived, then start it again.

What if I lost the secrets archive?

After a restore, the DataMind Installer starts with new keys and everyone is signed out. Re-enter the values as the encr.key warning describes, before any Start, Apply changes or Update.