Service catalog

The service catalog is the list of services the installer presents for a DataMind OS deployment: the groups shown in the UI, the services inside each group, which of them are one-shot initialisers, and which are marked critical. It is the join of two sources:

the published schema (.env.unified.template, downloaded from blob storage), whose $services key holds the groups and per-service metadata;
the compose file on disk (deployment/docker-compose.yml and, when present, its override), read through docker compose config, which holds which services actually exist.

The installer never invents the service list on its own: a service is only shown if it appears in the compose topology, and a group label only comes from the schema.

Where the catalog comes from

The installer downloads the schema and the compose file as a matched pair, selected by a single variant switch rather than two independent URLs. They are validated against each other before publication, so a mismatch (a dev schema against a production compose) cannot report every service as ungrouped. See BLOB_ARTIFACT_VARIANT in Environment variables.

The schema's $services object has two parts:

groups — each entry has a key, a label and an icon.
services — a map of compose service name to { group, critical, description, icon }.

The schema also carries $groupMeta (group and subgroup labels and icons) and $memoryLimits (the per-service memory sizing rules). The memory rules produce the computed rows written into the deployed stack's environment file; see Environment variables.

Note

The schema file itself is not part of this repository. The installer only consumes it at runtime; the concrete group keys, service names and descriptions it contains are defined by the team that publishes it.

Fallback when the schema is unusable

If the schema template is missing, unparseable, has no usable $services key, or references a group that does not exist, the installer falls back to a synthesised catalog instead of failing:

one group, key services, labelled Services;
every compose service placed in it;
no descriptions, no critical flags.

The install API reports which source was used, so a fallback catalog is visible rather than silent. Services present in compose but absent from the schema are also reported as ungrouped, and services named in the schema but absent from compose as missing.

Composing a deployment's service list

When an action targets "all services", the installer takes every service in the compose topology. When an action names services, each name is validated against the topology and an unknown name is a hard error.

Two rules refine the list:

One-shot services are excluded from restart and targeted runs. A service carrying the compose label com.unistream.lifecycle: oneshot is an init container: it re-runs automatically on a full deploy and cannot be restarted on its own. Naming one in a restart is refused with an explanatory message.
Lifecycle defaults to service. Any label value other than oneshot or service is treated as service. Lifecycle drives how container health is reported: a one-shot is completed when it exits zero, a long-running service is up when healthy.

The compose label com.unistream.lifecycle is the mechanism that marks init containers; there is no separate flag in the installer's own configuration.

Grouping, critical flags and health

Each service view carries:

group — from the schema's $services.
lifecycle — from the compose label.
critical — true when the schema marks critical: true; targeting a critical service emits a warning before the job runs.
hasHealthcheck — whether the compose service defines a healthcheck. Services with no healthcheck are unverifiable: after recreation the installer rechecks them for crash-looping rather than trusting a missing health signal.
description and icon — from the schema.

Enable and disable

There is no boolean "enabled" flag in the catalog itself. A service's presence is governed by the compose topology: if it is in the compose file, it is in the catalog. Two mechanisms affect whether a service actually runs:

One-shot lifecycle — init containers run on a full deploy and exit, rather than staying up.
Computed enablement rows — the memory-limits rules can mark a service optional or disabled by default for a given host class. The installer materialises this as a <PREFIX>_ENABLED row in the deployed stack's environment file, which the service reads. This is sizing-driven, not a UI toggle.

Whether individual services can be additionally enabled or disabled through compose profiles is not visible in this repository, so it is not documented here.

How the catalog relates to the compose files on disk

The catalog is a view over the compose file, not a copy of it. Consequences:

Adding or removing a service in compose changes the deployment's service list on the next read; the installer caches the topology but invalidates the cache when the compose or environment file changes.
Each compose command runs against the compose file and, when present, docker-compose.override.yml. The override file is optional and user-managed — the installer never writes it. See Compose files and files on disk.
The published compose file is overwritten on every deploy and on boot-time restore, so service definitions added to it by hand do not survive.

Service names referenced in this repository

The concrete service list lives in the published schema and compose file, which are not in this repository. The names below are the ones that appear in the installer's code and scripts, so they document behaviour that is defined here — not the catalog's full contents.

Name	Where it appears	Why it matters
`backend`	installer's own compose	The installer's backend service; the self-update helper targets it
`backend-init`	installer's own compose	One-shot migration runner that must complete before `backend` starts
`postgres`	installer's own compose	The installer's own database
`lurien`	deployment engine	Treated specially on restart: it re-reads the mounted `.env` but keeps its baked-in environment, so a plain restart can leave it running mixed configuration — recreate it with Apply changes instead
`curato`	deployment engine, secrets	The service that presents the deployment token; the backend advertises itself to it under a pinned network alias
`openresty`, `unistream-redis`, `victorialogs`, `fluent-bit`	deployment engine (commented-out protected set)	The observability chain the deployment engine once refused to target; the guard is currently disabled in code

Configuration groups that the UI knows by default (used for grouping the deployed stack's config rows) include platform_urls, network_ports, postgres, clickhouse, clickhouse_api, lurien, pentaho_carte, rustfs, redash_reporting, service_metrics, backend, secrets, ssh, meltano, qdrant, host_volumes, two_factor, redis, nats, mcp_server, rosetta, vespa and logging. These are UI group keys from the published schema, not service names.

For the host-migration path, the installer retires a fixed allowlist of bare-metal systemd units (services such as clickhouse-server, jdbc-bridge, meltano-api, minio, carte, reporting-system, service-metrics-api, lurien, postgresql and nginx) so the containerised stack can take the host over. That list is in the migration runbook, not the service catalog.