Use a RabbitMQ source when your messages already sit in queues on a broker. DataMind OS reads the messages into a buffer, and the tables you import from the connection load from it. The same Broker card also serves the RabbitMQ destination form, which publishes rows out of the Lake.
Creating a RabbitMQ connection needs Add/Modify Lake Objects. See Who can do what in the Lake. Editing an existing connection opens on Broker.
| Setting | What it does | Default |
|---|---|---|
| Host | The broker's host name or IP address. Required. | None |
| Port | The AMQP port. Switching TLS changes it while it still holds its default value. | 5672, or 5671 with TLS |
| TLS | Connects over AMQPS. The ports follow the switch while they are defaults. | Off |
| Virtual host | The virtual host the connection uses. | / |
| Management API port | Used to find queues. Leave the default unless the management plugin listens elsewhere. | 15672, or 15671 with TLS |
| Username | The broker user's name. Required. | None |
| Password | The broker user's password. Required. | None |
The optional Test connection logs in over AMQP and asks the management API which queues it can see. Its result names what answered: AMQP login ✓ · Management API ✓ with the number of queues it found, or AMQP login ✓ · Management API ✗ followed by the error, or no answer. A stored password is never shown; the form shows it as stored, with Replace beside it.
Under Queues on the Tables step, choose how tables map to queues:
The queue mode, Exchange and Exchange type stay locked while the connection has imported tables; see the questions below.
Refresh under Discovered tables asks the management API and lists what it can offer as tables: the routing keys bound to your exchange in managed-queue mode, or the queues on the virtual host with Consume existing queues. Refresh is disabled while the broker test runs.
Discovered tables reports each state in one line:
When you would rather not rely on discovery, open Define tables manually and click Add table.
_message_id and _ingested_at.When at least one table is listed here, only these tables are offered and discovery is skipped. Remove every row to go back to discovery.
On Load & review:
The review summarises Name, Broker, Virtual host, Username, Queue mode, Tables and Buffer; Exchange appears in managed-queue mode, and Consumers and Queues per table when they are set above 1.
Click Add to create the connection, Save when editing, or Duplicate to copy it.
The same Broker card serves the writeback direction. Open WriteBack, click Add destination and choose RabbitMQ. This form is a single page, not a wizard: General, Broker, Test connection, Exchange & Queues and Message Schemas (optional). Its footer has no Test connection action of its own; the broker check sits in the Test connection card. When you edit a destination, the broker fields prefill and a stored password shows as stored with Replace beside it.
Exchange & Queues adds Exchange and Exchange Type for publishing, and Message Schemas (optional) controls routing keys and the message format. See Publish to RabbitMQ and Add and manage destinations.
The first option fits a new setup: DataMind OS creates a queue for each table and binds it to your exchange with the routing key, so fill in Exchange and Exchange type. Consume existing queues fits a broker whose queues are already set up; DataMind OS declares nothing.
Only for discovery and the management check in the test. The connection reads messages over AMQP, and the management API is used to find queues and routing keys. Keep Management API port at 15672, or 15671 with TLS, unless the plugin listens elsewhere. If it cannot be reached, define the tables manually.
Because tables are imported from the connection. The form shows: "Locked while tables are imported from this connection. Delete those tables first to change it." Delete those tables, then edit the connection.
No. Click Refresh under Discovered tables and every queue or routing key the broker reports becomes a table you can import. If you add tables manually, you can still leave Columns empty and let them be sampled from live messages at import.
They are added to every table imported from a RabbitMQ connection. Without a key column, _message_id is the key.
30 days by default. Change Keep consumed messages for (days) on Load & review; the minimum is 1. Older messages are deleted from the buffer, and tables already imported keep the window they were created with.
Open Advanced on Load & review and raise Consumers or Queues. Each one applies when set above 1: more parallel consumers per table, or more queues per table.
Yes, while the ports still hold their defaults: Port switches between 5672 and 5671, and Management API port between 15672 and 15671. A port you edited yourself keeps its value.
A stored password is tied to the broker Host and Username. Change either one and the form clears the stored password and asks for it again; enter it and save.
Yes. The test is optional: "Optional - you can continue without it." If the login fails, check Host, Port, Username and Password. If only the management API did not answer, continue and define the tables manually.
Add a RabbitMQ destination under WriteBack. It is a single-page form that reuses the Broker fields and adds Exchange & Queues and optional Message Schemas. See Publish to RabbitMQ.