Connect Webhook

Use a Webhook source when another system pushes events to DataMind OS. Events land in the buffer as they arrive, and the table you import from the connection loads them on its next sync. This page covers the connection form; the endpoint, the buffer and the responses a sender can get are on Webhook and RabbitMQ.

Add a Webhook source

  1. Open Lake and click Add datasource, then choose Webhook under Data Storage, or under Popular.
  2. On General, enter a Name of at least 3 characters and a Code of letters and numbers, at least 2 characters, and choose a Folder if you use them. These fields work the same way for every source; see Add a data source. Click Next.
  3. On Sender & security, choose a Sender and fill in the security fields that appear. The next two sections explain them.
  4. Click Next. On Table & retention, set the Table name and choose the Retention.
  5. Click Add. The wizard is replaced by the Connect your sender screen: copy the Endpoint URL into your sender's webhook settings, copy the token or signing secret if you generated one, then click Done.

Editing a saved connection opens on Sender & security.

Creating a webhook connection needs Add/Modify Lake Objects. See Who can do what in the Lake.

Choose a sender

The Sender list decides how the connection checks what arrives. Each sender checks exactly what its row describes, and nothing else.

SenderWhat it checks
GitHubThe vendor's HMAC-SHA256 signature, hex, with the sha256= prefix, in the X-Hub-Signature-256 header.
ShopifyThe vendor's HMAC-SHA256 signature, base64, in the X-Shopify-Hmac-Sha256 header.
WoltThe vendor's HMAC-SHA256 signature, hex, in the WOLT-SIGNATURE header.
Static token in a headerA shared token the sender puts in a header. This is the default; the header name starts as X-API-Key.
Custom HMAC signatureAn HMAC signature whose format you set yourself under Signature format.
No authenticationNothing.

The three vendor senders fix their own scheme: their signature format settings stay hidden and the Header name field does not appear, because the vendor decides the header.

Stripe and Slack are not supported, because they pack several values into one header.

Warning

No authentication verifies nothing. Anyone who learns the URL can write events into this connection. Only use it when the sender cannot sign or send a token.

Set up the security

Under Security:

After the first save the value is never shown again: the field reads Stored token or Stored signing secret with Replace beside it. Replace clears the field for a new value, and Keep the stored token or Keep the stored signing secret puts the stored value back if you cleared it by mistake.

On a saved connection, when you replace the secret and save, the checkbox Keep the old secret working during changeover appears, on by default: both the new and the old value are accepted until you stop the old one, so events signed before the sender catches up are not rejected. While the old value is still accepted, the section reads A previous secret is still accepted. with a Stop accepting button; clicking it changes the line to The previous secret stops being accepted when you save. before you save, and Undo takes the click back.

If you switch to a sender with a different scheme, the stored secret cannot be reused: the form says The sender uses a different scheme now, so the stored secret cannot be reused. and asks for the new token or signing secret.

Signature format

For Custom HMAC signature, the Signature format section sets how the digest is written:

SettingWhat it doesDefault
AlgorithmThe hash the sender signs with: SHA-256, SHA-1 or SHA-512. The sender's documentation names it.SHA-256
Digest encodingHow the digest is written in the header: Hex or Base64. A hex digest is twice as long as base64.Hex
Signature prefixLiteral text the sender puts before the digest, such as sha256=. Leave it empty when the header is the digest alone.Empty
Timestamp headerSet it only if the sender signs "timestamp.body". It also enables replay rejection.Empty
Replay window (seconds)Appears when Timestamp header is set. How far the sender's timestamp may drift before the event is rejected.300

Set the table and retention

On Table & retention:

OptionWhat it does
Keep foreverEvery event is kept, so any sync can rebuild from the full history. This is the default.
Delete after N daysOlder events are deleted and cannot be replayed. Days takes the number, at least 1.

With Delete after N days, the form warns: Expired events cannot be replayed. A table on this connection using a truncate-insert sync will lose that history on its next full reload. See Choose a sync type.

The step ends with a summary of Name, Endpoint (Created when you add the connection until the first save), Sender, Header, Token or Signing secret as Stored or New, Table and Retention.

After you save: the endpoint and the Request guide

After Add, the wizard is replaced by the Connect your sender screen. A banner reads Connection added, or Connection duplicated for a copy: Connect your sender: point it at the endpoint below. Events land in the buffer as they arrive and are imported on the next sync.

Then import the table you want from the connection and load it like any other source; see Import tables and Run, stop and troubleshoot loads. What the endpoint answers and how the buffer behaves is on Webhook and RabbitMQ.

Edit or duplicate a saved connection

For the messages you may see while you connect a webhook source, read Lake troubleshooting.

Questions, answered

Which sender should I choose?

Use a vendor sender when your system is GitHub, Shopify or Wolt: its scheme is set for you and you only provide the signing secret. Use Static token in a header when the sender can put a fixed token in a request; it is the default, with the header name X-API-Key. Use Custom HMAC signature when the sender signs the body with its own format, which you then set under Signature format. Keep No authentication as the last resort.

Can I connect a Stripe or Slack webhook?

No. Stripe and Slack are not supported, because they pack several values into one header. If your system can send a fixed token or its own HMAC signature instead, use Static token in a header or Custom HMAC signature.

What happens if I choose No authentication?

Nothing is verified: Nothing is verified. Anyone who learns the URL can write events into this connection. Only use it when the sender cannot sign or send a token.

What should I put in Header name?

The header the sender puts its credential in; matching ignores case. For Static token in a header it starts as X-API-Key; for Custom HMAC signature enter the header the sender uses (the field suggests X-Signature). GitHub, Shopify and Wolt fix their own header, so the field is not shown for them.

Should I paste the secret or generate one?

Paste the value the sender and the connection share: the token the sender will send, or the signing secret the sender signs with. Click Generate only when the sender is your own service; it creates a value and shows it so you can copy it into the sender. Either way, the value is masked after saving and cannot be shown again.

Can I change the sender after the connection is saved?

Yes. Open the connection, pick another Sender on Sender & security, and save. If the new sender uses a different scheme, the stored secret cannot be reused: the form says The sender uses a different scheme now, so the stored secret cannot be reused. and asks for the new token or signing secret.

Why does my sender show as Custom HMAC signature when I reopen the connection?

A saved connection stores the scheme, not the sender's name. When the form opens, it matches the stored scheme: an exact match to GitHub, Shopify or Wolt shows that sender, and anything else opens as Custom HMAC signature, with the saved header, algorithm, encoding and prefix filled in. The connection keeps checking the same scheme.

Can I change the table name later?

No. Table name is fixed once the connection is saved: Fixed once the connection is saved: the importer bakes it into the core table name. To use a different name, add a new webhook connection with the name you want.

Should I keep events forever or delete them after N days?

Keep Keep forever when a table on the connection must be able to rebuild from the full history; it is the default. Choose Delete after N days when you only need recent events: older ones are deleted and cannot be replayed, and a table using a truncate-insert sync loses that history on its next full reload. See Choose a sync type.

What exactly does my sender have to send?

Open Request guide on the connection. It shows the exact request for the sender you chose: the endpoint (POST plus the URL), the headers the request must carry, and an example you can copy. The secret is never shown there: The secret is never shown here - paste the one set on this connection.

Can I duplicate a webhook connection?

Yes. A duplicate starts without the stored secret and without the endpoint; enter the token or signing secret again and save. The copy gets its own endpoint when it is saved, so the two connections stay apart.