Add a data source

Every source in the Lake starts from the same shared flow: choose a source type, fill in General, complete the steps that type needs, and save. This page explains the parts that work the same way for every type; each source type's own fields are on its page.

Adding a source needs the Add/Modify Lake Objects permission. See Who can do what in the Lake.

Start adding a source

The picker opens next. Choose All Sources, Popular or a category on the left, or search in Search sources..., then click a card to open its form.

Fill in the General step

The form loads with Loading connection form and a collapsible Documentation panel beside it. Point at the panel's handle to read Expand documentation or Collapse documentation.

FieldWhat it is
NameThe name you see in the Lake, at least 3 characters.
CodeA short code, at least 2 characters, using letters and numbers only. It becomes the prefix of the table names the source forms.
FolderOptional. The folder the source is filed under.

Until you fill them, the fields read Enter name, Enter code and Select folder (optional).

The Code fills in from the Name while you type. Edit it to set your own, and it stays as you set it until you change the Name again.

While you type a Name or Code, the field shows a spinner, then a green check when the value is free. When it is taken, the field shows This name is already in use or This code is already in use. The check runs about half a second after you stop typing.

Note

If the check cannot run, the field does not treat the value as taken; only an answer that the value is used counts as a conflict.

Which steps will I see?

The steps after General depend on the source type.

Source typeStepsRead more
SQL databases and SnowflakeGeneral > Connection > DatetimeConnect a SQL database, Connect Snowflake
MongoDBGeneral > Connection > ReviewSupported connectors
Connect AnythingGeneral > Build > Connection > ReviewConnect Anything (custom connector)
Connect Any APIGeneral > Recipe > Connection > Streams > ReviewConnect any API
Amazon S3General > Bucket > FormatConnect Amazon S3
RabbitMQGeneral > Broker > Tables > Load & reviewConnect RabbitMQ
WebhookGeneral > Sender & security > Table & retentionWebhook and RabbitMQ

File sources, such as CSV, TSV, JSON, Parquet and Excel Workbook, follow General > Source > Format > Tables > Review on Load files. The other types have their own steps or single-page forms, and Supported connectors routes each one to its page.

Datetime, where a source shows it, works the same way everywhere: Timezones and datetime values explains it.

Move between steps

Test and save

Open a form with a link

Add ?addSource= and a source type's name to the Lake address, for example ?addSource=Postgres, to open that type's form directly. The form opens once the source list has loaded, and the parameter is dropped from the address after it is used. ?openAddDataSource=true opens the add view for a new source the same way.

Edit or duplicate a saved source

Questions, answered

How do I fix "This name is already in use"?

Choose a different Name, or a different Code for the code message. Names and codes are checked against the other datasources on the platform, and the check is case-sensitive. The message clears when the value is free. While you edit a saved source the check does not run, because Name and Code are locked there.

Why did my Code change while I was typing the Name?

Code is derived from Name while you fill in the form. Once you edit the Code yourself, your version stays; it is derived again only when you change the Name again.

Why are Name and Code greyed out when I edit a source?

They lock when the source is saved, so its identity and its table-name prefix stay stable. Folder stays editable. To use another Name or Code, duplicate the source and set them on the copy.

Why can't I open a step further ahead?

Steps open in order: a step you have not visited yet is off. Finish the current step and click Next. When you edit a saved source, every step is open.

Why did the form jump to another step when I clicked Add or Save?

Something on an earlier step is not valid. The form moves to the step with the first error and marks it. Fix that step, then save again.

Where do I read the error when a connection test fails?

On a wizard, the test card shows Connection test failed with the message and a Copy button. On a single-page form, the footer shows Connection failed; point at it to read Error Details: and the message. For common causes, see Lake troubleshooting.

Why did the test result disappear after I changed a field?

The test describes the settings it ran with. Any change to one of those fields returns the test to Not tested, so run it again when you have finished editing.

Why is there no Test Connection button on my form?

The connection has nothing to reach out to, so there is nothing to test. Those forms leave the test out entirely.

Why is Add or Save greyed out?

The button is off while the form saves. It also stays off while there is nothing to save: on a single-page form, and when you edit a saved source, it turns on with your first change. If it never turns on, a required field or an earlier step is still unfinished; the reason beside Next names what is missing.

Can I link straight to a source type's form?

Yes. Add ?addSource= and the type's name to the Lake address, for example ?addSource=Postgres. It opens the form once the source list loads, and the parameter is dropped from the address after it is used. ?openAddDataSource=true opens the add view for a new source.

What happens to my entries if I close the form?

A form with changes asks first, with Unsaved Changes and the note that all changes will be lost. Confirm to close, or cancel to stay in the form with your entries. Back to Sources asks the same way.