Getting started

Sign in and pick a workspace

Everything in Pipeloom (connectors, pipelines, variables, custom components) belongs to a workspace. If your organization already has one, sign in and select it. Otherwise, create one from the workspace switcher.

Create a pipeline and add a source and a destination

Create a new pipeline to open its canvas. From Add component → Connectors, place a Source step and a Destination step.

Select each step to choose its connector. Pick one your workspace already has, or set up a new one without leaving the canvas. For example, a Postgres database as the source and Snowflake or BigQuery as the destination. The connector reference has setup guides for individual connectors.

Connect them: the line is the sync

Drag a line from the source to the destination. That line is the sync: it copies data from one to the other. Select the line to set it up. Use an existing sync between those two connectors, or set up a new one and choose which streams (tables) to copy.

The pipeline can't be deployed until every sync on it is set up.

Add what should happen after the data lands

Connect more steps after the destination, such as a sql.query transform, a dbt.build, or a Slack notification. They run once the sync has landed its data. The orchestration reference lists every step type.

Prefer to describe it instead? Ask the AI Copilot. It can create the connectors and the sync, lay out the steps, and deploy and run the pipeline for you.

Deploy and run it

Deploy the pipeline, then run it once to confirm it works end to end. To run it regularly, give it a schedule, from daily down to near real-time. Each run's status and logs are in the pipeline's run history.