Pipeloom Docs

Custom Components

Write reusable Python components and use them as pipeline nodes.

A custom component is a reusable, named piece of Python code that shows up in the node palette like any built-in node. Write it once, give it a set of parameters, and drop it into any pipeline — including generating one from a description with the AI Copilot.

A component's source code lives as a file in your workspace's file tree, alongside your pipelines and documents, so it gets the same folder organization and editor tabs everything else there does.

Writing a component

A component is plain Python. The runtime makes a small set of helpers available via conduit_runtime:

from conduit_runtime import env, secret, connection, set_output

def run(params):
    api_key = env("API_KEY")
    region = params["region"]

    # ... do the work ...

    set_output({"rows_processed": 42})
HelperWhat it does
param(key)Reads a value the caller configured for this node instance.
env(key)Reads a workspace or pipeline variable injected as an environment variable.
set_output(dict)Publishes structured output that downstream nodes can reference, the same way any other node's output can be referenced.
secret(name)Intended to read a named workspace secret.
connection(name)Intended to read connection details (host/port/user/password/database) for a configured warehouse connection.

Declare pip dependencies for your component alongside its source; they're installed into an isolated environment before your code runs.

Secrets and connections: current limitations

secret() and connection() exist in the runtime today, but nothing wires them to real values yet — calling either one currently returns None. If your component's logic depends on a secret or a connection to run correctly, it will not work as written today.

param() and env() are fully functional — parameters and workspace/pipeline variables both flow into a component correctly. If you need a component to authenticate against something external right now, pass the credential in as a workspace or pipeline secret variable and read it with env(), rather than relying on secret() or connection().

This is a known, tracked gap, not a bug you need to work around cleverly — secret() and connection() will be wired up in a future update. This page will be updated when they are.

On this page