Monitors

Built-in checks on every connection, your own checks on any saved query, and the anomalies they raise.

A monitor checks something on a schedule and raises an anomaly when it looks wrong. There are two kinds.

Built-in monitors

These are on for every connection from the start, using Pipeloom's own data. They cost nothing.

Monitor	Alerts when
Volume	A stream loads far more or far fewer rows than usual, or none when it normally loads some.
Freshness	A scheduled connection hasn't loaded data for much longer than its usual gap between syncs.
Failures	A connection fails twice in a row, or more than three times in a day.
Rejected records	A sync rejects records when it usually rejects none, or many more than usual.
Duration	A sync takes much longer than usual.
Schema changes	A source's schema changes.

Open a built-in monitor to set its sensitivity (low, medium or high), where its alerts go, or to turn it off. You can also set these for one connection only, so it differs from the rest.

Custom monitors

A custom monitor runs one of your saved queries and looks at one column of the first row. It alerts when the value:

is greater than, or less than, a number;
changes by more than a percentage since the last run; or
is unusual compared with its own history, the same way the Volume monitor works.

Pick how often it runs: every 5 or 15 minutes, every hour, every 6 hours or once a day. Before you save, a chart shows the last 30 days with the usual range and where it would have alerted.

This is how you watch your own warehouse: "orders with no customer", or "revenue today compared with usual".

Where alerts go

Each monitor can tell any of these when it starts alerting:

email addresses,
a Slack channel, through an incoming webhook,
any other webhook.

Starting a pipeline

A monitor can also start a pipeline when it starts alerting, for example a re-sync or a backfill. Pick it under Then start a pipeline. Only deployed pipelines of the same environment are listed.

It starts once each time the monitor turns to Alerting, not on every check while it stays there. When one check of a built-in monitor finds several anomalies, they share one run.
For a built-in monitor, the pipeline is set for the whole environment, not for one connection.
Choosing or changing the pipeline needs the right to start pipelines (Start and cancel runs in Settings → Access). Anyone who can see the monitor can see which pipeline it starts.
The anomaly links to the run it started. If the pipeline didn't start, for example because it isn't deployed any more, the anomaly says why.
A run started this way uses credits like any other run.

How "usual" is worked out

Pipeloom keeps this simple, so you can check it yourself:

The baseline is the last 28 days of the same measure. Once there are enough points, it compares with the same weekday only.
The usual range is the median, plus or minus a multiple of the median absolute deviation. Low sensitivity allows the widest range and high the narrowest.
Small changes on small streams don't alert.
A stream that normally loads rows and loads none always counts.
With fewer than 8 points of history the monitor shows Learning and doesn't alert yet.

Every alert says the value, the usual range and how the range was worked out.

Monitor states

State	Meaning
OK	Checked, and nothing is wrong.
Alerting	Something is wrong. An anomaly is open.
Learning	Not enough history to know what is usual.
Muted	Alerts are paused for a while.
No data	The query returned nothing to check.
Paused	Stopped by Pipeloom, for example when your organization is out of credits. The reason is shown.
Error	The query failed. The error is shown.
Off	Turned off.

Anomalies

Insights → Anomalies lists what the monitors found, with a chart around the point. Open anomalies also show as a count on the Insights rail entry. For each anomaly you can:

Acknowledge it, to say someone is on it;
mark it Not an anomaly, so that point no longer counts towards what is usual; or
Mute the monitor for an hour, 4 hours, a day or a week.