ClickHouse ClickPipes health monitoring with Slack alerts

PromptCreate

Every 30 minutes, check the health of my ClickHouse Cloud ClickPipes ingestion pipelines and alert my team in Slack whenever a pipe is not running. Run this on a cron schedule, and keep the 30-minute cadence configurable so I can make it more or less frequent.

First, use ClickHouse Cloud's List Services operation to get every service (ClickHouse Cloud instance) in my organization. Discover my ClickHouse Cloud organization first if the service list requires an organization id. Then, for each service, use List ClickPipes to get that service's managed ingestion pipelines.

For every pipe, read its state field and flag any pipe whose state is not running, for example failed, stopped, or paused. ClickPipes stops itself after about 15 minutes without a source connection or about 1 hour without a reachable destination, which leaves the pipe in a non-running state that List ClickPipes exposes. That state field is exactly what this workflow filters on.

For each flagged pipe, post an alert to the #data-ingestion Slack channel using Slack Bot's Send a Message operation. Each alert must include the pipe's name, the service it belongs to, and its current state. Keep the target Slack channel configurable so alerts can be pointed at any channel my team watches.

De-duplicate repeat alerts. Only alert once for a pipe while it stays in the same non-running state, so a pipe that stays down for hours does not repost to the channel every 30 minutes. Keep a record of the last state I alerted on per pipe, and send a fresh alert only when a pipe's state changes, for example if it recovers to running and then fails again.

This is a deterministic check: every step is a straightforward list-and-filter over the pipe state field with no judgement required. If every pipe is running, do nothing and post nothing.

What does this prompt do?

Checks all of your ClickHouse Cloud ingestion pipelines every 30 minutes.
Flags any pipeline that is not running, such as ones that have failed, stopped, or paused.
Posts a Slack alert naming the pipeline, its service, and its current state.
Skips repeat alerts for a pipeline that stays down, so your channel only pings when something actually changes.

What do I need to use this?

A ClickHouse Cloud account with ingestion pipelines (ClickPipes).
A Slack workspace and the channel you want alerts to land in, for example #data-ingestion.

How can I customize it?

Change how often the check runs, for example every 15 minutes or once an hour.
Point alerts at any Slack channel your team watches.
Adjust which pipeline states count as a problem.

FAQs

Why would a ClickHouse pipeline stop on its own?

ClickHouse pauses a pipeline after about 15 minutes without a source connection or about an hour without a reachable destination. When that happens the pipeline stops moving data, and this workflow tells you right away instead of letting the gap go unnoticed.

Will this spam my Slack channel if a pipeline stays down?

No. Once a pipeline is flagged, it will not be re-alerted while it stays in the same state. You only get a fresh alert when the situation changes, such as a pipeline recovering and then failing again.

Can I send alerts somewhere other than #data-ingestion?

Yes. You can point the alerts at any Slack channel your team watches, and you can change it at any time.

How quickly will I hear about a problem?

By default the check runs every 30 minutes, so you learn about a stalled pipeline within half an hour. You can set it to run more often if you need faster alerts.

Does this replace ClickHouse's built-in notifications?

It complements them. ClickHouse can notify you inside its own console, and this pushes the same signal into the Slack channel your team already lives in, on your own schedule.