Alert Slack when a ClickHouse ingestion pipe stalls

Every 30 minutes, check the health of my ClickHouse Cloud ClickPipes ingestion pipelines and alert my team in Slack whenever a pipe is not running. Run this on a cron schedule, and keep the 30-minute cadence configurable so I can make it more or less frequent.

First, use ClickHouse Cloud's List Services operation to get every service (ClickHouse Cloud instance) in my organization. Discover my ClickHouse Cloud organization first if the service list requires an organization id. Then, for each service, use List ClickPipes to get that service's managed ingestion pipelines.

For every pipe, read its state field and flag any pipe whose state is not running, for example failed, stopped, or paused. ClickPipes stops itself after about 15 minutes without a source connection or about 1 hour without a reachable destination, which leaves the pipe in a non-running state that List ClickPipes exposes. That state field is exactly what this workflow filters on.

For each flagged pipe, post an alert to the #data-ingestion Slack channel using Slack Bot's Send a Message operation. Each alert must include the pipe's name, the service it belongs to, and its current state. Keep the target Slack channel configurable so alerts can be pointed at any channel my team watches.

De-duplicate repeat alerts. Only alert once for a pipe while it stays in the same non-running state, so a pipe that stays down for hours does not repost to the channel every 30 minutes. Keep a record of the last state I alerted on per pipe, and send a fresh alert only when a pipe's state changes, for example if it recovers to running and then fails again.

This is a deterministic check: every step is a straightforward list-and-filter over the pipe state field with no judgement required. If every pipe is running, do nothing and post nothing.