Daily backup compliance check for ClickHouse Cloud

Every morning at 7am, check backup health across all of my ClickHouse Cloud services and flag any that are out of compliance with their own backup schedule. Use a cron trigger on a daily timer. The ClickHouse Cloud management API does not send webhooks and is not a poll provider, so a scheduled run is the right way to do this.

First, call List Services to get every ClickHouse Cloud service in my organization. Then, for each service, pull its backups with List Service Backups, which returns them most recent first with a status on each one, and read its schedule with Get Service Backup Configuration to learn the configured backup interval and retention for that service.

For each service, decide whether it is out of compliance. Treat a service as non-compliant when the most recent successful backup is older than that service's configured backup interval, when there is no backup at all, or when the most recent backup is in a failed state. Judge staleness relative to each service's own configured interval, never a fixed number of hours, because each service can have a different schedule and retention. Identify the newest backup whose status indicates success, treat that as the last good backup, and compute its age from its timestamp.

Post a single summary to the #data-ops Slack channel using the Slack Bot Send a Message action. List every service with its compliance status, the age of its last good backup, and its configured schedule, and call out the non-compliant services clearly at the top. Write it as a readable Slack message using Slack formatting, not raw JSON.

For every non-compliant service, open a Linear issue with Create Issue that names the service, the age of its last good backup, the configured backup schedule, and what is wrong (stale, missing, or failed). Before creating anything, dedupe by a stable per-service title such as 'ClickHouse backup out of compliance: <service name>'. Use Search Issues to look for an existing open issue with that title, and only create a new one if none exists, so a persistent problem does not spawn a fresh ticket every day. If a matching open issue already exists, leave the existing ticket in place instead of creating a duplicate.