Morning Grafana On-Call Briefing in Slack (Weekdays at 8am)

Every weekday at 8am ET, build a short morning on-call briefing in Slack that summarizes what actually misbehaved in Grafana over the last 24 hours. Trigger: cron, Monday through Friday at 8am America/New_York.

Step 1. Pull alert activity from Grafana. Use Grafana Find Annotations filtered to type=alert and time range = the last 24 hours. These annotations carry the rule name, the state transition (firing, resolved, normal, no_data), the severity if tagged, and timestamps. Pull the full set for the window.

Step 2. Analyze and rank. Group the annotations by alert rule. For each rule, count how many state transitions happened, what fraction were firings versus resolves, the longest sustained firing duration, and the severity. Classify each rule as flapping (many short fire/resolve cycles) or sustained (one or a few long firings). Rank rules by impact, weighting severity, total fire count, and sustained duration. Take the top three.

Step 3. Find a dashboard link for each top offender. For each of the top three rules, call Grafana Search Dashboards and Folders to find the dashboard most associated with that rule, matching on the rule name, the service or component in the rule name, or shared tags. Construct a deep link to that dashboard using the Grafana instance URL. If nothing reasonable matches, omit the link rather than guess.

Step 4. Decide whether to post at all. If the last 24 hours had no firings, or only trivial transient blips with no severity and no repeats, skip the briefing entirely. Do not post an empty digest. The goal is signal, not noise.

Step 5. Post one Slack message into the on-call channel using Slack Bot Send a Message. Ask the user at setup which channel to use (default the channel name should be configurable, for example #on-call or #incidents). The message should be short and skimmable in Slack mrkdwn, structured like this:

Header: *Overnight on-call briefing* with the date range covered.

Top three to watch today: a numbered list. For each item include the rule name in bold, the fire count, the current state (firing, resolved, flapping), the severity if known, and the dashboard deep link as <url|dashboard name>. Underneath each item add one short synthesized observation in plain English, for example: "auth-api p99 latency fired twelve times overnight, all clustered around the 2am deploy window" or "checkout-worker queue-depth has been sustained-firing since 3:14am and has not recovered".

Footer: a one-line summary of total alerts in the window and how many distinct rules fired.

Keep the whole message under roughly 1500 characters. Do not paste raw JSON, do not list every alert, do not include rules that did not fire. The prioritization, grouping, and natural-language synthesis are the entire point of this being an agent workflow, so do the thinking before posting.

This is distinct from a monthly alert-rule hygiene audit. This workflow does not change any Grafana configuration, does not file tickets, and does not mute or silence anything. It only reads and reports.

Additional information

What does this prompt do?

Runs every weekday morning and pulls the last 24 hours of alert activity from Grafana so the on-call shift starts informed.
Groups alerts by rule and severity, counts repeats, and flags which rules were flapping versus genuinely sustained.
Picks the top three things to watch today, writes a one-line observation about each, and links to the most relevant dashboard for fast triage.
Posts a single short message into your on-call Slack channel, and stays silent on quiet nights so the channel does not become noise.

What do I need to use this?

A Grafana account with permission to read alert annotations and search dashboards
A Slack workspace with a channel for your on-call team
The name of the Slack channel where the morning briefing should be posted

How can I customize it?

Change the schedule to match your shift handoff time, your timezone, or a different cadence such as daily including weekends.
Point the briefing at a different Slack channel, a specific incident channel, or a private channel for senior on-call only.
Adjust the lookback window, the number of top offenders shown, or the threshold for what counts as flapping versus sustained.

Frequently asked questions

What if nothing notable happened overnight?

The workflow stays quiet. If there is no meaningful alert activity in the last 24 hours, it skips posting entirely so your on-call channel does not fill up with empty digests.

How is this different from a Grafana alert notification?

Grafana pings you the moment something fires. This is the morning recap that tells the next on-call what already happened, what kept repeating, and what is worth checking first. It is a calmer, prioritized view rather than a live alarm.

Can I change which Slack channel it posts to?

Yes. Tell the workflow which channel to use during setup, and you can change it any time. Many teams point it at a dedicated on-call or incidents channel.

Does it work with Grafana Cloud and self-hosted Grafana?

Both. Connect with a Grafana service account token and your instance URL, whether that is a Grafana Cloud stack or your own self-hosted address.

How does it decide what to put in the top three?

It ranks by how many times each alert rule fired in the last 24 hours, weighted by severity and how sustained the firing was, then picks the three most worth a human look. You can adjust the ranking rules in plain language.