Deploy maintenance window console for Grafana alerts
Open a maintenance window before every deploy, silence exactly the right Grafana alerts, and get them back on automatically.
Build me an app my team opens before every deploy or planned maintenance, so we silence exactly the right Grafana alerts instead of muting everything by hand and forgetting to unmute. The people using it are engineers about to ship, plus whoever is on call and wants to know what is currently quiet.
The main view is a single console with three panels side by side. The first panel is what is currently suppressed, read from Grafana with List Mute Timings, split into windows active right now and windows scheduled to start later, each showing its time range, when it ends, the service it covers, and who opened it. The second panel is the alert rules from List Alert Rules grouped by folder, with each rule marked quiet or live depending on whether a currently active window covers it. The third panel is the notification targets from List Contact Points. Tie the three together so each rule answers the question 'where would this page if it fired right now': read the routing tree with Get Notification Policy Tree and match each rule's labels against the routes to resolve the contact point it would land on, then show that contact point next to the rule. Let me filter the whole view by folder, by service, and by quiet-versus-live.
To open a window I pick a service or a folder, set a time range, and type a short reason. On confirm the app suppresses the alerts with Create Mute Timing using a predictable name that encodes the service and the date so windows are easy to spot and delete later, drops a marker on each affected dashboard with Create Annotation covering the same time range and carrying the reason plus a shared tag so the app can find its own markers later, and files a change record in Linear with Create Issue containing the service, the time range, the reason, who opened it, and the list of rules that went quiet. Store the window record in the app, including the mute timing name, the annotation ids, and the Linear issue url, so the console can link everything back together.
One thing to get right: in Grafana a mute timing only takes effect if a notification policy route actually references it. After creating a window, check the routing tree with Get Notification Policy Tree and show me plainly whether the window is really in effect or whether it was created but is not attached to any route. If it is not attached, surface that as a warning in the console rather than pretending the alerts are quiet, and offer an explicit confirm action that attaches the new window to the routes I select, leaving the rest of the tree untouched.
Each active window has an End early button that removes it with Delete Mute Timing and updates the record so the history shows the real end time rather than the planned one. A history view lists past windows using Find Annotations filtered to the app's tag, joined with the stored records, showing service, planned versus actual duration, reason, who opened it, and a link to the Linear change record. Let me search that history by service and by date range.
Add a Prepare window button that runs a background agent. It takes the service name I typed and works out the scope for me: it finds the matching dashboards and folders with Search Dashboards and Folders, pulls the rules with List Alert Rules, and resolves where those rules would page using Get Notification Policy Tree and List Contact Points. It writes its proposal back into the app as a pre-filled window: the folders and rules it wants to suppress, the dashboards it will annotate, a suggested time range, and a draft reason. Critically, it also writes a short note listing anything it deliberately left live and why, for example customer-facing availability rules or alerts that belong to a different service that happens to share a name. The agent never opens the window itself. I review the proposal, edit the selection, and confirm.
A few rules to bake in. Windows always carry a fixed end time, with a configurable default length and a hard maximum so nobody can accidentally silence something for a week. Record the person who opened each window and show a 'my windows' filter alongside the full list. Never suppress anything without an explicit confirm, and always show the exact list of rules that will go quiet before I confirm. If the reason field is empty, block the window, since the whole point is that the annotation and the change record explain the gap later.
What does this prompt do?
- One screen shows every alert that is currently silenced, every alert still live grouped by folder, and where each one would page if it fired right now.
- Pick a service and a time range to open a window: the right alerts go quiet, the affected dashboards get a marker explaining the gap, and a change record is filed in Linear.
- A Prepare window button hands the scoping to a background agent that finds the dashboards and rules matching the service you name, then drafts the window for you to review, including a note on anything it left live on purpose.
- End a window early with one click, and keep a searchable history of past windows so nobody has to guess why the graphs went quiet last Tuesday.
What do I need to use this?
- A Grafana account, cloud or self-hosted, with permission to manage alert rules, notification settings, and annotations
- A Linear workspace where change records for each window should be filed
- Your dashboards and alert rules organized into folders, since the app groups everything by folder
How can I customize it?
- Change the naming and tagging convention for windows so they match how your team labels deploys and services.
- Choose which Linear team the change records land in, and who gets assigned by default.
- Set a default window length and a hard maximum, so nobody silences an alert for a week by accident.
FAQs
Does this actually stop alerts from firing, or does it just hide them?
What happens if we forget to close a window?
Can we find out why a dashboard went quiet three weeks ago?
Do we need PagerDuty or another on-call tool for this to work?
Will the Prepare window button silence things without asking?
Can two people open overlapping windows for different services?
Related templates
Open one board each morning, see which voice calls went badly, replay the exact moment the caller got frustrated, and file the fix.
See every customer waiting on a bug fix, file the ticket without creating a duplicate, and tell them the moment engineering ships it.
See every recent deploy across all your Heroku apps in one timeline, then roll back, restart, or file an incident without leaving the screen.
Pick a date range, see which support topics are climbing fastest, read the real customer threads behind them, and file the worst ones as tickets.
See every Hacker News story and comment that mentions your product or your competitors, mark each one new, needs reply or handled, and stop losing threads.
One grid of every table you track, when it last landed, and what broke, with stale tables pinned to the top and an agent that explains why.
Stop muting alerts by hand before every deploy.
Open a window, silence exactly what belongs to the service you are shipping, and let it turn itself back on when the deploy is done.