Datadog alert noise cleanup board for monthly monitor review

Build an app my SRE team works out of during our monthly monitor review. It is about monitor hygiene, not about the alerts firing right now. The question it answers is which monitors generate too much noise, which have no clear owner, and which should be silenced, retagged, or retired. This is deliberately not a real time alert feed, so do not build a live firing alerts view.

The main view is a single table listing every monitor ranked by how many times it alerted over the last 30 days, highest first. Build the monitor inventory using Datadog "Search Monitors" and "List Monitors", then get the alert counts by querying "Search events" for that monitor's alert events over the trailing 30 day window and counting them per monitor. Columns: monitor name, alert count over the last 30 days, owning team, notify target, current monitor state, whether a downtime is already active, and the linked Jira cleanup ticket key once one exists.

For the downtime column, fetch "Get all downtimes" once when the table loads and match downtimes to monitors in memory rather than making a call per row. Use "Get active downtimes for a monitor" to refresh a single row after the user schedules or cancels a downtime.

Ownership comes from monitor tags, for example team:payments or owner:platform. Datadog has no team or ownership lookup available here, so tags are the only source of truth. Any monitor without a team style tag should display as Unowned rather than being hidden.

Three filter chips sit above the table, one per hygiene problem worth fixing. First, chronic flappers: monitors whose alert count over the window exceeds a firing threshold I can set, defaulting to around 20 and remembered between visits. Second, monitors with no team tag at all. Third, monitors that notify a named individual, meaning the notify target looks like a personal email or user handle rather than a team alias or shared channel. The chips should combine with each other and with a free text search over monitor names.

Clicking a row opens a detail panel for that monitor. It shows the full configuration from "Get a Monitor", including the query, thresholds, evaluation window, notification message, and tags, alongside that monitor's recent alert history pulled from "Search events" so I can see the pattern of when it fired.

From the detail panel I can act on the monitor. Schedule a downtime to silence it while it gets fixed, using "Schedule a downtime" with a duration picker. Cancel a downtime that is no longer needed with "Cancel a downtime". Retag it with the right owning team or adjust its notification settings using "Update a Monitor", sending only the changed fields. Delete a stale monitor with "Delete a Monitor", behind an explicit confirmation step that makes me confirm the monitor name before anything is removed. Open a Jira cleanup ticket with "Create Issue" for a monitor that needs a real threshold fix, prefilling the summary and description with the monitor name, its id, the 30 day alert count, the owning team tag, and its current thresholds.

Before creating any ticket, check Jira with "Search Issues (JQL)" for an existing open cleanup ticket that references the same monitor id in the configured project. If one already exists, show me that ticket key and a link to it instead of filing a duplicate. When the table loads, resolve linked tickets for all listed monitors with a single JQL query so the ticket key can render on each row without a call per monitor.

Persist my settings between sessions: the flapper threshold, the Jira project key that cleanup tickets are filed into, and the length of the review window.