Catch deployments that made your app slower or less reliable

Every hour, tell me whether anything we just shipped made the product worse. Find the deployments New Relic recorded recently, measure what happened to each application right after the release, and speak up only when a release clearly degraded things.

Start with List Applications in New Relic to get every APM application reporting to the account. For each application, call List Deployments and keep the deployment markers recorded in the last 2 hours. Each marker gives you the revision, the description or changelog, the user who recorded it, and its timestamp.

Only evaluate a marker once it is at least 30 minutes old and less than 90 minutes old. Anything younger than 30 minutes does not yet have a complete after-window, and anything older was already assessed on an earlier run. This band means every deployment gets judged exactly once, with a full 30 minutes of data on both sides, and never gets reported twice.

For each qualifying marker, use Run NerdGraph GraphQL Query to compare the 30 minutes after the deployment against the 30 minutes before it. One NRQL query per window returns all three numbers you need at once:

SELECT count(*), percentage(count(*), WHERE error IS true), percentile(duration, 95) FROM Transaction WHERE appName = '<application name>' SINCE '<window start>' UNTIL '<window end>'

Wrap it in NerdGraph as actor { account(id: <account id>) { nrql(query: "...") { results } } } and run it twice per deployment: once for the 30 minutes ending at the marker timestamp, once for the 30 minutes starting at it. Remember that NerdGraph reports query errors as a top-level errors array inside an HTTP 200 response, so check for that rather than trusting the status code.

Do not raise anything unless the change is both real and measurable. Check throughput first: skip the deployment entirely unless both windows saw at least 100 transactions, because a quiet service with three requests must never start a rollback conversation. Then treat it as a regression only when a metric clears both an absolute and a relative bar. For error rate, that is up by at least 1 percentage point and at least 50 percent relative. For 95th percentile response time, that is up by at least 200ms and at least 25 percent relative. Either metric clearing its pair of bars is enough to report.

When a deployment clears those bars, post to the engineering channel with Send a Message in Slack. Name the application, the revision, and who deployed it, then show the before and after numbers side by side so the reader can judge the size of the change without opening New Relic. Include the throughput in both windows so nobody has to wonder whether the sample was big enough to trust.

Then use Create Issue in Linear for the same regression so it is owned rather than merely announced. Title it with the application and revision, put the before and after numbers plus the deployment timestamp and the deployer in the description, and reference the Slack message. Set priority by severity, treating an error rate regression as more urgent than a latency one.

Stay completely silent when nothing regressed. No Slack message, no Linear issue, no all-clear summary. Most hours will produce no output at all, and that is the correct behaviour: the value of this workflow is that a notification from it always means something genuinely got worse.