Page on-call when a production deploy fails in Buildkite
The moment a build fails on one of your production deploy pipelines, automatically open a PagerDuty incident so on-call gets paged right away.
When a build finishes on one of my Buildkite pipelines, run this workflow automatically. Trigger on Buildkite's build.finished webhook, and only continue when the build's state is failed and its pipeline is on my allowlist of production deploy and release pipelines. Any other pipeline, or any non-failed state, should stop the workflow immediately so routine feature-branch failures never page anyone. No AI reasoning is needed: this is a fixed, deterministic mapping from the failed-build payload to a PagerDuty incident.
First, read the full build using Buildkite's Get a Build operation for the pipeline slug and build number from the webhook payload. This gives me the build number, branch, commit hash, commit author, build message, and the web URL of the build.
Then create a PagerDuty incident using PagerDuty's Create Incident operation on my deployment service. Set the incident title to name the pipeline, the branch, and the build number, for example 'Deploy failed: <pipeline> on <branch> (build #<number>)'. Set the incident body to include the commit hash, the commit author, and a direct link to the failed build in Buildkite so on-call can jump straight to it.
Derive the incident urgency from the pipeline: release and production deploy pipelines should page at high urgency, while lower-priority deploy pipelines can page at low urgency. Keep the mapping from pipeline name to urgency explicit so it is easy to adjust. Because only allowlisted pipelines reach this step, on-call is only paged for the deployments that actually matter.
Additional information
What does this prompt do?
- Watches your production deploy and release pipelines and reacts the instant a build fails.
- Opens a PagerDuty incident on your deployment service so the on-call engineer is paged.
- Fills the incident with the pipeline, branch, build number, commit, author, and a direct link to the failed build.
- Ignores routine feature-branch failures so on-call is only paged for the pipelines that matter.
What do I need to use this?
- A Buildkite account with the deploy and release pipelines you want to monitor.
- A PagerDuty account with a deployment service and an on-call schedule set up.
- The list of pipeline names that are allowed to page on-call, so feature branches never do.
How can I customize it?
- Choose exactly which pipelines are allowed to page on-call.
- Set the urgency (high or low) per pipeline so release builds can page louder than others.
- Adjust what goes into the incident title and body, such as the branch, commit, and build link.
Frequently asked questions
Will this page on-call for every failed build?
Does this use AI to decide when to page?
Can I control how urgent the incident is?
What information ends up in the PagerDuty incident?
Do I need to write any code to set this up?
Related templates
Stop missing failed production deploys.
Page the right on-call engineer automatically the moment a deploy pipeline goes red in Buildkite.