Release impact review board for post launch product checks

Build me a release impact review board where my product team runs the post launch check we currently do from memory. The point of the app is to judge whether a release worked and to record that judgement somewhere it survives until the next review.

The main view is a board of recent releases. Pull them from GitHub with List Releases for the repositories I configure, showing the release name, tag, and publish date. Alongside each release, show whether a marker already exists in Amplitude for it, read with List Chart Annotations and matched to the release by date and name. Make a missing marker visible on the row, because a release nobody annotated is usually a release nobody reviewed.

Sort the board so that releases which shipped in the last 14 days and have no verdict recorded yet sit at the top, in their own clearly labelled group. Those are exactly the ones the team forgets to look at. Everything else follows in reverse chronological order, showing its recorded verdict.

Give me a window toggle at the board level that switches between 7 and 14 days and applies to every comparison in the app. Seven days is the quick check right after launch, 14 is the fuller read a week later, and I want to flip between them without leaving the page.

Be careful about how much analytics work happens on first paint. Amplitude's dashboard queries are cost limited, roughly 360 concurrent and 1,080 per hour, with longer ranges costing more, so do not run every funnel and retention query for every release when the board loads. Fetch one cheap summary metric per release row, and defer the funnel and retention work until I open a release. Cache what has been fetched so flipping the window or reopening a release does not re-run everything.

Clicking a release opens a detail view, which is where the real work happens. For the selected window W of 7 or 14 days and a release that shipped on day D, the before period is the W days ending the day before D, and the after period is the W days starting at D, truncated at today if the release is too recent. Every metric must render as a before value, an after value, and the delta between them. Never show a single post launch number on its own, because without the baseline there is no way to attribute a change to the release.

The metrics in the detail view come from Amplitude. Use Get Active or New Users for the top line audience numbers, Get Event Segmentation Data for each of the key events I configure, Get Funnel Analysis Data for the activation funnel including its overall conversion rate and the per step drop off, and Get User Retention Analysis for retention after the release compared with retention before it. Run each of these once for the before period and once for the after period.

Below the metric table, show the events that moved most: rank my key events by the size of their change between the before and after periods, showing gainers and decliners separately so a regression is not buried under a list of things that went up. Mark clearly when the after window is still incomplete, so nobody reads a two day old release as a finished result.

In the detail view I record a verdict for the release: Working, Watching, or Regressed, along with free text notes. Persist the verdict and notes per release, with who recorded it and when, and show them on the board row and at the top of the detail view. This is the part that makes the ritual repeatable, so the next review starts where the last one ended rather than from scratch. Let me change a verdict later, since a Watching often becomes a Working or a Regressed once more data arrives.

From the detail view, if the release has no marker in Amplitude, let me backfill it with Create Chart Annotation and Create Release so the charts stay annotated for everyone looking at them, not just for me. Prefill the date and the release name from the GitHub release and let me edit before saving. Once it is saved, the board row should stop showing the release as missing a marker.

When I mark a release as Regressed, offer to file a Linear issue with Create Issue, prefilled with the release name in the title and a description listing the metrics that dropped with their before value, after value, and delta, plus my notes. Let me choose the team and edit the draft before it is submitted, and never create the issue automatically. After it is filed, show the link to the issue on the release so the review board points at the follow up work.

One implementation detail that will bite otherwise: Amplitude's dashboard queries take dates as YYYYMMDD with no separators, while annotations and releases use ISO 8601 or YYYY-MM-DD. These two formats must not be mixed. Build the date range once per period and convert at the point of each call.

Note for context: annotation backfill is a secondary action here, not the point of the app. There is a separate automation that marks releases on Amplitude charts as they are published, and this board is the review surface that comes after it, for the launches that already shipped.