Weekly support QA review board for your Gladly team

Build me an app my support team leads open every week to review and coach agents on the conversations they handled in Gladly. Today this lives in a spreadsheet and we review almost nothing. I want a review board: a queue of recently handled conversations on the left, and a review pane where a lead reads the transcript beside our scorecard.

Important constraint on the data. Gladly has no organization wide conversation list, conversations are only reachable per customer, so do not try to build a global conversation feed. Seed the queue by walking recent tasks with List Tasks to collect the customers our team has been working, resolve each one with Get Customer for name and email, then load that customer's timeline with List Customer Conversations and pull the actual message history for each conversation with List Conversation Items. Also give me a second way to build a queue: let a reviewer paste a list of customer email addresses and resolve them with Find Customers. Pull the agent roster once with List Agents and use it for the handling agent filter and for mapping a conversation to the person who worked it.

The board needs a date range picker and a filter by handling agent, defaulting to the current review period. Each row shows the customer, the handling agent, when the conversation was handled, the channel, a short preview, and its review state, which is one of not reviewed, drafted, or confirmed. Above the queue put a coverage strip listing every agent from the roster with how many confirmed reviews they have in the current period, so a reviewer can see at a glance which agents have not been reviewed yet. Agents at zero should stand out.

Clicking a row opens the review pane. Left side is the full conversation transcript, in order, showing who said what and when, including notes. Right side is the scorecard: our rubric criteria, which start as tone, accuracy, resolution and next step clarity, each with a score and a comment box, plus an overall comment field. The rubric itself lives in app settings so a lead can add, rename, reweight or remove criteria and change the scoring scale without losing past reviews. Scores, comments, the reviewer's identity and the timestamp all persist in the app, keyed to the conversation and the agent, so we can see each agent's history and the team's trend over time.

Add a Score this batch button that fires a background agent over the conversations currently in view. The agent reads each sampled transcript, scores it against the current rubric, and for every criterion writes a short justification plus one quoted line from the conversation as evidence. It writes those back into the board as draft scores attached to the right conversation, so the reviewer opens a pre filled grid and is editing rather than starting from blank. Show progress while it runs, and make it safe to re run on a subset without wiping scores a human already confirmed.

Draft and confirmed have to be unmistakably different. Draft scores are visually distinct in both the queue and the scorecard, labelled as drafted by the assistant, and they never count toward an agent's score history, the coverage strip or the team trend. A review only counts once a reviewer confirms it. Confirming captures whatever the reviewer left in the fields, whether they accepted the draft, adjusted a number, or overrode it completely, and records who confirmed it and when. Let a reviewer discard a draft outright and score from scratch.

Once a review is confirmed, offer two coaching actions from the same pane. First, send the agent private feedback as a Slack direct message using Slack Bot Send a Message, resolving the agent's Slack user from their Gladly email with Look Up User by Email. Pre fill the message with the confirmed scores and the reviewer's comments, and let the reviewer edit it before it sends. Second, open a coaching follow up in Gladly with Create Task, assigned to that agent, with the rubric points to work on in the body and a link back to the review. Both are optional per review, and record on the review whether each was sent.

Finally, give me two read views built only from confirmed reviews: a per agent history showing their scores over time, criterion by criterion, with the reviewer comments attached, and a team trend showing average scores per criterion by period so we can see whether coaching is moving anything. Keep the sampling modest by default, a handful of conversations per agent per period, since Gladly is rate limited to roughly ten requests a second and fetching transcripts is chatty. Make the whole thing usable by a non technical team lead.