# Search relevance workbench for your merchandising team

> Spot the searches losing you sales, test fixes against your live catalog, and queue up synonym, pin, and boost rules for engineering to apply.

- Workflow type: app
- Services: Elasticsearch, Slack Bot
- Categories: Marketing, Operations
- Published: 2026-08-21

## What it does

- Opens on three ranked backlogs side by side: your highest volume site searches, the ones that come back empty, and the busy ones nobody clicks.
- Sorts every list by search volume against how often that query returns nothing, so the biggest revenue leaks sit at the top of the page.
- Lets a merchandiser click any query, see the exact products a shopper would get, and try an alternate phrasing or an added synonym side by side before committing to anything.
- Saves each agreed fix as a reviewed rule (synonym, pin, or boost) with the target products, who approved it, and the date, then posts the week's fixes to your merchandising Slack channel.

## What you'll need

- An Elasticsearch cluster holding your product catalog, and an API key the app can connect with.
- A record of your site searches in that cluster, with the query text, how many results came back, and whether the shopper clicked. This is the standard setup for site search analytics; if nobody is logging searches yet, that has to come first.
- A place to keep the saved fixes. A fresh, empty index works, and the app will fill it.
- A Slack workspace and the channel your merchandising team already works in.

## Prompt

Build me a search quality workbench that our merchandising team opens every week to fix what our site search is getting wrong. Right now only engineers can touch relevance, so nothing ever gets fixed. This app should give merchandisers a surface where they can see what search is failing at, test a fix against the live catalog themselves, and hand engineering a reviewed queue of proposed rules.

Everything reads from Elasticsearch. Let me configure three index names in the app: a search-events index (one document per site search, with the query string, the result count, and the click count), our live product index, and a curation-rules index where the app writes the proposed fixes.

The main view is a backlog board with three ranked lists side by side, all built with the Search Documents (Query Index) operation, using aggregations in the query body over the search-events index for a configurable window that defaults to the last seven days: top queries by volume, queries that returned zero results, and high-volume queries that got zero clicks. Each row shows the query text, its volume, a trend indicator versus the prior window, and a reviewed or not-reviewed badge. Rank every list by search volume multiplied by zero-result rate so the biggest revenue leaks sit at the top.

Clicking any row opens a test panel. It runs that exact query against the live product index with Search Documents and shows the top ten products a shopper would actually see: image, title, price, and relevance score. Next to it sits a 'try a variation' box where the merchandiser types an alternate phrasing or adds a synonym. Running it fires a second Search Documents call and renders both result sets side by side, so they can see exactly what changed: which products entered, which dropped out, and how the order moved. This test-then-fix loop is the heart of the app, so give it room.

When they settle on a fix, a save-rule form writes a curation-rule document into the curation-rules index with the Index (Create or Replace) Document operation, capturing the original query, the rule type (synonym, pin, or boost), the variation or synonym text, the proposed target product IDs picked straight from the test results, who approved it (the signed-in user), and the date. Saving a batch of rules at once should use Bulk Index / Update / Delete. A Rules tab lists everything saved, filterable by type and status, so engineering has a reviewed queue to work from.

Keep a reviewed state per query so the team works the list down each week instead of re-treading the same rows. Marking a row reviewed writes a small review-state document keyed by the query string (query, reviewed by, reviewed at, optional note) into the same curation-rules index under a distinct document type, and the backlog reads those back on load so reviewed rows are dimmed or filtered out. The reviewed state is shared across the whole team, not per user.

A 'Post this week's fixes' button at the top of the board sends a summary to our merchandising channel using the Slack Bot Send a Message operation: how many queries were reviewed this week, the rules saved grouped by type, and the top three unresolved zero-result queries still sitting at the top of the backlog. Let me pick the channel.

Keep it dense and fast. A merchandiser should be able to open a row, test two variations, save a rule, mark it reviewed, and move to the next one without ever leaving the board.

## How to customize

- Change how the backlog is ranked: pure volume, revenue at risk, or filtered to one category, instead of the default volume against empty-result rate.
- Adjust the review window (seven days by default) and how many products the test panel previews.
- Point the weekly summary at a different channel, or add an owner and category to the fix form so each rule routes to the right person.

## FAQ

### Do we need an engineer to use this?

No. Merchandisers test and propose here on their own. The app only writes proposed fixes to your own list of rules, it never changes what shoppers see. Engineering picks up the reviewed queue and applies it.

### Will testing a query change live search results?

No. The test panel runs the same read-only search a shopper would run against your catalog. Nothing takes effect until engineering ships the rule.

### What if we are not recording our site searches anywhere?

You will need that first. All three backlogs are built from a record of each search: the words typed, how many products came back, and whether anyone clicked. Most site search setups can log this, and it is the standard starting point for search analytics.

### Can several people work the list at the same time?

Yes. The reviewed state is saved per query and shared across the team, so once someone marks a row reviewed it drops out of the list and nobody re-treads it the following week.

### Does this work with hosted or self-managed Elasticsearch?

Either one, as long as the app can reach your cluster and you have an API key with read access to the catalog and write access to the index holding the saved rules.

Use this prompt in General Input: https://www.generalinput.com/prompts/search-relevance-workbench-for-your-merchandising-team