Amazon SQS dead-letter queue workbench for on-call engineers

Build me a dead-letter queue workbench for Amazon SQS that my on-call engineers open whenever a queue starts filling up. The whole point is to read the actual failed messages instead of guessing from a count, then recover the right ones. This is a hands-on tool, not a metrics dashboard, so never reduce a queue to a chart when the payloads are what matter.

The landing view is a list of my dead-letter queues. Get every queue with List Queues, then for each one call List Dead Letter Source Queues. A queue that returns one or more source queues IS a dead-letter queue, and those results are the queues it protects, so use this instead of guessing from name suffixes. For each dead-letter queue call Get Queue Attributes for ApproximateNumberOfMessages, ApproximateNumberOfMessagesNotVisible, QueueArn and FifoQueue. Show each row as the dead-letter queue name, its depth, its in-flight count, the source queue or queues it protects, and a FIFO badge where relevant. Sort deepest first and let me click through to a queue.

The queue view is the heart of the app: a list of real messages. Load them with Receive Message using MaxNumberOfMessages of 10 and WaitTimeSeconds up to 20, requesting all message attributes and the system attributes SentTimestamp, ApproximateReceiveCount, MessageGroupId and MessageDeduplicationId. A Load more button fetches the next batch and appends to what is already on screen. Each row shows the body (pretty-printed when it parses as JSON, raw text otherwise), the message attributes, the sent time as both absolute and relative, and the receive count. Clicking a row opens a detail panel with the full payload and a copy button.

Bake in the browsing gotcha. Amazon's own docs warn that viewing a message counts as a delivery attempt, so reading a queue can itself cause damage. As soon as a batch has been read into the UI, immediately release those messages with Change Message Visibility set to a VisibilityTimeout of zero, using Change Message Visibility Batch to release a whole page in one call. The app must never sit holding messages invisible while an engineer reads them. Because the receive count still increments when we peek, label that column honestly as including views from this app rather than presenting it as a pure processing count.

Bake in the stale receipt handle gotcha. Every receive issues a new receipt handle and only the most recent one is valid, so handles captured on load go stale by the time I act. Before any replay, delete or archive, refetch the target messages with a fresh Receive Message, match them to my selection by MessageId, and act using the newly issued handles. If a message cannot be refetched because it is in flight elsewhere or already gone, mark that row as skipped with the reason and carry on with the rest instead of failing the whole operation.

Give me a text filter that searches across every loaded payload, covering the body and the message attributes, and make it clear it filters what has been loaded rather than the entire queue so nobody assumes an empty result means the queue is clean. Alongside it, group messages that share an error signature. Derive the signature from a configurable source: a named message attribute such as an error type or exception class when present, otherwise a normalized version of the body such as the first line with numbers, ids and timestamps masked. Show groups collapsed with a count and a representative payload, expandable to the individual messages, and let me select an entire group at once.

Selective replay is the differentiating feature, because Amazon's native redrive is all-or-nothing per dead-letter queue. Let me tick individual messages or whole groups and replay only those. For each one, Send Message to the source queue identified from List Dead Letter Source Queues, carrying the original body and message attributes across unchanged. When the destination is a FIFO queue the MessageGroupId must be preserved exactly as it was, with a fresh MessageDeduplicationId so the replay is not silently swallowed as a duplicate. Only after a send succeeds may that message be removed from the dead-letter queue with Delete Message using its fresh handle, so a failed send can never lose the payload. If a dead-letter queue protects more than one source queue, ask me which destination to use before sending.

For the all-at-once path, give me a redrive button that calls Start Message Move Task with the dead-letter queue ARN as the source and an optional MaxNumberOfMessagesPerSecond so I can throttle a replay into a fragile service. Poll List Message Move Tasks to show status, messages moved so far, and any failure reason, and surface a Cancel button wired to Cancel Message Move Task while a task is still running. Make it obvious in the UI that this moves everything, unlike selective replay.

Discarding must never be destructive. When I select messages and choose Discard, first archive each payload to Amazon S3 with Put Object, writing a JSON envelope containing the body, message attributes, system attributes, MessageId, MessageGroupId, the dead-letter queue, the source queue, who discarded it and when. Use a predictable key such as dlq-archive/<queue-name>/<year>/<month>/<day>/<message-id>.json. Only once every archive write has succeeded may Delete Message Batch run, and any message whose archive failed must be left in the queue untouched. Record each archived entry in the app's own storage so the archive is browsable without scanning the bucket.

Add a restore view listing archived messages with their queue, error signature and archive date, filterable by text. Selecting an entry pulls the payload back with Get Object and re-sends it to the source queue with Send Message, preserving MessageGroupId for FIFO destinations exactly as selective replay does. This is the undo path that makes discarding safe, so make it easy to find.

Surface partial failures everywhere. Batch calls in SQS return HTTP 200 while individual entries fail in a Failed array, each carrying Code, Id, Message and SenderFault. Never show a blanket success message after a batch operation. Every replay, discard and delete returns a per-message result table showing which succeeded and which failed with the reason, plus a Retry failed only button that acts on just the failures. The same applies to archive writes before a discard.

Keep an in-app action log of every replay, discard and redrive with the message ids, who did it and when, so the next engineer on shift can see what has already been attempted. Purge Queue may be included but only behind an explicit typed confirmation of the queue name, clearly marked as destructive and as the one path that does not archive anything first. Let me configure the archive bucket, the batch size, the error signature source and the default destination per dead-letter queue, and persist those settings between sessions.