Airbyte backfill console with a Google Sheets audit trail
Run Airbyte resyncs and clears from one screen, with a typed reason, a confirmation step on destructive runs, and every launch logged to a sheet.
I want an internal app my data team can open to run Airbyte backfills safely, instead of clicking through the Airbyte UI where a wrong click is unrecoverable and nothing records why the backfill happened.
The main screen lists my Airbyte connections using List Connections, showing each connection's name, source, destination, and status. When I select one, load its full detail with Get Connection Details, and show the streams it covers using Get Stream Properties so I can see exactly what data a clear would affect. Also show that connection's recent runs using List Jobs, with status, job type, start time, and duration, and let me expand any run to pull its full detail with Get Job Details.
Below that, a launch panel. It has a required free text Reason field, and a choice of two modes: Resync, which starts a normal sync, and Clear and resync, which erases the data Airbyte wrote to the destination and then rebuilds it. Default the selection to Resync. I should not be able to launch anything without typing a reason.
Clear and resync is destructive and needs to feel that way. Airbyte's own documentation says a clear erases all of the data Airbyte has created in the destination, that there is no way to recover from a clear, and that if the source does not retain all of its records this may lead to data loss. Put that warning on screen, in plain language, whenever the clear option is selected. Then require me to type the connection's exact name into a confirmation box before the launch button becomes active. Use the word Clear rather than Reset throughout the interface, since Airbyte renamed the operation in version 1.0.0.
Launching calls Trigger Sync or Reset Job. For a plain resync that is a single sync job. For Clear and resync, the app runs the clear job first, waits for it to succeed, and then automatically starts a sync job, because Airbyte no longer re-syncs automatically after a clear and historical data is only backfilled once a sync runs after the clear succeeds. Present that chain as one operation with two stages rather than two separate things I have to babysit, and if the clear fails, stop and say so instead of starting the sync.
While a job runs, poll Get Job Details and show live progress in the app: current stage, status, rows synced, and elapsed time. Show a Cancel button that calls Cancel a Job on the running job, and reflect the cancelled state once it takes effect.
Every launch appends one row to a Google Sheet using Append Values, capturing the timestamp, who launched it, the connection name and id, whether it was a resync or a clear and resync, the reason typed, and the resulting job id. Write the row at launch time so nothing is lost if the job later fails or is cancelled. On the main screen, read the last five rows back with Get Values and display them as a recent backfill activity list, so whoever is about to run a backfill can see what the team already did.
Two constraints. The app is read only with respect to configuration: there is no supported operation to update an Airbyte connection, so it must never try to change stream selections, schedules, or any other connection settings. Get Stream Properties is there purely as context for the decision. And this audit sheet is only for human initiated backfills, so keep it in its own sheet and do not turn it into a general export of every Airbyte job.
What does this prompt do?
- Pick any of your Airbyte connections and see what it syncs, plus how its recent runs went, all on one screen.
- Choose a plain resync or a clear followed by a resync, with the safe option selected by default and live progress while it runs.
- Because clearing permanently erases what Airbyte wrote to the destination, the app shows a warning and makes you type the connection name to confirm.
- Every launch is appended to a Google Sheet with who ran it, what they ran, and why, and the last five entries stay visible on screen.
What do I need to use this?
- An Airbyte Cloud account with connections already set up
- Permission to run syncs on the connections you want to backfill
- A Google account and a spreadsheet you can write to, ideally a fresh tab kept just for backfill history
How can I customize it?
- Change how many recent runs and past backfill entries appear on screen
- Add extra columns to the audit sheet, such as the team or the ticket number behind the request
- Narrow the connection list to the pipelines one team owns, so people only see what they are responsible for
FAQs
What is the difference between a resync and a clear?
Can I undo a clear?
Why does it ask for a reason before running anything?
Can I change which streams a connection syncs from this app?
What happens if I close the app while a backfill is running?
Related templates
Stop buying stale lists. Reps run a saved search, work the results like an inbox, and only the accounts they approve ever reach your CRM.
Drag creators through Sourced to Wrapped on a board grouped by campaign, with audience stats on every card and a one-click brief for each creator.
Pick a repricing rule, send an assistant out to check competitor pages, then approve the new prices that clear your margin floor.
See the projected bounce rate for any outbound send before it goes out, and keep the launch button locked until the list is clean enough to be safe.
Pick any date range and see what each device in your home actually cost to run, not just how many kilowatt-hours it used.
Search three months of global coverage on your topic, rank the outlets actually writing about it, and draft a tailored pitch for each one.
Stop running risky backfills from memory.
Give your team one screen for Airbyte clears and resyncs, with a reason attached to every single run.