Databricks to Google Sheets sync on a nightly schedule

PromptCreate

Every night on a cron schedule (default 2:00 AM), run a saved SQL query against my Databricks SQL warehouse and mirror the results into a Google Sheet, so my team always has a fresh, no-login copy of the numbers. This is a deterministic data sync with no judgement involved, so build it as a code workflow.

Use the Databricks Execute SQL Statement operation, passing my configured warehouse and SQL statement. Capture the returned statement_id, then poll the Databricks Get SQL Statement operation until status.state is SUCCEEDED. While the state is PENDING or RUNNING, wait and poll again; if it comes back FAILED, CANCELED, or CLOSED, stop the run and surface the error instead of writing partial data to the sheet.

Read the column names from the result schema (manifest) to build the header row. When the result set is large and split across chunks, page through it with the Databricks Get SQL Statement Result Chunk operation, following the chunk index and next-chunk link until every chunk has been fetched, and concatenate the rows in order so no data is dropped.

By default (snapshot mode), write the results into a specific tab using the Google Sheets Update Values operation so the target range is overwritten with the latest snapshot on every run. Write the column headers as the first row, then all data rows beneath them, into the configured spreadsheet, tab, and A1 range.

Also offer an append-only history mode as a configurable option: instead of overwriting, use the Google Sheets Append Values operation to add the latest rows after the last row of the tab so the team keeps a running history rather than only the newest snapshot. In append mode, only write the header row when the target tab is still empty.

Make the SQL query, the Databricks warehouse, the destination spreadsheet, the tab name, the A1 range, and the write mode (snapshot overwrite vs append history) all configurable inputs, so I can point the same workflow at different reports without editing it. The schedule should also be configurable so I can run it nightly, hourly, or only on business days.

What does this prompt do?

Runs your saved Databricks query automatically every night on a schedule you choose.
Copies the column headers and every row of results straight into a Google Sheet tab.
Overwrites the same range each run so the sheet always shows the latest snapshot, with an optional history mode that keeps every past run.
Lets non-technical teammates read the numbers in a familiar spreadsheet with no Databricks login required.

What do I need to use this?

A Databricks workspace with a SQL warehouse and the query you want to run.
A Google account with edit access to the destination spreadsheet.
The spreadsheet, the tab name, and the cell range where the results should land.

How can I customize it?

Change the schedule so it runs nightly, hourly, or only on weekday mornings.
Point it at a different query, warehouse, spreadsheet, tab, or range whenever your reporting needs change.
Switch between snapshot mode (overwrite the range each run) and history mode (append each run below the last).

FAQs

Do my teammates need a Databricks login to see the numbers?

No. The whole point is that the results live in a Google Sheet, so anyone with access to the sheet can read the latest numbers without ever logging in to Databricks.

Will it overwrite my sheet or keep a history?

You choose. Snapshot mode overwrites the same range every night so the sheet always shows the freshest data. History mode instead adds each run's rows below the previous ones so you build up a record over time.

How often does it run?

It runs on a schedule you set. The default is every night around 2 AM, but you can make it hourly, daily, or only on business days.

What happens if my query returns a lot of rows?

Large result sets are handled automatically. The workflow pulls every page of results from Databricks before writing them to the sheet, so nothing gets cut off.

Can I use this for more than one report?

Yes. The query, warehouse, spreadsheet, tab, and range are all configurable, so you can set up a separate copy for each report you want mirrored.

Related templates

Call overdue Xero customers with an AI collections agent

Every weekday at 10am, an AI voice agent phones your most overdue Xero accounts, logs what each customer promised, and reports back to finance in Slack.