# Shut down idle Databricks compute from one live console

> See every cluster and SQL warehouse running right now, what it is costing you, and stop the idle ones without leaving the screen.

- Workflow type: app
- Services: Databricks, Slack Bot
- Categories: Finance, Operations
- Published: 2026-08-17

## What it does

- Lists every all-purpose cluster and SQL warehouse running right now, with how long it has been up, how long it has sat idle, and when it is set to switch itself off
- Flags anything running with automatic shutoff turned off or set longer than your threshold, so the money pits stand out immediately
- Ranks the last 30 days of spend by cluster, job, owner and tag, and every row links straight back to the compute that caused it
- Lets you stop, restart, or tighten the shutoff window on any item behind a confirmation that names exactly what is about to stop
- Posts a summary of what you shut down and the rough saving to a Slack channel after a cleanup pass

## What you'll need

- A Databricks workspace and a login with permission to manage clusters and SQL warehouses
- A SQL warehouse the app can use to run the spend query
- Billing system tables enabled in your workspace for the spend breakdown. Without them the live compute list and all the controls still work
- A Slack workspace if you want to post the cleanup summary

## Prompt

Build me a Databricks compute cost console that our platform lead opens to find and shut down compute that is burning money right now, rather than reading about it in a report a week later. It is one screen with two stacked sections: live running compute at the top, spend attribution below it. Keep the whole thing focused on acting now on what is running. This is not a scheduled recap.

The top section is the live compute table. Load every all-purpose cluster with List Clusters and every SQL warehouse with List SQL Warehouses, then filter to the ones actually consuming money, meaning clusters in RUNNING, PENDING, RESIZING or RESTARTING and warehouses in RUNNING or STARTING. Note that List Clusters also returns pinned clusters and anything terminated in the last 30 days, so filter on state rather than assuming everything returned is live. Show one row per item with name, type (all-purpose cluster or SQL warehouse), current state, creator or owner, size (node type and worker count for clusters, cluster size and scaling range for warehouses), uptime computed from the cluster start time or the warehouse state timestamp, idle time, and the configured shutoff window, which is autotermination_minutes for clusters and auto_stop_mins for warehouses. Timestamps from this API are Unix epoch milliseconds.

Derive idle time for clusters from List Cluster Events. For each running cluster pull its recent events and find the most recent one that indicates real work, such as the last resize, autoscaling stats report, or run start, then show the elapsed time since then as idle time. If nothing has happened since the cluster started, idle time equals uptime and that is the strongest signal on the page. Fetch these events lazily or in small batches and cache them per refresh, because the Clusters API caps concurrent and per second calls and will return 429 with REQUEST_LIMIT_EXCEEDED if you hammer it. If events are unavailable for a cluster, show idle time as unknown rather than blocking the row.

Flag rows loudly. Anything running with automatic shutoff disabled (0 or unset) gets the strongest warning, and anything with a window longer than my threshold gets a softer one. Make the threshold a setting I can change, defaulting to 60 minutes. For context when you write the copy: cluster auto-termination accepts 10 to 10,000 minutes, and Pro and classic SQL warehouses default to a 45 minute auto stop with a minimum of 10. Sort the table so the worst offenders sit at the top, roughly by idle time multiplied by size, so a large idle cluster outranks a small one. Show a compact banner above the table with the count of running items and how many are flagged.

The section below is the spend breakdown, built by querying the Databricks system billing tables with Execute SQL Statement on a warehouse I pick in settings. Query system.billing.usage over the last 30 days and, where it is available, join system.billing.list_prices on sku_name and the price validity window so you can show estimated dollars rather than raw DBUs. Group the results four ways behind tabs or a selector: by cluster, by job, by owner, and by tag, reading the identifiers out of the usage_metadata struct (cluster_id, job_id, warehouse_id), the owner out of identity_metadata (run_as or owned_by), and tags out of custom_tags. Rank each grouping by cost descending and show the top drivers with their 30 day total and a small trend. Verify the exact column names against the workspace rather than trusting this description, since these tables evolve.

Every cost row must click through to the compute it came from. When a row carries a cluster_id or warehouse_id that matches something in the live table above, clicking it should scroll to and highlight that row so I can see what it is doing right now and act on it in the same motion. When the compute no longer exists or is already stopped, say so on the row instead of linking nowhere.

The spend section has to degrade gracefully. Plenty of workspaces do not have system tables enabled, and the query will simply fail. When it does, catch it and show a calm explanatory notice in that section saying the billing tables are not reachable and naming the likely reason, while leaving the live compute list and every control fully working. The app must never be useless just because the cost query failed. Cache the last successful cost result so a transient failure does not blank the section, and show when it was last refreshed.

Each live row gets an action column. For clusters offer Terminate Cluster and Restart Cluster, plus Start Cluster for anything stopped that I want back. For warehouses offer Stop SQL Warehouse and Start SQL Warehouse, plus a control to tighten the auto stop window using Update SQL Warehouse. Put every action behind a confirmation dialog that names exactly what is about to stop, its current state and uptime, and its owner, so nobody kills a cluster somebody is actively working in. When tightening an auto stop window, send the warehouse's existing configuration along with the new auto_stop_mins so the update does not reset unrelated fields, keep the value at 10 minutes or above, and warn me in the confirmation that applying it may bounce the warehouse. State changes are asynchronous, so after any action poll Get Cluster or Get SQL Warehouse until the state settles and update the row in place with a visible pending indicator rather than making me refresh.

Keep a running log of everything I stop during a session, capturing the name, type, size, uptime at the moment I stopped it, and the estimated hourly rate taken from the cost data where it is available. Then give me a Post summary button that sends that cleanup pass to a Slack channel with Send a Message: what was shut down, why each one qualified (idle time and shutoff setting), and the rough estimated saving, clearly labelled as an estimate. Let me pick the channel in settings and preview the message before it goes. Persist my threshold, the warehouse used for billing queries, the Slack channel, and the session log so the console remembers my setup between visits.

## How to customize

- Change the shutoff threshold that decides what gets flagged. Most teams start at 30 or 60 minutes
- Change the spend window from 30 days, or regroup the cost table by cluster, job, owner or tag
- Pick which Slack channel the cleanup summary goes to, and what the message includes

## FAQ

### Will this work if my workspace does not have billing system tables turned on?

Yes, with one part missing. The spend breakdown needs those tables, so it will show a short notice instead of numbers. The live list of running compute, the idle flags, and every stop, restart and shutoff control keep working normally.

### Can it shut something down by accident?

No. Every action opens a confirmation that names the exact cluster or warehouse about to stop and what it is currently doing, so you always see what you are agreeing to before anything happens.

### Does this cover job clusters as well?

The live list covers all-purpose clusters and SQL warehouses, which is where idle spend actually collects, because job clusters shut themselves down when the job finishes. Job spend still appears in the cost ranking so you can see the full picture.

### How accurate is the saving figure?

It is an estimate based on your recent usage rates for that compute, meant to show the rough scale of a cleanup pass. Treat it as a directional number rather than a replacement for your actual bill.

### Do I need to be a Databricks admin to use it?

You need permission to manage the clusters and warehouses you want to stop. If you only have read access you can still open the console and see everything running, you just will not be able to act on it.

Use this prompt in General Input: https://www.generalinput.com/prompts/shut-down-idle-databricks-compute-from-one-live-console