Weekly Databricks cost digest to catch spend spikes
Every Monday, get a plain-language recap of last week's Databricks spend in Slack, with the top cost drivers and any spending spikes flagged for you.
Databricks
Slack BotFinanceEngineeringAI ReportsNotifications & AlertsDaily DigestsEvery Monday at 8am, summarize last week's Databricks spend and post a plain-language cost digest to Slack so finance and platform leads can catch cost spikes early. Run this on a cron schedule (Mondays at 8:00am in our timezone).
Use the Databricks Execute SQL Statement operation to query the Unity Catalog billing usage system table (system.billing.usage) for DBU consumption over the last 7 days versus the prior 7 days. Provide the warehouse_id of a running SQL warehouse to run the statement. Break the results down three ways: by SKU (the billed product, such as Jobs Compute or SQL warehouse usage), by workspace, and by job or warehouse. Compute two windows so week-over-week change can be calculated per group: the current 7-day window (day -7 through today) and the prior 7-day window (day -14 through day -7). Note that system.billing.usage records DBU quantities; if the workspace also exposes system.billing.list_prices, join to it to translate DBUs into an estimated dollar amount, otherwise report spend in DBUs. The workspace personal access token must have access to the system.billing schema.
After submitting the statement, poll the Databricks Get SQL Statement operation until status.state is SUCCEEDED. Keep waiting and re-checking while the state is PENDING or RUNNING, and treat FAILED or CANCELED as an error to report in the digest rather than failing silently. If the result set is paged into multiple chunks, fetch the remaining chunks so you have the complete data before analyzing.
Once the full data is back, have the agent reason over it rather than dumping the table. Identify the top cost drivers (the SKUs, workspaces, and jobs or warehouses consuming the most DBUs or dollars this week), the biggest week-over-week changes in both directions, and any anomalies (for example a job that suddenly appears with no prior baseline, or a large jump concentrated in one workspace). Flag any job or warehouse whose spend jumped more than a configurable threshold percentage (default 30%) week over week, naming it and showing the prior amount, the current amount, and the percent change.
Post the recap to Slack using the Slack Bot Send a Message operation (posted as the bot) to a configurable channel such as #databricks-costs. Write it in business terms: a short headline with total spend and the week-over-week direction, a few bullets naming the top cost drivers and notable movers, and a clearly labeled section for any threshold breaches and anomalies. Use Slack mrkdwn formatting (single asterisks for bold) and keep it skimmable for a non-technical finance or platform lead. Make the threshold percentage, the target channel, and the schedule easy to change.
Example output
Additional information
What does this prompt do?
- Pulls last week's Databricks usage and compares it to the week before, broken down by product, workspace, and job or warehouse.
- Turns the numbers into a short, plain-language recap of total spend and where the money went, so nobody has to read a raw usage table.
- Automatically flags any job or warehouse whose spend jumped more than the percentage you choose, plus anything that looks unusual.
- Posts the summary to a Slack channel every Monday morning so finance and platform leads see it before the week starts.
What do I need to use this?
- A Databricks account with permission to read your workspace's billing and usage data.
- A Databricks SQL warehouse the report can use to run its weekly query.
- A Slack workspace, plus the channel where finance and platform leads should get the digest.
How can I customize it?
- Set the alert sensitivity: the percentage jump that flags a job or warehouse as a spending spike.
- Pick the day and time it runs and the Slack channel it posts to.
- Adjust the comparison window or the breakdown (by product, workspace, or job) to match how your team thinks about cost.
Frequently asked questions
How does it know how much we spent on Databricks?
Will it ping us for every little change?
Which Slack channel does it post to?
How often does it run?
Do we need to be technical to read the digest?
Related templates
Stop finding out about Databricks overruns too late.
Put a plain-language weekly cost recap in front of your finance and platform leads every Monday, with spending spikes flagged automatically.