Weekly Airbyte cost and volume report to leadership

Every Monday at 7am, pull the last 7 days of Airbyte sync activity and email a volume and cost report to data leadership and finance.

Step 1. Call Airbyte List Connections to fetch every connection across accessible workspaces. Keep the connection id, connection name, source name, destination name, and workspace for each one.

Step 2. Call Airbyte List Jobs twice, both scoped to jobType=sync. First call: createdAtStart set to 7 days ago and createdAtEnd set to now (this week). Second call: createdAtStart 14 days ago and createdAtEnd 7 days ago (prior week). Page through with limit=100 and increasing offset until all jobs are returned.

Step 3. For every job in both windows, call Airbyte Get Job Details to retrieve status, rowsSynced, bytesSynced, and duration. Group both sets by connectionId.

Step 4. For each connection with jobs in the current week, calculate total rows synced (sum of rowsSynced), total sync duration (sum of duration, formatted as hours plus minutes), success rate (jobs with status=succeeded divided by total jobs, as a percentage), prior-week rows synced (same sum for the prior 7-day window), and growth ratio (this-week rows divided by prior-week rows).

Step 5. Build a single HTML email with two sorted tables and a footer. Table 1 header 'Top 10 connections by volume this week', with columns Connection, Source, Destination, Rows Synced, Sync Duration, and Success Rate, sorted by rows synced descending, first 10 rows. Table 2 header '5 slowest connections', same columns, sorted by total sync duration descending, first 5 rows. Footer section titled 'Cost watch: connections growing more than 2x week over week', with one line per connection whose growth ratio is greater than 2.0, formatted like 'Connection Name: 1,234,567 rows this week vs 480,000 last week (2.6x)'. If no connection tripped the threshold, write 'No connections doubled week over week.'. Use inline CSS for basic table styling (borders, header row background, padding, right-aligned numeric columns). No external images or scripts. Include a plain-text summary line at the top with total rows synced this week across all connections and the delta versus prior week.

Step 6. Send the email with Gmail Send a Message. Set the To header to the data leadership and finance distribution list, Subject to 'Airbyte weekly volume report for <this Monday's date>', and the body to the HTML report from step 5. Set the Content-Type header to text/html so recipients see the formatted table.

Notes. Only include jobs with jobType=sync in the metrics; ignore resets. If Get Job Details returns rowsSynced as null or zero for a succeeded job, treat it as zero and keep going. If a connection has no jobs this week but did last week, do not list it in the two tables, but still evaluate it for the growth footer (its growth ratio will be zero, which is not greater than 2). Round success rate to the nearest whole percentage. Format row counts with thousands separators.