# Model bake-off bench for fal.ai image and video models

> Run one brief across candidate fal.ai models, compare the outputs side by side with real cost and speed, then save the winner to a shared register.

- Workflow type: app
- Services: fal.ai, Google Sheets, Slack Bot
- Categories: Marketing, Operations
- Published: 2026-08-21

## What it does

- Type one plain English brief, choose a job type, and either pick the models yourself or let a background helper shortlist candidates that fit your budget, write a tuned prompt for each one, run the whole grid, and hand back a recommendation with its reasoning.
- Compare every result in a side-by-side grid where each tile shows the finished image or video next to what that run actually cost and how long it took, always priced per output rather than as one lump total.
- Blind mode keeps model names hidden until someone picks a favorite, so the team judges the work instead of voting for the brand they recognize.
- Save the winner to a shared register your team can look up later: which model you chose for which job, at what cost, and when you last checked, with a one-click push of the comparison to Slack.

## What you'll need

- A fal.ai account with credit on it, since every run is billed to that balance
- A Google account and a spreadsheet to use as the house register of model decisions
- A Slack workspace, if you want to share comparisons with a channel
- A rough cost ceiling per job, so the bench can tell you when a lineup is too expensive before you run it

## Prompt

Build me a model bake-off bench for fal.ai that my creative and engineering team opens every time we need to decide which generation model to use for a job. The point is to turn model choice from a recurring argument into a measurable decision, and to leave behind a house register of what we picked and what it cost.

The main surface is the bench. I type one plain English brief, pick a job type from product shot, hero illustration and social video, and set a cost ceiling for the run. Then I choose candidate models from the fal.ai catalog. A handler uses List Models to browse and filter the catalog down to models that suit the selected job type, and it should request them with expand=openapi-3.0 so the app has each model's own input schema and can build a valid payload per model instead of assuming every model takes the same fields.

Before I commit, show me the projected spend. A handler calls Get Model Pricing for each shortlisted model and Estimate Pricing for the planned grid, and the bench shows the per-model unit price, the projected cost of the whole grid, and whether that lands under my cost ceiling. If the lineup would blow the ceiling, say so plainly and let me drop models until it fits.

Hitting Run fires every candidate in parallel. Use Submit Queue Request (Async) to fire the runs and Stream Queue Request Status to drive a live status on each tile, so I watch the grid fill in rather than staring at one spinner. For the models that have dedicated operations, use them: Generate Image with FLUX1.1 [pro], Generate Image with FLUX.1 [dev], Generate Image with Recraft V3, Generate or Edit Image with Nano Banana, and Generate Video with Kling, Generate Video with Google Veo 3, or Generate Video with MiniMax Hailuo. Record the wall clock time from submit to completion for every single run.

The results view is a side-by-side grid, one tile per candidate model. Every tile shows the generated image or video, what that run cost, how long it took, and the cost per output. Never show me only a grid total, because cost per output is the number I actually compare on and it has to be on every tile. Reconcile the billed figure with Get Model Usage once the runs land, so the tile shows what fal.ai really charged rather than just the estimate. Keep the tuned prompt used for each tile visible or one click away so any run is reproducible.

Bake in a blind mode and have it on by default. While blind mode is on, tiles are labelled Model A, Model B, Model C with model names and vendor branding hidden, so nobody just votes for the brand they recognize. Cost and generation time stay visible, because those are part of the judgement. As soon as I mark a favorite, reveal every name at once and tell me whether the one I picked was the cheap option or the expensive one.

Add a Run the bake-off button that kicks off a background agent. The agent reads my plain English brief and the job type, browses the catalog with List Models to shortlist candidate models that suit the job and fit my cost ceiling, checks Get Model Pricing and Estimate Pricing to keep that shortlist inside budget, writes a tuned prompt for each candidate built from that model's own input schema from List Models with expand=openapi-3.0 rather than reusing one generic prompt everywhere, fires the whole grid, waits for the runs to finish, and then writes a recommendation back into the app: which model it would pick for this job, why, and what the tradeoff against the runners up is in quality versus cost. The recommendation lands next to the grid with its reasoning visible so I can agree with it or overrule it.

When I pick a winner it saves to a house register in Google Sheets using Append Values: the date, the job type, the brief, the winning model, the cost per output, the generation time, the runners up, who picked it, and a link to the output. The register view inside the app reads that sheet back with Get Values and is the thing people come back to, so make it good: which model we chose for which job, at what cost, and when we last checked. Let me group it by job type, show the last checked date on every row, and flag any decision older than 90 days as worth re-running, because prices and models move fast.

Give me a button to push a comparison to a channel with the Slack Bot Send a Message operation: the brief, the job type, the winning model with its cost per output, and the runners up with theirs, so the team sees the decision without needing to open the app.

House rules across the whole app. Model input schemas differ per model on fal.ai, so always build each payload from that model's own schema instead of a shared shape. Every price shown is in US dollars. Keep past bake-offs browsable so I can reopen one and see the grid exactly as it was. And treat the register as the product: if someone opens this app and only ever reads the register, it should still have earned the visit.

## How to customize

- Change the job types to match the work you actually brief, such as packaging renders, ad variations, or explainer clips, and give each one its own default cost ceiling.
- Decide whether blind mode is on by default and whether anyone can reveal model names before a favorite has been picked.
- Point the register at a different spreadsheet, add your own columns, and set how old a decision gets before the bench flags it as worth re-checking.

## FAQ

### Do I need a separate account for every model I want to test?

No. One fal.ai account reaches dozens of hosted image and video models, so there is no per-vendor signup or billing to wrangle before you can compare them.

### How much does running a bake-off cost?

It depends entirely on which models you line up. The bench shows a projected spend before you commit and the real cost of each run afterwards. Cheap image models cost a fraction of a cent per picture while flagship models can be more than a hundred times that, which is exactly why the comparison is worth running.

### What is blind mode for?

It hides model names and vendor branding until someone has picked a favorite, so people vote on the output in front of them rather than the brand they already trust. Cost and generation time stay visible, because those are part of the judgement.

### Can I compare video models as well as image models?

Yes. Video jobs work the same way as image jobs, and each tile shows the clip alongside what it cost and how long it took to generate.

### Where do the decisions actually live?

In a Google Sheets register you own. Every winner is appended with the job type, the model, the cost per output and the date, so the next person can look it up instead of re-litigating the choice.

Use this prompt in General Input: https://www.generalinput.com/prompts/model-bake-off-bench-for-falai-image-and-video-models