Screen Typeform survey answers for AI-generated responses
Every morning, new open-text survey answers get checked for AI writing and logged to a spreadsheet, with a Slack alert when too many look suspect.
Typeform
Pangram
Google Sheets
SlackOperationsProductFeedback TriageNotifications & AlertsEvery morning at 6am, screen my Typeform survey's open-ended answers for AI-generated text so bot and LLM-assisted submissions do not quietly poison my research data. Every step is fixed, so keep this fully deterministic with no judgement calls at runtime.
Start with Typeform "Retrieve Responses" to fetch the responses submitted since the previous run. Keep a watermark of the latest submission timestamp I actually retrieved and use that as the lower bound on the next run, rather than the clock time the run started. Typeform can take around 30 minutes to expose very recent submissions through the responses API, so anchoring on retrieved data instead of run time keeps late arrivals from being skipped forever. If no new responses came in, finish the run without writing anything or alerting.
For each response, collect the free-text answers and join them into a single block of text per response, keyed by the response ID. Skip any response whose combined text is shorter than my minimum length, defaulting to 25 words, because it is too short to score reliably. Record those as skipped with the reason rather than dropping them silently, so the log still accounts for every response.
Submit the remaining texts together with Pangram "Create Bulk AI Detection Job", one item per response. Bulk requests are capped at 1,000 billable units, where a unit is one started word block per item, so estimate the units before submitting and split a large batch into multiple sequential jobs when it would exceed the cap. The job is asynchronous, so poll it until it reaches a terminal status of succeeded, failed, or partial.
Read the scores back with Pangram "Get Bulk Job Results", paging until every item is retrieved. Handle partial jobs properly: individual items can fail with a per-item error string such as "Text must contain at least one valid token", so treat those as unscored and log them next to the too-short skips instead of failing the whole run. Bulk results are only retained for 48 hours after completion, so write everything to the spreadsheet in the same run rather than deferring it.
Append one row per response to my Google Sheets tracking tab with "Append Values", containing the response ID, the submission date, the AI likelihood score, and a verdict of clean or suspect. Mark a response suspect when its AI likelihood score is at or above my threshold, defaulting to 0.75. Responses that could not be scored carry their reason in the verdict column instead of a score. Write rows in submission date order so the tab reads as a running audit log I can filter before analysis.
Finally, work out the share of suspect responses among the scored responses in this batch. If that share is at or above my alert percentage, defaulting to 10 percent, send a Slack alert with "Send a Message" naming the suspect count, the number scored, and the percentage, along with the form name and the batch date, so I can pause the panel or tighten screening before I waste more incentive budget. Stay silent when the batch comes in under the threshold.
What does this prompt do?
- Each morning it collects the survey responses that arrived since the last check and pulls out the open-ended answers.
- Every answer gets scored for how likely it is to be AI written, and answers too short to score are marked as skipped instead of guessed at.
- One row per response is added to your tracking tab with the response ID, submission date, the AI likelihood score, and a clean or suspect verdict.
- If too much of a batch looks suspect, you get a Slack message with the count and percentage so you can pause the panel before spending more incentive money.
What do I need to use this?
- A Typeform account and the survey you want to screen
- A Pangram account on a paid plan, which is what checks the writing for AI
- A Google account and a spreadsheet to use as your audit log
- A Slack workspace and a channel where the alerts should land
- A view on your own limits: how high a score counts as suspect, and what share of a batch should trigger an alert
How can I customize it?
- The schedule: 6am daily is the default, but you can run it twice a day or hourly while a panel is actively fielding
- The suspect threshold: how high an AI likelihood score has to be before a response is called suspect, and the alert percentage that triggers Slack
- The minimum answer length worth scoring, plus which channel gets the alert and which tab holds the log
FAQs
Will this delete or reject bad responses for me?
Why does it skip short answers?
Will it rescore responses it has already checked?
Can I screen more than one survey?
Do I need a paid Pangram plan?
Related templates
Stage a batch of filings overnight, then approve each completed form from a screenshot before anything is ever submitted.
Work every partner tier decision from one board, with the revenue and commission numbers, clear promotion thresholds, and AI review briefs.
Open one board at month end to find every closed deal missing a commission, every commission with no deal behind it, and every amount that does not add up.
Review every conversation Fin closed as resolved, judge which ones actually stuck, and see what the gap is worth against your bill.
Work every return, damage and warranty claim in one queue, with the order, the delivery date and a policy-backed recommendation already on screen.
See warmup status, daily send volume, and bounce risk for every sending account in one grid, with the riskiest mailboxes floating to the top.
Stop paying incentives for AI-written survey answers.
Set the morning screen up once and every batch gets audited before it reaches your analysis.