# Scrapfly integration

> Web scraping, screenshot, AI data extraction and site crawling APIs with proxies, JavaScript rendering and anti-bot bypass.

- Authentication: apiKey

## Tools (47)

- **Ask Question Over Crawls**: Get a cited, model-generated answer grounded in the indexed pages of one or more crawls.
- **Ask Question Over One Crawl**: Get a cited, model-generated answer grounded in the indexed pages of a single crawl.
- **Cancel Crawl**: Stop a running crawl; already crawled pages stay retrievable.
- **Cancel Crawl Schedule**: Permanently cancel a crawler schedule.
- **Cancel Scrape Schedule**: Permanently cancel a scrape schedule; it stops firing and cannot be resumed.
- **Cancel Screenshot Schedule**: Permanently cancel a screenshot schedule.
- **Classify Response For Anti-Bot Block**: Check whether an HTTP response you already hold is an anti-bot block page and name the anti-bot product.
- **Create Crawl**: Start a site crawl from a seed URL, a URL list or a remote URL list and return the crawler UUID.
- **Create Crawl Schedule**: Schedule a recurring crawl; each fire creates a new crawl and reports to a webhook.
- **Create Scrape Schedule**: Schedule a recurring scrape whose results are delivered to a webhook.
- **Create Screenshot Schedule**: Schedule recurring screenshot captures delivered to a webhook.
- **Download Crawl Artifact**: Download the full crawl as a WARC or HAR archive once the crawl has finished.
- **Execute Crawl Schedule Now**: Trigger an immediate crawl from a crawler schedule on top of its regular cadence.
- **Execute Scrape Schedule Now**: Trigger an immediate fire of a scrape schedule on top of its regular cadence.
- **Execute Screenshot Schedule Now**: Run the pending fire of a screenshot schedule immediately.
- **Extract Data From Document**: Extract structured data or answer questions from HTML, Markdown, text or XML you send, using a template, an LLM prompt or an AI model.
- **Extract Structured Data From Crawls**: Return a JSON object matching your JSON Schema, built from the indexed pages of one or more crawls.
- **Get Account**: Return account, project, subscription, quota and concurrency usage for the API key.
- **Get Crawl Contents**: Fetch crawled page content for all pages (paginated) or one URL, in one or more formats.
- **Get Crawl Contents For URL List**: Fetch crawled content for up to 100 specific URLs in one multipart response.
- **Get Crawl Refresh History**: List the refresh runs of a crawl with counts of added, updated, removed and unchanged pages.
- **Get Crawl Schedule**: Return one crawler schedule by id.
- **Get Crawl Status**: Poll a crawl's progress, counters, stop reason, search index state and refresh state.
- **Get Scrape Schedule**: Return one scheduled scrape by id.
- **Get Screenshot Schedule**: Return one screenshot schedule by id.
- **List All Schedules**: List scrape, screenshot and crawler schedules together in one cross-product list.
- **List Crawled URLs**: List the visited, failed or skipped URLs of a crawl as plain text, one per line.
- **List Crawl Schedules**: List the account's crawler schedules.
- **List Scrape Schedules**: List the account's scheduled scrapes, optionally filtered by status.
- **List Screenshot Schedules**: List the account's screenshot schedules.
- **Pause Crawl Schedule**: Pause a crawler schedule so it stops firing until resumed.
- **Pause Scrape Schedule**: Pause a scrape schedule so it stops firing until resumed.
- **Pause Screenshot Schedule**: Pause a screenshot schedule so it stops firing until resumed.
- **Resume Crawl Schedule**: Resume a paused crawler schedule; missed fires are not replayed.
- **Resume Scrape Schedule**: Resume a paused scrape schedule; missed fires are not replayed.
- **Resume Screenshot Schedule**: Resume a paused screenshot schedule; missed fires are not replayed.
- **Run Crawl Refresh Now**: Start an immediate refresh run that re-scrapes the URLs a crawl already holds.
- **Scrape Batch**: Run up to 100 scrape configurations in one request and stream each result back as a multipart part.
- **Scrape URL**: Fetch a web page through Scrapfly proxies with optional JavaScript rendering, Unblocker, format conversion, sessions, screenshots and inline extraction.
- **Scrape URL With Request Body**: Forward a POST, PUT or PATCH request with a body to the target URL through Scrapfly.
- **Search Crawl**: Search one crawl's index (GET form) for ranked passages by meaning, keyword or both.
- **Search Crawls**: Search the indexes of up to 256 crawls at once for ranked passages, with filters.
- **Take Screenshot**: Capture a screenshot of a web page, full page or element, with proxy, resolution and accessibility options.
- **Update Crawl Refresh Schedule**: Turn auto refresh on or off for a crawl or change its interval.
- **Update Crawl Schedule**: Change fields of an active crawler schedule.
- **Update Scrape Schedule**: Change the recurrence, scrape configuration, retry settings or notes of an active scrape schedule.
- **Update Screenshot Schedule**: Change fields of an active screenshot schedule.

Connect Scrapfly in General Input: https://www.generalinput.com/apps/scrapfly