How do I add a backup crawler to Firecrawl?
Keep Firecrawl as your primary and send UpRock only the pages that come back with a 401, 403, 429 or 5xx, time out, or need a country Firecrawl doesn't cover. UpRock loads the page on a real desktop or mobile device in the country you pick and returns markdown, at $0.50 per 1,000 pages.
Updated · Read as markdown
What Firecrawl does well
Firecrawl turns websites into LLM-ready data, and it does more than fetch pages. One API key gets you scrape, crawl (follow links across a site), map (list a site's URLs), web search, batch scraping, JSON extraction with a schema, browser actions and change tracking. It has SDKs in nine languages, an official MCP server (remote or local), and an AGPL-3.0 core you can self-host. Scrapes are cached for two days by default, so repeat reads are fast.
Its default proxy setting, auto, already handles a lot of trouble for you: when a target page answers 401, 403 or 429, Firecrawl retries once on its enhanced proxies, at no extra credit cost.
If Firecrawl returns what you need, keep using it. A backup is for the rest.
When a second crawler helps
A Firecrawl call can come back without the page you wanted in a few distinct ways. Each one has a different right answer, and only some of them are a job for UpRock.
| What you see from Firecrawl | What it means | Send it to UpRock? |
|---|---|---|
metadata.statusCode 401, 403 or 429, after the auto retry | The target site refused Firecrawl's request | Yes |
| Error 408 or 5xx | The page was slow, or Firecrawl had a problem | Yes |
| Error 429 | You hit your plan's limit (Hobby: 100 scrapes a minute, 5 browsers at once) | Yes, or queue and retry |
| Error 402 | Your credits ran out | Yes, or top up |
| You need a country outside Firecrawl's list | Firecrawl fetches from the closest region (EU or US) and sets the browser location to your country | Go straight to UpRock |
Error 403 UNSUPPORTED_SITE | Firecrawl doesn't scrape this site, by policy | Usually no. UpRock only fetches public pages and never logs in, so most of these won't work for us either |
metadata.statusCode 404 or 410 | The page is gone | No. Every crawler will see the same |
The country row is easy to miss. Firecrawl's basic proxies cover 26 countries and its enhanced proxies cover two (the Netherlands and the US). Ask for a page from South Korea, Indonesia or Argentina and Firecrawl serves it from the nearest supported region instead. UpRock takes the country per request (ISO code), and the fallback below goes straight to it for those.
Firecrawl and UpRock side by side
Firecrawl numbers come from its own pricing and docs pages, checked 24 Oct 2026. Prices are per 1,000 plain markdown pages (1 credit each on Firecrawl).
| Firecrawl | UpRock | |
|---|---|---|
| Price per 1,000 pages | $3.80 on Hobby ($19 for 5,000 credits), $0.99 on Standard, $0.80 on Growth, $0.75 on Scale ($749 for 1,000,000), $0.60 on Scale billed yearly | $0.50 by card or USDC, $0.40 with UPT. Prepaid credits, no plan |
| Extra-cost options | JSON extraction +4 credits a page, PDFs +1 a page, zero data retention +1 | None. Every page is 1 credit |
| Where requests come from | Rotating proxies and a headless browser; mobile is emulated | Real desktop and mobile devices whose owners opted in |
| Countries | 26 on basic proxies, 2 on enhanced; US by default | 63, or a region |
| Crawl a whole site, map, search, JSON extraction | Yes | No. One page per call |
| MCP server | Official, remote or local | Yes, remote (streamable HTTP) |
| Video | A video format returns a file link for supported video URLs | Search public video by moment, $1 per 1,000 searches |
| Self-host | AGPL-3.0; some features are cloud-only | No. The device runtime (Fuel Engine) is open source |
| Rate limits | Published per plan, from 10 scrapes a minute (Free) to 10,000 (Scale) | Not published yet |
Where Firecrawl is better: whole-site crawling, structured extraction, browser actions, a Free plan with 1,000 credits a month to try it, and published rate limits. Where UpRock fits: the page Firecrawl couldn't return, a real device instead of a proxy, a country outside Firecrawl's list, and one flat price per page.
The fallback, in TypeScript and Python
Every version does the same thing:
- If the country is on Firecrawl's list, call Firecrawl.
- If Firecrawl throws, or returns a target status worth retrying, call UpRock.
- Respect
UNSUPPORTED_SITEinstead of working around it.
The HTTP versions make one POST /v1/crawl request with your key in an Authorization: Bearer header. UpRock loads the page on a device in the country you ask for, tries a fresh device every 7 seconds, up to 3 times, and returns the markdown in the response. If you already use UpRock's MCP server or CLI, the other tabs do the same through those: crawl_fetch loads the page and resource_fetch returns the full markdown, or the CLI prints it as JSON.
// npm i firecrawl (Node 18+) env: FIRECRAWL_API_KEY, UPROCK_API_KEYimport { Firecrawl, SdkError } from "firecrawl";const firecrawl = new Firecrawl(); // reads FIRECRAWL_API_KEY// Firecrawl's basic-proxy countries (docs.firecrawl.dev/features/proxies, checked 19 Oct 2026).const FIRECRAWL_COUNTRIES = new Set( "AE AT AU BE BR CA CH CN DE DK EG ES FR GB GR IL IN IT JP MX NL PL QA SE TR US".split( " ", ),);// Target statuses worth a second try from a real device. 404 and 410 mean the page is gone.const RETRY = new Set([401, 403, 429, 500, 502, 503, 504]);async function viaUpRock(url: string, country: string) { // One call: UpRock loads the page on a real device in `country`, tries a fresh device // every 7 s (up to 3 tries) and returns markdown. Failed calls aren't billed. const res = await fetch("https://edge.uprock.com/v1/crawl", { method: "POST", headers: { Authorization: `Bearer ${process.env.UPROCK_API_KEY}`, "Content-Type": "application/json", }, body: JSON.stringify({ url, country }), signal: AbortSignal.timeout(90_000), }); const page = await res.json(); if (!res.ok) throw new Error(`UpRock ${res.status} ${page.code}: ${page.error}`); return { via: "uprock", markdown: page.markdown ?? "" };}export async function scrape(url: string, country = "US") { if (FIRECRAWL_COUNTRIES.has(country)) { try { const doc = await firecrawl.scrape(url, { formats: ["markdown"], location: { country }, }); const status = doc.metadata?.statusCode ?? 200; if (!RETRY.has(status)) return { via: "firecrawl", markdown: doc.markdown ?? "" }; } catch (err) { // Firecrawl answers 403 UNSUPPORTED_SITE for sites it won't scrape. Respect that. if ( err instanceof SdkError && (err.status === 403 || err.code === "UNSUPPORTED_SITE") ) throw err; console.warn(`Firecrawl failed (${(err as Error).message}), trying UpRock`); } } return viaUpRock(url, country);}// node fallback.ts https://developer.mozilla.org/en-US/docs/Web/HTTP KRconst [url = "https://developer.mozilla.org/en-US/docs/Web/HTTP", country = "US"] = process.argv.slice(2);const page = await scrape(url, country);console.log(page.via, page.markdown.slice(0, 200));# pip install firecrawl-py requests env: FIRECRAWL_API_KEY, UPROCK_API_KEYimport osimport sysimport requestsfrom firecrawl import Firecrawl, WebsiteNotSupportedErrorfirecrawl = Firecrawl() # reads FIRECRAWL_API_KEY# Firecrawl's basic-proxy countries (docs.firecrawl.dev/features/proxies, checked 19 Oct 2026).FIRECRAWL_COUNTRIES = set( "AE AT AU BE BR CA CH CN DE DK EG ES FR GB GR IL IN IT JP MX NL PL QA SE TR US".split())# Target statuses worth a second try from a real device. 404 and 410 mean the page is gone.RETRY = {401, 403, 429, 500, 502, 503, 504}def via_uprock(url: str, country: str) -> dict: # One call: UpRock loads the page on a real device in `country`, tries a fresh device # every 7 s (up to 3 tries) and returns markdown. Failed calls aren't billed. res = requests.post( "https://edge.uprock.com/v1/crawl", headers={"Authorization": f"Bearer {os.environ['UPROCK_API_KEY']}"}, json={"url": url, "country": country}, timeout=90, ) page = res.json() if not res.ok: raise RuntimeError(f"UpRock {res.status_code} {page.get('code')}: {page.get('error')}") return {"via": "uprock", "markdown": page.get("markdown", "")}def scrape(url: str, country: str = "US") -> dict: if country in FIRECRAWL_COUNTRIES: try: doc = firecrawl.scrape( url, formats=["markdown"], location={"country": country} ) status = (doc.metadata and doc.metadata.status_code) or 200 if status not in RETRY: return {"via": "firecrawl", "markdown": doc.markdown or ""} except WebsiteNotSupportedError: raise # Firecrawl answers 403 UNSUPPORTED_SITE for sites it won't scrape. Respect that. except Exception as err: print(f"Firecrawl failed ({err}), trying UpRock", file=sys.stderr) return via_uprock(url, country)if __name__ == "__main__": # python fallback.py https://developer.mozilla.org/en-US/docs/Web/HTTP KR url = sys.argv[1] if len(sys.argv) > 1 else "https://developer.mozilla.org/en-US/docs/Web/HTTP" country = sys.argv[2] if len(sys.argv) > 2 else "US" page = scrape(url, country) print(page["via"], page["markdown"][:200])// npm i firecrawl @modelcontextprotocol/sdk (Node 18+) env: FIRECRAWL_API_KEY, UPROCK_API_KEYimport { Firecrawl, SdkError } from "firecrawl";import { Client } from "@modelcontextprotocol/sdk/client/index.js";import { StreamableHTTPClientTransport } from "@modelcontextprotocol/sdk/client/streamableHttp.js";const firecrawl = new Firecrawl(); // reads FIRECRAWL_API_KEYconst uprock = new Client({ name: "firecrawl-fallback", version: "1.0.0" });await uprock.connect( new StreamableHTTPClientTransport( new URL("https://mcp.uprock.ai/mcp"), { requestInit: { headers: { Authorization: `Bearer ${process.env.UPROCK_API_KEY}` } } }, ),);// Firecrawl's basic-proxy countries (docs.firecrawl.dev/features/proxies, checked 19 Oct 2026).const FIRECRAWL_COUNTRIES = new Set( "AE AT AU BE BR CA CH CN DE DK EG ES FR GB GR IL IN IT JP MX NL PL QA SE TR US".split( " ", ),);// Target statuses worth a second try from a real device. 404 and 410 mean the page is gone.const RETRY = new Set([401, 403, 429, 500, 502, 503, 504]);const textOf = (content: unknown) => (content as { type: string; text?: string }[]).find((c) => c.type === "text")?.text ?? "";async function viaUpRock(url: string, country: string) { // crawl_fetch loads the page on a device in `country`, retrying on another device every 7 s. const res = await uprock.callTool( { name: "crawl_fetch", arguments: { url, country } }, undefined, { timeout: 300_000 }, ); if (res.isError) throw new Error(`UpRock: ${textOf(res.content)}`); const page = JSON.parse(textOf(res.content)); const uri = page.content?.markdown?.resource; if (!uri) throw new Error(`UpRock: no markdown for ${url}`); const md = await uprock.callTool({ name: "resource_fetch", arguments: { uri } }); return { via: "uprock", markdown: textOf(md.content) };}export async function scrape(url: string, country = "US") { if (FIRECRAWL_COUNTRIES.has(country)) { try { const doc = await firecrawl.scrape(url, { formats: ["markdown"], location: { country }, }); const status = doc.metadata?.statusCode ?? 200; if (!RETRY.has(status)) return { via: "firecrawl", markdown: doc.markdown ?? "" }; } catch (err) { // Firecrawl answers 403 UNSUPPORTED_SITE for sites it won't scrape. Respect that. if ( err instanceof SdkError && (err.status === 403 || err.code === "UNSUPPORTED_SITE") ) throw err; console.warn(`Firecrawl failed (${(err as Error).message}), trying UpRock`); } } return viaUpRock(url, country);}// node fallback.ts https://developer.mozilla.org/en-US/docs/Web/HTTP KRconst [url = "https://developer.mozilla.org/en-US/docs/Web/HTTP", country = "US"] = process.argv.slice(2);const page = await scrape(url, country);console.log(page.via, page.markdown.slice(0, 200));await uprock.close();# pip install firecrawl-py; npm i -g uprock env: FIRECRAWL_API_KEY, UPROCK_API_KEYimport jsonimport subprocessimport sysfrom firecrawl import Firecrawl, WebsiteNotSupportedErrorfirecrawl = Firecrawl() # reads FIRECRAWL_API_KEY# Firecrawl's basic-proxy countries (docs.firecrawl.dev/features/proxies, checked 19 Oct 2026).FIRECRAWL_COUNTRIES = set( "AE AT AU BE BR CA CH CN DE DK EG ES FR GB GR IL IN IT JP MX NL PL QA SE TR US".split())# Target statuses worth a second try from a real device. 404 and 410 mean the page is gone.RETRY = {401, 403, 429, 500, 502, 503, 504}def via_uprock(url: str, country: str) -> dict: # The UpRock CLI loads the page on a device in `country`, retries on other devices, # and prints JSON. It exits non-zero, with the reason on stderr, when every attempt fails. run = subprocess.run( ["uprock", "ai", "crawl", url, "--country", country, "--content"], capture_output=True, text=True, timeout=330, ) if run.returncode != 0: raise RuntimeError(f"UpRock: {run.stderr.strip()}") page = json.loads(run.stdout) return {"via": "uprock", "markdown": page.get("inlined_markdown", "")}def scrape(url: str, country: str = "US") -> dict: if country in FIRECRAWL_COUNTRIES: try: doc = firecrawl.scrape( url, formats=["markdown"], location={"country": country} ) status = (doc.metadata and doc.metadata.status_code) or 200 if status not in RETRY: return {"via": "firecrawl", "markdown": doc.markdown or ""} except WebsiteNotSupportedError: raise # Firecrawl answers 403 UNSUPPORTED_SITE for sites it won't scrape. Respect that. except Exception as err: print(f"Firecrawl failed ({err}), trying UpRock", file=sys.stderr) return via_uprock(url, country)if __name__ == "__main__": # python fallback.py https://developer.mozilla.org/en-US/docs/Web/HTTP KR url = sys.argv[1] if len(sys.argv) > 1 else "https://developer.mozilla.org/en-US/docs/Web/HTTP" country = sys.argv[2] if len(sys.argv) > 2 else "US" page = scrape(url, country) print(page["via"], page["markdown"][:200])The HTTP versions were tested on 3 Nov 2026 against the live UpRock API, with an invalid Firecrawl key to force the fallback and with KR and ID, which are outside Firecrawl's list; the MCP and CLI versions on 19 Oct 2026. Only send the cases from the table above: a blocked page costs a Firecrawl credit and then an UpRock credit. On 6 Nov 2026 we ran the HTTP and MCP versions again with an UpRock API key in an Authorization header: with an invalid Firecrawl key, each fell back to UpRock and got the page.
If you'd rather not write code
Agents can use both tools side by side over MCP. Add UpRock next to Firecrawl's MCP server:
claude mcp add --transport http uprock https://mcp.uprock.ai/mcp --header "Authorization: Bearer $UPROCK_API_KEY"Then tell the agent when to use which, for example in CLAUDE.md or AGENTS.md:
Fetching web pages: use Firecrawl first. If a Firecrawl call fails, or the page comes backwith status 401, 403 or 429, or I ask for a country Firecrawl doesn't cover, retry withUpRock's crawl_fetch and pass the country. Don't retry 404s.Or from a terminal:
curl -s https://edge.uprock.com/v1/crawl \ -H "Authorization: Bearer $UPROCK_API_KEY" -H "Content-Type: application/json" \ -d '{"url": "https://developer.mozilla.org/en-US/docs/Web/HTTP", "country": "KR"}'npm i -g uprockuprock ai crawl https://developer.mozilla.org/en-US/docs/Web/HTTP --country KR --contentFAQ
Does Firecrawl charge for a blocked page?
Yes. Firecrawl's pricing page says a page that responds with an error status such as 403 or 404 is still returned and costs 1 credit, while a scrape that returns no result is not charged. With a fallback, a blocked page costs one Firecrawl credit plus an UpRock credit for the retry.
Can I point the Firecrawl SDK at UpRock by changing the base URL?
Not today. UpRock has its own API, so the fallback on this page calls it directly. It's about 30 lines in each language.
Is UpRock cheaper than Firecrawl?
Per page, yes. A plain markdown page costs $0.50 per 1,000 on UpRock, against $0.60 to $3.80 per 1,000 on Firecrawl's self-serve plans. But Firecrawl's credits also buy crawling, site maps, search and JSON extraction, which UpRock doesn't offer. Most teams that use both keep Firecrawl as the primary.
Why not just use Firecrawl's enhanced proxies?
Use them first. Firecrawl's default proxy setting, auto, already retries a blocked page once on enhanced proxies at no extra charge. UpRock helps with what's left, from a different kind of source (a real device) and in countries outside Firecrawl's list.
Which countries can UpRock fetch from?
63, by ISO country code, or a region (NA, EU, APAC, LATAM, MEA) when any country in it will do. That includes South Korea, Indonesia and Argentina, which aren't on Firecrawl's list. Firecrawl lists 26 countries for basic proxies and 2 for enhanced proxies.
Other use cases to explore
- See a page as it looks in another country: one store page from five countries, with five prices, two languages and a cookie banner in one country only.
- Decide which failed pages are worth retrying: a diagnosis table from status code to the legitimate fix, and when to stop.
- Add a fallback to a crawler you host: send only the pages your server can't load to a device in the country you choose.
- Search public video by moment: find the second something is shown or said, across platforms.
References
- Firecrawl, Pricing, checked 3 Nov 2026
- Firecrawl docs, Billing (credit costs and surcharges), checked 3 Nov 2026
- Firecrawl docs, Proxies (countries per proxy type), checked 3 Nov 2026
- Firecrawl docs, Enhanced mode (what auto escalation retries), checked 19 Oct 2026
- Firecrawl docs, Errors, checked 19 Oct 2026
- Firecrawl docs, Rate limits, checked 3 Nov 2026
- Firecrawl docs, MCP server, checked 19 Oct 2026
- Firecrawl on GitHub (license and self-hosting), checked 19 Oct 2026
- UpRock, everything in one file (API, CLI, MCP, pricing), checked 3 Nov 2026
Get started
Add UpRock in one paste.
Paste this into Claude Code, Codex, Cursor or any agent with a terminal. It reads our docs, sets up the MCP server or CLI, and tells you the cost before it spends anything.
Set up UpRock for me: read https://uprock.ai/agents.md, connect the UpRock MCP server, and run one test crawl. Tell me what it costs before you spend anything.