Spider costs less per page than UpRock. When is a backup still worth it?
Keep Spider as your primary, because by its own published averages a page costs a fraction of a cent, well under UpRock's $0.50 per 1,000. Add UpRock only for the pages Spider comes back without, meaning a status outside 2xx, no content or a timeout, so they get a second try from a different network, on an opted-in device in the country you pick.
Updated · Read as markdown
What Spider does well
Spider is a fast, inexpensive crawling API built on an open-source Rust crawler (spider-rs/spider, MIT). It crawls whole sites, streams each page back as it finishes, returns markdown or other formats, and also does search, screenshots, link discovery and HTML-to-markdown transforms. It has SDKs for Python, JavaScript, Rust and Go, integrations with LangChain, LlamaIndex, CrewAI and others, proxies in 199 countries, and a hosted MCP server with 21 tools. Its pay-as-you-go rate limit is 10,000 requests a minute.
And it's cheap. Spider bills two things per request: bandwidth ($1 per GB of content before conversion) and compute. JavaScript rendering isn't a multiplier; it just takes more compute. Spider's own pricing page puts 100 average pages at $0.002 to $0.01 over plain HTTP and $0.0035 to $0.0155 in a browser, which is $0.02 to $0.16 per 1,000 pages. That's less than UpRock.
So this page isn't about price. Keep Spider as your primary crawler.
How Spider reports a page it couldn't get
Spider returns a JSON array for every request, even a single-page /scrape, with one object per page: url, content, status, error and costs. The HTTP status of the API call says whether Spider took your request, not whether the site gave you the page.
We called Spider's API on 21 Oct 2026 with a few test pages and noted what came back:
| Test page | What Spider returned | What to make of it |
|---|---|---|
| example.com | HTTP 200; status 200, content present | Fine |
| A URL that always answers 403 (no body) | HTTP 200; status 403, empty content, error: null | Failed, though error says nothing. Send to UpRock |
| A URL that always answers 403 (short JSON body) | Three times: HTTP 200, status 403, the body as content. Twice: accepted as a normal page, with the body in a code block | A status check alone missed it twice in five tries |
A Wikipedia article in plain http mode | HTTP 200; status 526 after 40 seconds, no content, error "address unreachable" (the same article came back fine in browser mode) | Failed. Send to UpRock |
| A page that loaded | HTTP 200; status 200 with content, and a non-null error ("address unreachable") | Fine, despite the error text |
Two lessons from this. Decide on status and content, never on error. And if you know what a good page looks like (a title, a price, a minimum length), check for it, because a refused page can occasionally look like a success.
The API call itself can fail too: 401 for a bad key, 402 when you're out of credits, 429 when you're rate limited (Spider asks you to honor Retry-After), and 500 or 503 on Spider's side, which its docs say don't use credits.
Spider and UpRock side by side
Spider's numbers come from its pricing page and docs, checked 21 Oct 2026.
| Spider | UpRock | |
|---|---|---|
| Price per 1,000 pages | $0.02 to $0.16 by Spider's averages (bandwidth $1 per GB plus compute); more with residential proxies or zero data retention | $0.50 by card or USDC, $0.40 with UPT |
| Minimum top-up | $5, credits don't expire | Prepaid credits |
| Failed pages | "$0" per the pricing page; a page-level 403 "consumes credits" per the error-codes page | Not documented yet |
| Whole-site crawl, search, screenshots, streaming | Yes | No. One page per call |
| Where requests come from | Residential and ISP proxy pools, or your own server if self-hosted | Devices whose owners opted in |
| Countries | 199 | 63, or a region |
| Open source | spider-rs/spider, MIT | No. The device runtime (Fuel Engine) is |
| Cache | On by default, two-day freshness | A repeat request soon after can reuse the recent result |
| MCP server | Hosted, 21 tools | Yes, remote (streamable HTTP) |
| Video | No (only recordings of its own browser sessions) | Search public video by moment, $1 per 1,000 searches |
| Rate limits | 10,000 requests a minute pay-as-you-go | Not published yet |
Where Spider is better: price, speed, whole-site crawling, more countries and an open-source core. Where UpRock fits: a second, independent network for the pages Spider returns without content, and video.
The fallback, in TypeScript and Python
Every version call Spider's /scrape endpoint for markdown in its default smart mode, accept the page only when status is 2xx and there's content, treat 404 and 410 as the site's answer, and send everything else to UpRock. They call Spider's HTTP API directly rather than its SDKs, which wrap /scrape in a crawl call. UpRock gets one POST /v1/crawl request with your key in an Authorization: Bearer header; the MCP and CLI tabs do the same through UpRock's MCP server (TypeScript) or CLI (Python).
// Node 18+ env: SPIDER_API_KEY, UPROCK_API_KEYtype SpiderPage = { url: string; content: string; status: number; error: string | null;};async function viaSpider(url: string): Promise<SpiderPage> { const key = process.env.SPIDER_API_KEY; const res = await fetch("https://api.spider.cloud/scrape", { method: "POST", headers: { "Content-Type": "application/json", ...(key ? { Authorization: `Bearer ${key}` } : {}), }, body: JSON.stringify({ url, return_format: "markdown", request: "smart" }), }); // 401 bad key, 402 out of credits, 429 rate limited, 5xx Spider's side (not billed). if (!res.ok) throw new Error(`Spider ${res.status}: ${(await res.text()).slice(0, 200)}`); const [page] = (await res.json()) as SpiderPage[]; // always an array, even from /scrape return page;}async function viaUpRock(url: string, country: string) { // One call: UpRock loads the page on a real device in that country, tries a fresh device // every 7 s (up to 3 tries) and returns markdown. Failed calls aren't billed. const res = await fetch("https://edge.uprock.com/v1/crawl", { method: "POST", headers: { Authorization: `Bearer ${process.env.UPROCK_API_KEY}`, "Content-Type": "application/json", }, body: JSON.stringify({ url, country }), signal: AbortSignal.timeout(90_000), }); const page = await res.json(); if (!res.ok) throw new Error(`UpRock ${res.status} ${page.code}: ${page.error}`); return page.markdown ?? "";}export async function read(url: string, country = "US") { try { const page = await viaSpider(url); // Spider reports the site's answer per page. A 403 arrives inside an HTTP 200 with // status 403, empty content and error: null, so judge by status and content, not error. if (page.status >= 200 && page.status < 300 && page.content) { return { via: "spider", markdown: page.content }; } if (page.status === 404 || page.status === 410) throw new Error(`The page is gone (${page.status})`); console.warn( `Spider got status ${page.status} with ${page.content.length} chars; trying UpRock`, ); } catch (err) { if ((err as Error).message.startsWith("The page is gone")) throw err; console.warn(`${(err as Error).message}; trying UpRock`); } return { via: "uprock", markdown: await viaUpRock(url, country) };}// node read.ts https://developer.mozilla.org/en-US/docs/Web/HTTP DEconst [url = "https://developer.mozilla.org/en-US/docs/Web/HTTP", country = "US"] = process.argv.slice(2);const page = await read(url, country);console.log(page.via, page.markdown.slice(0, 200));# pip install requests env: SPIDER_API_KEY, UPROCK_API_KEYimport osimport sysimport requestsdef via_spider(url: str) -> dict: key = os.environ.get("SPIDER_API_KEY") res = requests.post( "https://api.spider.cloud/scrape", headers={"Authorization": f"Bearer {key}"} if key else {}, json={"url": url, "return_format": "markdown", "request": "smart"}, timeout=120, ) # 401 bad key, 402 out of credits, 429 rate limited, 5xx Spider's side (not billed). if not res.ok: raise RuntimeError(f"Spider {res.status_code}: {res.text[:200]}") return res.json()[0] # always an array, even from /scrapedef via_uprock(url: str, country: str) -> str: # One call: UpRock loads the page on a real device in that country, tries a fresh device # every 7 s (up to 3 tries) and returns markdown. Failed calls aren't billed. res = requests.post( "https://edge.uprock.com/v1/crawl", headers={"Authorization": f"Bearer {os.environ['UPROCK_API_KEY']}"}, json={"url": url, "country": country}, timeout=90, ) page = res.json() if not res.ok: raise RuntimeError(f"UpRock {res.status_code} {page.get('code')}: {page.get('error')}") return page.get("markdown", "")def read(url: str, country: str = "US") -> dict: try: page = via_spider(url) except Exception as err: print(f"{err}; trying UpRock", file=sys.stderr) else: status, content = page.get("status") or 0, page.get("content") or "" # Spider reports the site's answer per page. A 403 arrives inside an HTTP 200 with # status 403, empty content and error: None, so judge by status and content, not error. if 200 <= status < 300 and content: return {"via": "spider", "markdown": content} if status in (404, 410): raise LookupError(f"The page is gone ({status})") print( f"Spider got status {status} with {len(content)} chars; trying UpRock", file=sys.stderr, ) return {"via": "uprock", "markdown": via_uprock(url, country)}if __name__ == "__main__": # python read.py https://developer.mozilla.org/en-US/docs/Web/HTTP DE url = sys.argv[1] if len(sys.argv) > 1 else "https://developer.mozilla.org/en-US/docs/Web/HTTP" country = sys.argv[2] if len(sys.argv) > 2 else "US" page = read(url, country) print(page["via"], page["markdown"][:200])// npm i @modelcontextprotocol/sdk (Node 18+) env: SPIDER_API_KEY, UPROCK_API_KEYimport { Client } from "@modelcontextprotocol/sdk/client/index.js";import { StreamableHTTPClientTransport } from "@modelcontextprotocol/sdk/client/streamableHttp.js";const uprock = new Client({ name: "spider-fallback", version: "1.0.0" });await uprock.connect( new StreamableHTTPClientTransport( new URL("https://mcp.uprock.ai/mcp"), { requestInit: { headers: { Authorization: `Bearer ${process.env.UPROCK_API_KEY}` } } }, ),);type SpiderPage = { url: string; content: string; status: number; error: string | null;};const textOf = (content: unknown) => (content as { type: string; text?: string }[]).find((c) => c.type === "text")?.text ?? "";async function viaSpider(url: string): Promise<SpiderPage> { const key = process.env.SPIDER_API_KEY; const res = await fetch("https://api.spider.cloud/scrape", { method: "POST", headers: { "Content-Type": "application/json", ...(key ? { Authorization: `Bearer ${key}` } : {}), }, body: JSON.stringify({ url, return_format: "markdown", request: "smart" }), }); // 401 bad key, 402 out of credits, 429 rate limited, 5xx Spider's side (not billed). if (!res.ok) throw new Error(`Spider ${res.status}: ${(await res.text()).slice(0, 200)}`); const [page] = (await res.json()) as SpiderPage[]; // always an array, even from /scrape return page;}async function viaUpRock(url: string, country: string) { const res = await uprock.callTool( { name: "crawl_fetch", arguments: { url, country } }, undefined, { timeout: 300_000 }, ); if (res.isError) throw new Error(`UpRock: ${textOf(res.content)}`); const uri = JSON.parse(textOf(res.content)).content?.markdown?.resource; if (!uri) throw new Error(`UpRock: no markdown for ${url}`); return textOf( (await uprock.callTool({ name: "resource_fetch", arguments: { uri } })).content, );}export async function read(url: string, country = "US") { try { const page = await viaSpider(url); // Spider reports the site's answer per page. A 403 arrives inside an HTTP 200 with // status 403, empty content and error: null, so judge by status and content, not error. if (page.status >= 200 && page.status < 300 && page.content) { return { via: "spider", markdown: page.content }; } if (page.status === 404 || page.status === 410) throw new Error(`The page is gone (${page.status})`); console.warn( `Spider got status ${page.status} with ${page.content.length} chars; trying UpRock`, ); } catch (err) { if ((err as Error).message.startsWith("The page is gone")) throw err; console.warn(`${(err as Error).message}; trying UpRock`); } return { via: "uprock", markdown: await viaUpRock(url, country) };}// node read.ts https://developer.mozilla.org/en-US/docs/Web/HTTP DEconst [url = "https://developer.mozilla.org/en-US/docs/Web/HTTP", country = "US"] = process.argv.slice(2);try { const page = await read(url, country); console.log(page.via, page.markdown.slice(0, 200));} finally { await uprock.close();}# pip install requests; npm i -g uprock env: SPIDER_API_KEY, UPROCK_API_KEYimport jsonimport osimport subprocessimport sysimport requestsdef via_spider(url: str) -> dict: key = os.environ.get("SPIDER_API_KEY") res = requests.post( "https://api.spider.cloud/scrape", headers={"Authorization": f"Bearer {key}"} if key else {}, json={"url": url, "return_format": "markdown", "request": "smart"}, timeout=120, ) # 401 bad key, 402 out of credits, 429 rate limited, 5xx Spider's side (not billed). if not res.ok: raise RuntimeError(f"Spider {res.status_code}: {res.text[:200]}") return res.json()[0] # always an array, even from /scrapedef via_uprock(url: str, country: str) -> str: run = subprocess.run( ["uprock", "ai", "crawl", url, "--country", country, "--content"], capture_output=True, text=True, timeout=330, ) if run.returncode != 0: raise RuntimeError(f"UpRock: {run.stderr.strip()}") return json.loads(run.stdout).get("inlined_markdown", "")def read(url: str, country: str = "US") -> dict: try: page = via_spider(url) except Exception as err: print(f"{err}; trying UpRock", file=sys.stderr) else: status, content = page.get("status") or 0, page.get("content") or "" # Spider reports the site's answer per page. A 403 arrives inside an HTTP 200 with # status 403, empty content and error: None, so judge by status and content, not error. if 200 <= status < 300 and content: return {"via": "spider", "markdown": content} if status in (404, 410): raise LookupError(f"The page is gone ({status})") print( f"Spider got status {status} with {len(content)} chars; trying UpRock", file=sys.stderr, ) return {"via": "uprock", "markdown": via_uprock(url, country)}if __name__ == "__main__": # python read.py https://developer.mozilla.org/en-US/docs/Web/HTTP DE url = sys.argv[1] if len(sys.argv) > 1 else "https://developer.mozilla.org/en-US/docs/Web/HTTP" country = sys.argv[2] if len(sys.argv) > 2 else "US" page = read(url, country) print(page["via"], page["markdown"][:200])Tested 21 Oct 2026 on Spider's keyless tier (the code sends your key when SPIDER_API_KEY is set). Spider returned example.com itself, and the 403 test URL went to UpRock whenever Spider reported its real status, which it didn't twice, as the table above shows. On 6 Nov 2026 we ran the HTTP and MCP versions again with an UpRock API key in an Authorization header, with an invalid Spider key: the 403 test URL went to UpRock.
FAQ
Is UpRock cheaper than Spider?
No. Spider bills bandwidth ($1 per GB) plus compute, and its own averages put a page well under a cent; UpRock is $0.50 per 1,000 pages. Use Spider for the bulk of your pages and UpRock for the ones Spider can't return.
How do I know Spider failed on a page?
Look at each page's status and content, not the error field. In our tests a page the site refused came back inside an HTTP 200, with status 403 and error set to null, and a page Spider couldn't reach in plain HTTP mode came back as status 526 after 40 seconds. Also note that the same refused test page twice came back looking like a success, so a status check can't catch every case.
Does Spider charge for failed pages?
Its pricing page says failed requests cost $0, and server-side errors don't use credits. Its error-codes page says a page-level 403 does consume credits, and the responses we got for refused pages carried a non-zero cost field. Check your own balance if it matters.
Which covers more countries?
Spider, which lists proxies in 199 countries. UpRock covers 63, or a region such as EU or APAC.
Can I self-host Spider instead?
Yes. The crawler is open source under MIT (spider-rs/spider), written in Rust. Self-hosted, it fetches from your own server's IP address unless you add proxies, which is another case for a second network.
Other use cases to explore
- Add a fallback to a crawler you run: hand the requests Crawlee gives up on to a device in the country you choose.
- Decide which failed pages are worth retrying: a diagnosis table from status code to the legitimate fix, and when to stop.
- See a page as it looks in another country: one store page from five countries, with five prices, two languages and a cookie banner in one country only.
- Search public video by moment: find the second something is shown or said, across platforms.
References
- Spider, Pricing, checked 21 Oct 2026
- Spider, llms.txt (credits, billing notes), checked 21 Oct 2026
- Spider docs, API reference, checked 21 Oct 2026
- Spider docs, Rate limits, checked 21 Oct 2026
- Spider, Proxy locations, checked 21 Oct 2026
- Spider, MCP server, checked 21 Oct 2026
- spider-rs/spider on GitHub (MIT), checked 21 Oct 2026
- UpRock, everything in one file (API, CLI, MCP, pricing), checked 21 Oct 2026
Get started
Add UpRock in one paste.
Paste this into Claude Code, Codex, Cursor or any agent with a terminal. It reads our docs, sets up the MCP server or CLI, and tells you the cost before it spends anything.
Set up UpRock for me: read https://uprock.ai/agents.md, connect the UpRock MCP server, and run one test crawl. Tell me what it costs before you spend anything.