How do I add a fallback for Exa contents errors?
Keep Exa for finding pages, and when its contents call can't return one, retry that URL on UpRock, which loads it live on a real device in the country you choose. Exa doesn't throw an error for these, it lists each failed URL in a statuses field inside a normal response, so the fallback reads that list.
Updated · Read as markdown
What Exa does well
Exa runs its own search index. Its docs say it tracks over 2 trillion URLs and serves 100 billion pages, and it searches them by meaning: you can describe the page you want in plain language instead of guessing keywords. Results can be narrowed to categories like companies, people, news and financial reports, filtered by date and domain, and returned with page text, highlights or summaries. A plain auto search takes about a second, and the lite type about 450 ms.
On top of search there are Deep Search, Answer, Websets, Monitors and an official MCP server you can use without a key at a limited rate. For finding pages, Exa does something UpRock doesn't try to do. Keep it.
How Exa reports a page it couldn't fetch
When you ask /contents for a list of URLs, Exa doesn't fail the request because one page didn't load. It returns HTTP 200, leaves the failed URL out of results, and adds an entry to statuses with an error tag and the target's HTTP status code. Code that only catches exceptions never notices.
Here's every tag Exa lists today, and what to do with each:
statuses[].error.tag | Exa's description | Retry on UpRock? |
|---|---|---|
CRAWL_HTTP_{status}, e.g. CRAWL_HTTP_403 | The target returned an HTTP error | Yes for 401, 403, 429 and 5xx. Not for 404 or 410 |
CRAWL_TIMEOUT | The crawl timed out while fetching the page | Yes |
CRAWL_LIVECRAWL_TIMEOUT | Live retrieval took longer than your livecrawlTimeout | Yes, or raise the timeout (up to 90 s) |
SOURCE_NOT_AVAILABLE | Access to the source is forbidden or the source is unavailable | Yes |
CRAWL_UNKNOWN_ERROR | The crawl failed for another reason | Yes |
CRAWL_NOT_FOUND | The target page was not found | No |
UNSUPPORTED_URL | The URL scheme isn't supported for fetching | No |
Exa's crawler also doesn't try to get past logins, paywalls or CAPTCHAs. Neither does UpRock: it fetches public pages only, so a URL that needs a login fails on both.
Cached or live, and from where
Exa serves page text from its cache when it has a copy, and fetches the page when it doesn't. You control that with maxAgeHours: leave it out for cache-then-fetch, 0 to always fetch fresh, -1 for cache only. The lite search type is cache-only. Its search takes a userLocation to localize results, but /contents has no country option.
UpRock fetches the page when you call it, in a real browser on a desktop or mobile device in the country you pass (a repeat request for the same page and country shortly after can return that recent result). That's a reason for the fallback beyond errors, too: if a page shows different prices, stock or content by country, fetch it from that country.
Exa and UpRock side by side
Exa numbers come from its pricing and docs pages, checked 24 Oct 2026.
| Exa | UpRock | |
|---|---|---|
| Page text, per 1,000 pages | $1.00 per content type (text and highlights together cost $2.00) | $0.50 by card or USDC, $0.40 with UPT |
| Search, per 1,000 | $7 for auto with up to 10 results and their contents; $4 for lite (cache only); $1 per extra result above 10 | Not compared here. Use Exa for search |
| Own index, search by meaning | Yes | No |
| Fresh vs cached | Cached by default; maxAgeHours: 0 fetches fresh | Fetched when you call; a repeat request soon after can reuse that result |
| Where pages are fetched from | As "ExaUser"; location not stated | Real devices in the country you choose |
| Country | userLocation localizes search; contents has none | Per request: 63 countries, or a region |
| Rate limits | 100 requests a second for contents, 10 for search | Not published yet |
| MCP server | Official, remote, keyless tier | Yes, remote (streamable HTTP) |
| Video | Finds video and podcast transcripts by text | Search public video by moment, $1 per 1,000 searches |
| To try it | $10 of credits a month at no cost | Prepaid credits |
Where Exa is better: finding pages by meaning, its own large index, speed, high rate limits and monthly credits to try it. Where UpRock fits: the URLs Exa's statuses mark as failed, a page as it looks in a chosen country, and video moments.
The fallback, in TypeScript and Python
Every version ask Exa for the text of a batch of URLs, pick out the ones whose status tag is worth retrying (urlsToRetry, the table above as code), and fetch those on UpRock in parallel. UpRock gets one POST /v1/crawl request with your key in an Authorization: Bearer header; the MCP and CLI tabs do the same through UpRock's MCP server (TypeScript) or CLI (Python).
// npm i exa-js (Node 18+) env: EXA_API_KEY, UPROCK_API_KEYimport { Exa } from "exa-js";const exa = new Exa(process.env.EXA_API_KEY);// Exa status tags worth a second try (exa.ai/docs/admin/error-codes, checked 19 Oct 2026).// Not retried: CRAWL_NOT_FOUND, CRAWL_HTTP_404/410 and UNSUPPORTED_URL.const RETRY_TAG = /^(CRAWL_HTTP_(401|403|429|5\d\d)|CRAWL_TIMEOUT|CRAWL_LIVECRAWL_TIMEOUT|SOURCE_NOT_AVAILABLE|CRAWL_UNKNOWN_ERROR)$/;async function viaUpRock(url: string, country: string) { // One call: UpRock loads the page on a real device in that country, tries a fresh device // every 7 s (up to 3 tries) and returns markdown. Failed calls aren't billed. const res = await fetch("https://edge.uprock.com/v1/crawl", { method: "POST", headers: { Authorization: `Bearer ${process.env.UPROCK_API_KEY}`, "Content-Type": "application/json", }, body: JSON.stringify({ url, country }), signal: AbortSignal.timeout(90_000), }); const page = await res.json(); if (!res.ok) throw new Error(`UpRock ${res.status} ${page.code}: ${page.error}`); return page.markdown ?? "";}/** Which URLs to send to UpRock, from an Exa /contents response. */export function urlsToRetry( urls: string[], res: { results: { id: string }[]; statuses?: { id: string; status: string; error?: { tag?: string } | null }[]; },) { const statusOf = new Map((res.statuses ?? []).map((s) => [s.id, s])); const returned = new Set(res.results.map((r) => r.id)); return urls.filter((url) => { const s = statusOf.get(url); if (s?.status === "error") return RETRY_TAG.test(s.error?.tag ?? ""); return !returned.has(url) && !s; // no result and no status at all: try again });}export async function contents(urls: string[], country = "US") { const text = new Map<string, { text: string; via: string }>(); let retry = urls; try { const res = await exa.getContents(urls, { text: true }); for (const r of res.results) text.set(r.id, { text: r.text, via: "exa" }); retry = urlsToRetry(urls, res); } catch (err) { // Request-level failures (bad key, no credits, rate limit) throw; URL-level ones don't. console.warn(`Exa failed (${(err as Error).message}); sending every URL to UpRock`); } const retried = await Promise.allSettled(retry.map((url) => viaUpRock(url, country))); retried.forEach((r, i) => { if (r.status === "fulfilled") text.set(retry[i], { text: r.value, via: "uprock" }); else console.warn(`UpRock also failed for ${retry[i]}: ${r.reason}`); }); return text;}// node contents.ts DE https://developer.mozilla.org/en-US/docs/Web/HTTP https://example.orgconst [country = "US", ...urls] = process.argv.slice(2);for (const [url, page] of await contents(urls, country)) console.log(page.via.padEnd(6), url, page.text.length, "chars");# pip install exa-py requests env: EXA_API_KEY, UPROCK_API_KEYimport osimport reimport sysfrom concurrent.futures import ThreadPoolExecutorimport requestsfrom exa_py import Exaexa = Exa(api_key=os.environ.get("EXA_API_KEY"))# Exa status tags worth a second try (exa.ai/docs/admin/error-codes, checked 19 Oct 2026).# Not retried: CRAWL_NOT_FOUND, CRAWL_HTTP_404/410 and UNSUPPORTED_URL.RETRY_TAG = re.compile( r"^(CRAWL_HTTP_(401|403|429|5\d\d)|CRAWL_TIMEOUT|CRAWL_LIVECRAWL_TIMEOUT|SOURCE_NOT_AVAILABLE|CRAWL_UNKNOWN_ERROR)$")def via_uprock(url: str, country: str) -> str: # One call: UpRock loads the page on a real device in that country, tries a fresh device # every 7 s (up to 3 tries) and returns markdown. Failed calls aren't billed. res = requests.post( "https://edge.uprock.com/v1/crawl", headers={"Authorization": f"Bearer {os.environ['UPROCK_API_KEY']}"}, json={"url": url, "country": country}, timeout=90, ) page = res.json() if not res.ok: raise RuntimeError(f"UpRock {res.status_code} {page.get('code')}: {page.get('error')}") return page.get("markdown", "")def urls_to_retry(urls: list[str], res) -> list[str]: """Which URLs to send to UpRock, from an Exa /contents response.""" status_of = {s.id: s for s in (res.statuses or [])} returned = {r.id for r in res.results} retry = [] for url in urls: s = status_of.get(url) if s is not None and s.status == "error": if RETRY_TAG.match((s.error.tag if s.error else "") or ""): retry.append(url) elif s is None and url not in returned: retry.append(url) # no result and no status at all: try again return retrydef contents(urls: list[str], country: str = "US") -> dict[str, dict]: pages, retry = {}, list(urls) try: res = exa.get_contents(urls, text=True) pages = {r.id: {"text": r.text, "via": "exa"} for r in res.results} retry = urls_to_retry(urls, res) except Exception as err: # Request-level failures (bad key, no credits, rate limit) raise; URL-level ones don't. print(f"Exa failed ({err}); sending every URL to UpRock", file=sys.stderr) with ThreadPoolExecutor(max_workers=8) as pool: futures = {url: pool.submit(via_uprock, url, country) for url in retry} for url, future in futures.items(): try: pages[url] = {"text": future.result(), "via": "uprock"} except Exception as err: print(f"UpRock also failed for {url}: {err}", file=sys.stderr) return pagesif __name__ == "__main__": # python contents.py DE https://developer.mozilla.org/en-US/docs/Web/HTTP https://example.org country, *urls = sys.argv[1:] or ["US"] for url, page in contents(urls, country).items(): print(page["via"].ljust(6), url, len(page["text"]), "chars")// npm i exa-js @modelcontextprotocol/sdk (Node 18+) env: EXA_API_KEY, UPROCK_API_KEYimport { Exa } from "exa-js";import { Client } from "@modelcontextprotocol/sdk/client/index.js";import { StreamableHTTPClientTransport } from "@modelcontextprotocol/sdk/client/streamableHttp.js";const exa = new Exa(process.env.EXA_API_KEY);const uprock = new Client({ name: "exa-fallback", version: "1.0.0" });await uprock.connect( new StreamableHTTPClientTransport( new URL("https://mcp.uprock.ai/mcp"), { requestInit: { headers: { Authorization: `Bearer ${process.env.UPROCK_API_KEY}` } } }, ),);// Exa status tags worth a second try (exa.ai/docs/admin/error-codes, checked 19 Oct 2026).// Not retried: CRAWL_NOT_FOUND, CRAWL_HTTP_404/410 and UNSUPPORTED_URL.const RETRY_TAG = /^(CRAWL_HTTP_(401|403|429|5\d\d)|CRAWL_TIMEOUT|CRAWL_LIVECRAWL_TIMEOUT|SOURCE_NOT_AVAILABLE|CRAWL_UNKNOWN_ERROR)$/;const textOf = (content: unknown) => (content as { type: string; text?: string }[]).find((c) => c.type === "text")?.text ?? "";async function viaUpRock(url: string, country: string) { const res = await uprock.callTool( { name: "crawl_fetch", arguments: { url, country } }, undefined, { timeout: 300_000 }, ); if (res.isError) throw new Error(textOf(res.content)); const uri = JSON.parse(textOf(res.content)).content?.markdown?.resource; if (!uri) throw new Error("no markdown for this page"); const md = await uprock.callTool({ name: "resource_fetch", arguments: { uri } }); return textOf(md.content);}/** Which URLs to send to UpRock, from an Exa /contents response. */export function urlsToRetry( urls: string[], res: { results: { id: string }[]; statuses?: { id: string; status: string; error?: { tag?: string } | null }[]; },) { const statusOf = new Map((res.statuses ?? []).map((s) => [s.id, s])); const returned = new Set(res.results.map((r) => r.id)); return urls.filter((url) => { const s = statusOf.get(url); if (s?.status === "error") return RETRY_TAG.test(s.error?.tag ?? ""); return !returned.has(url) && !s; // no result and no status at all: try again });}export async function contents(urls: string[], country = "US") { const text = new Map<string, { text: string; via: string }>(); let retry = urls; try { const res = await exa.getContents(urls, { text: true }); for (const r of res.results) text.set(r.id, { text: r.text, via: "exa" }); retry = urlsToRetry(urls, res); } catch (err) { // Request-level failures (bad key, no credits, rate limit) throw; URL-level ones don't. console.warn(`Exa failed (${(err as Error).message}); sending every URL to UpRock`); } const retried = await Promise.allSettled(retry.map((url) => viaUpRock(url, country))); retried.forEach((r, i) => { if (r.status === "fulfilled") text.set(retry[i], { text: r.value, via: "uprock" }); else console.warn(`UpRock also failed for ${retry[i]}: ${r.reason}`); }); return text;}// node contents.ts DE https://developer.mozilla.org/en-US/docs/Web/HTTP https://example.orgconst [country = "US", ...urls] = process.argv.slice(2);for (const [url, page] of await contents(urls, country)) console.log(page.via.padEnd(6), url, page.text.length, "chars");await uprock.close();# pip install exa-py; npm i -g uprock env: EXA_API_KEY, UPROCK_API_KEYimport jsonimport osimport reimport subprocessimport sysfrom concurrent.futures import ThreadPoolExecutorfrom exa_py import Exaexa = Exa(api_key=os.environ.get("EXA_API_KEY"))# Exa status tags worth a second try (exa.ai/docs/admin/error-codes, checked 19 Oct 2026).# Not retried: CRAWL_NOT_FOUND, CRAWL_HTTP_404/410 and UNSUPPORTED_URL.RETRY_TAG = re.compile( r"^(CRAWL_HTTP_(401|403|429|5\d\d)|CRAWL_TIMEOUT|CRAWL_LIVECRAWL_TIMEOUT|SOURCE_NOT_AVAILABLE|CRAWL_UNKNOWN_ERROR)$")def via_uprock(url: str, country: str) -> str: run = subprocess.run( ["uprock", "ai", "crawl", url, "--country", country, "--content"], capture_output=True, text=True, timeout=330, ) if run.returncode != 0: raise RuntimeError(run.stderr.strip()) return json.loads(run.stdout).get("inlined_markdown", "")def urls_to_retry(urls: list[str], res) -> list[str]: """Which URLs to send to UpRock, from an Exa /contents response.""" status_of = {s.id: s for s in (res.statuses or [])} returned = {r.id for r in res.results} retry = [] for url in urls: s = status_of.get(url) if s is not None and s.status == "error": if RETRY_TAG.match((s.error.tag if s.error else "") or ""): retry.append(url) elif s is None and url not in returned: retry.append(url) # no result and no status at all: try again return retrydef contents(urls: list[str], country: str = "US") -> dict[str, dict]: pages, retry = {}, list(urls) try: res = exa.get_contents(urls, text=True) pages = {r.id: {"text": r.text, "via": "exa"} for r in res.results} retry = urls_to_retry(urls, res) except Exception as err: # Request-level failures (bad key, no credits, rate limit) raise; URL-level ones don't. print(f"Exa failed ({err}); sending every URL to UpRock", file=sys.stderr) with ThreadPoolExecutor(max_workers=8) as pool: futures = {url: pool.submit(via_uprock, url, country) for url in retry} for url, future in futures.items(): try: pages[url] = {"text": future.result(), "via": "uprock"} except Exception as err: print(f"UpRock also failed for {url}: {err}", file=sys.stderr) return pagesif __name__ == "__main__": # python contents.py DE https://developer.mozilla.org/en-US/docs/Web/HTTP https://example.org country, *urls = sys.argv[1:] or ["US"] for url, page in contents(urls, country).items(): print(page["via"].ljust(6), url, len(page["text"]), "chars")Tested 19 Oct 2026 against the live UpRock API with an invalid Exa key (every URL goes to UpRock), and urlsToRetry against a response in Exa's documented statuses format. Without an Exa account, the success path on Exa's side follows its docs and SDK types. On 6 Nov 2026 we ran the HTTP and MCP versions again with an UpRock API key in an Authorization header: with an invalid Exa key, each fell back to UpRock and got the page.
FAQ
Does Exa throw an error when a URL can't be fetched?
No. Exa reports URL-level failures in the statuses field of a successful 200 response, and the failed URL is simply missing from results. Only request-level problems (a bad key, no credits, rate limits) throw.
Should I retry every failed Exa URL?
No. CRAWL_NOT_FOUND and CRAWL_HTTP_404 mean the page isn't there, and UNSUPPORTED_URL means the URL scheme can't be fetched. Retry HTTP errors like 403 and 429, timeouts, SOURCE_NOT_AVAILABLE and CRAWL_UNKNOWN_ERROR.
Isn't maxAgeHours 0 the same as a live fetch?
It forces Exa to fetch the page fresh instead of using its cache. The difference with UpRock is where the fetch happens. UpRock loads the page on a real device in the country you pick, while Exa's docs say its user-requested fetches go out as ExaUser and don't say from where.
Can UpRock do what Exa search does?
No. Exa searches its own index by meaning, with categories like companies, people and financial reports. UpRock has no index; it fetches the pages you give it. Use Exa to find pages, and UpRock to fetch the ones Exa couldn't.
How much does the fallback add?
Exa bills contents at $1 per 1,000 pages per content type. UpRock costs $0.50 per 1,000 pages by card or USDC, and you only call it for the URLs Exa reported as failed.
Other use cases to explore
- See a page as it looks in another country: one store page from five countries, with five prices, two languages and a cookie banner in one country only.
- Add a fallback to a crawler you host: send only the pages your server can't load to a device in the country you choose.
- Estimate what 1,000 pages will cost: token counts for six real pages, and the length where a flat per-page price is cheaper.
- Search public video by moment: find the second something is shown or said, across platforms.
References
- Exa, Pricing, checked 24 Oct 2026
- Exa docs, Pricing details (per endpoint and content type), checked 24 Oct 2026
- Exa docs, Error codes (content fetch status tags), checked 19 Oct 2026
- Exa docs, Contents quickstart (maxAgeHours, cached vs fresh), checked 19 Oct 2026
- Exa docs, Get contents API reference, checked 19 Oct 2026
- Exa docs, Search API reference (userLocation, categories), checked 19 Oct 2026
- Exa docs, FAQ (index size), checked 19 Oct 2026
- Exa docs, Rate limits and billing, checked 24 Oct 2026
- Exa crawler (how Exa fetches pages), checked 19 Oct 2026
- Exa docs, MCP server, checked 19 Oct 2026
- UpRock, everything in one file (API, CLI, MCP, pricing), checked 24 Oct 2026
Get started
Add UpRock in one paste.
Paste this into Claude Code, Codex, Cursor or any agent with a terminal. It reads our docs, sets up the MCP server or CLI, and tells you the cost before it spends anything.
Set up UpRock for me: read https://uprock.ai/agents.md, connect the UpRock MCP server, and run one test crawl. Tell me what it costs before you spend anything.