How do I add a backup to Bright Data's Web Unlocker?
Keep Bright Data as your primary and send UpRock the requests its Web Unlocker gives up on, like a site that keeps refusing, a page that never finishes loading or a target Bright Data has throttled. UpRock loads the page on an opted-in device in the country you pick for $0.50 per 1,000 pages, and the fallback leaves alone what Bright Data refuses on purpose, such as pages a site's robots.txt rules out.
Updated · Read as markdown
What Bright Data does well
Bright Data is one of the largest web data companies there is. It sells proxy networks (it claims over 400 million residential IPs in 195 countries), Web Unlocker for single pages, a SERP API for Google, Bing and other engines, a Browser API that works with Playwright and Puppeteer, a catalog of ready-made scrapers, and datasets you can buy outright. It's ISO 27001 and SOC 2 Type II certified, offers an uptime SLA and round-the-clock support, and has an official MCP server with 5,000 free requests a month.
If you run large jobs, need a city or ASN, or have compliance teams who want certificates, Bright Data is built for that. Keep it.
Two networks of opted-in devices
It's worth being clear about what's different, because it's less than you might expect. Bright Data's residential IPs come from consumer devices: app owners add Bright Data's SDK to their apps, and users opt in in exchange for an ad-free or upgraded app. UpRock also runs on devices whose owners opted in, by installing the UpRock app or node and earning UPT for the work their device does.
So the case for a backup isn't "real devices versus proxies". It's these:
- A second, independent network. When Web Unlocker gives up on a page, the same request through UpRock goes out through different devices and different software.
- Who can buy. Web Unlocker needs no verification but honors robots.txt in that mode. Residential proxies are for verified companies only, and Bright Data's docs say personal email addresses can't be approved. UpRock needs an API key.
- One price per page. $0.50 per 1,000 pages by card or USDC, $0.40 with UPT, without plans or minimum spends.
Reading a Web Unlocker failure
Through the Direct API, the HTTP response is 200 OK as soon as the request reaches the unlocker. What happened to your page is in the headers: x-brd-error with a message, x-brd-error-code with a code, and x-brd-status-code with the result status. Code that checks only the HTTP status misses every failure.
Bright Data's own docs sort the codes into ones that clear on a retry and ones that don't. Here's how we'd route each:
x-brd-error-code | What Bright Data says it means | What to do |
|---|---|---|
reject_block, resolve_failed_* | A CAPTCHA or protection page came back instead of the content | Retry on Bright Data first (each attempt uses a different peer), then UpRock |
http_status | The site answered with a blocking status, including its own 429 | Send to UpRock |
navigation_timeout, domcontentloaded_event_timeout, expect_element | The page didn't finish loading, or an element the unlocker waits for never appeared | Send to UpRock |
document_load_failed | Navigation failed before a page arrived | Send to UpRock (unless it's a TLS error, see below) |
sr_rate_limit (HTTP 429) | Bright Data throttled this target after its success rate fell | Send to UpRock |
ub_bad_endpoint_robots | The site's robots.txt disallows the path in no-verification mode | Respect it |
net_err_cert_* | The site's TLS certificate is expired or doesn't match | Don't retry: every client sees the same |
| 403, no code | The target is blocked by Bright Data policy (for example government or adult sites) | Respect it |
| Outer 401 or 400 | A bad API key, zone name or request body | Fix the request |
Bright Data and UpRock side by side
Bright Data numbers come from its pricing pages and docs, checked 21 Oct 2026.
| Bright Data | UpRock | |
|---|---|---|
| Price per 1,000 pages | Web Unlocker: $1.50 pay-as-you-go; $1.30, $1.10 and $1.00 on the $499, $999 and $1,999 monthly plans | $0.50 by card or USDC, $0.40 with UPT |
| Residential proxies | $4 to $8 per GB pay-as-you-go (the page shows a 50% promotion), down to $2.50 on plans | Included: every page is one credit |
| To try it | 5,000 requests a month at no cost | Prepaid credits |
| Who can buy | Web Unlocker: anyone. Residential: verified companies only | Anyone with an API key |
| Where requests come from | Residential IPs from opted-in app users, plus datacenter, ISP and mobile pools | Devices whose owners opted in |
| Countries | 195 for residential; city, state, ZIP and ASN on residential | 63, or a region |
| Search, scrapers, datasets, browser | Yes: SERP API, ready-made scrapers, datasets, Browser API | No. One page per call |
| MCP server | Official, remote or local, 32 tools | Yes, remote (streamable HTTP) |
| Video | Video Search API in beta, behind a KYC review, price not published | Search public video by moment, $1 per 1,000 searches |
| Certifications, SLA | ISO 27001, SOC 2 Type II, uptime SLA | Not published |
Where Bright Data is better: coverage, scale, targeting below the country level, a full product range, certifications and support, and a monthly allowance to try it. Where UpRock fits: a second, independent network when Web Unlocker gives up, one flat price per page, no company verification, and video search you can use today.
The fallback, in TypeScript and Python
Every version call Web Unlocker's Direct API for markdown, read the error headers, retry once on Bright Data for the codes it calls retryable, and send the rest of the routable failures to UpRock. Anything Bright Data refuses on purpose raises an error instead. UpRock gets one POST /v1/crawl request with your key in an Authorization: Bearer header; the MCP and CLI tabs do the same through UpRock's MCP server (TypeScript) or CLI (Python).
// Node 18+ env: BRIGHTDATA_API_KEY, BRIGHTDATA_ZONE, UPROCK_API_KEY// Web Unlocker error codes (docs.brightdata.com, error codes, checked 21 Oct 2026).// Bright Data says these clear on a retry, since each attempt uses a different peer.const RETRY_ON_BRIGHT_DATA = /^(reject_block|resolve_failed_.*)$/;// The site refused, timed out or never loaded, or Bright Data throttled the target.const TRY_UPROCK = new Set([ "http_status", "navigation_timeout", "domcontentloaded_event_timeout", "expect_element", "document_load_failed", "sr_rate_limit",]);// Everything else, including ub_bad_endpoint_robots (the site's robots.txt says no),// policy blocks and broken TLS certificates, is an answer: don't route around it.async function viaBrightData(url: string, country: string) { const res = await fetch("https://api.brightdata.com/request", { method: "POST", headers: { Authorization: `Bearer ${process.env.BRIGHTDATA_API_KEY}`, "Content-Type": "application/json", }, body: JSON.stringify({ zone: process.env.BRIGHTDATA_ZONE, // your Web Unlocker zone url, format: "raw", data_format: "markdown", country: country.toLowerCase(), }), signal: AbortSignal.timeout(180_000), // Bright Data asks for at least 180 s }); const code = res.headers.get("x-brd-error-code") ?? res.headers.get("x-brd-err-code"); const error = res.headers.get("x-brd-error"); // The outer status is 200 once the request reaches the unlocker; 401 and 400 mean a bad key, // zone or request body, with no x-brd headers. if (!res.ok) return { error: `HTTP ${res.status}: ${await res.text()}`, code: "request_rejected", }; if (error) return { error, code: code ?? "unknown" }; return { markdown: await res.text() };}async function viaUpRock(url: string, country: string) { // One call: UpRock loads the page on a real device in that country, tries a fresh device // every 7 s (up to 3 tries) and returns markdown. Failed calls aren't billed. const res = await fetch("https://edge.uprock.com/v1/crawl", { method: "POST", headers: { Authorization: `Bearer ${process.env.UPROCK_API_KEY}`, "Content-Type": "application/json", }, body: JSON.stringify({ url, country }), signal: AbortSignal.timeout(90_000), }); const page = await res.json(); if (!res.ok) throw new Error(`UpRock ${res.status} ${page.code}: ${page.error}`); return page.markdown ?? "";}export async function read(url: string, country = "US") { let result = await viaBrightData(url, country); if (result.code && RETRY_ON_BRIGHT_DATA.test(result.code)) result = await viaBrightData(url, country); if (result.markdown !== undefined) return { via: "brightdata", markdown: result.markdown }; const { code, error } = result; if ( !TRY_UPROCK.has(code ?? "") && !RETRY_ON_BRIGHT_DATA.test(code ?? "") && code !== "request_rejected" ) { throw new Error(`Bright Data: ${error} (${code})`); } console.warn(`Bright Data: ${error} (${code}); trying UpRock`); return { via: "uprock", markdown: await viaUpRock(url, country) };}// node read.ts https://developer.mozilla.org/en-US/docs/Web/HTTP DEconst [url = "https://developer.mozilla.org/en-US/docs/Web/HTTP", country = "US"] = process.argv.slice(2);const page = await read(url, country);console.log(page.via, page.markdown.slice(0, 200));# pip install requests env: BRIGHTDATA_API_KEY, BRIGHTDATA_ZONE, UPROCK_API_KEYimport osimport reimport sysimport requests# Web Unlocker error codes (docs.brightdata.com, error codes, checked 21 Oct 2026).# Bright Data says these clear on a retry, since each attempt uses a different peer.RETRY_ON_BRIGHT_DATA = re.compile(r"^(reject_block|resolve_failed_.*)$")# The site refused, timed out or never loaded, or Bright Data throttled the target.TRY_UPROCK = { "http_status", "navigation_timeout", "domcontentloaded_event_timeout", "expect_element", "document_load_failed", "sr_rate_limit",}# Everything else, including ub_bad_endpoint_robots (the site's robots.txt says no),# policy blocks and broken TLS certificates, is an answer: don't route around it.def via_bright_data(url: str, country: str) -> dict: res = requests.post( "https://api.brightdata.com/request", headers={"Authorization": f"Bearer {os.environ.get('BRIGHTDATA_API_KEY')}"}, json={ "zone": os.environ.get("BRIGHTDATA_ZONE"), "url": url, "format": "raw", "data_format": "markdown", "country": country.lower(), }, timeout=180, # Bright Data asks for at least 180 s ) code = res.headers.get("x-brd-error-code") or res.headers.get("x-brd-err-code") error = res.headers.get("x-brd-error") # The outer status is 200 once the request reaches the unlocker; 401 and 400 mean a bad # key, zone or request body, with no x-brd headers. if not res.ok: return { "error": f"HTTP {res.status_code}: {res.text.strip()}", "code": "request_rejected", } if error: return {"error": error, "code": code or "unknown"} return {"markdown": res.text}def via_uprock(url: str, country: str) -> str: # One call: UpRock loads the page on a real device in that country, tries a fresh device # every 7 s (up to 3 tries) and returns markdown. Failed calls aren't billed. res = requests.post( "https://edge.uprock.com/v1/crawl", headers={"Authorization": f"Bearer {os.environ['UPROCK_API_KEY']}"}, json={"url": url, "country": country}, timeout=90, ) page = res.json() if not res.ok: raise RuntimeError(f"UpRock {res.status_code} {page.get('code')}: {page.get('error')}") return page.get("markdown", "")def read(url: str, country: str = "US") -> dict: result = via_bright_data(url, country) if RETRY_ON_BRIGHT_DATA.match(result.get("code") or ""): result = via_bright_data(url, country) if "markdown" in result: return {"via": "brightdata", "markdown": result["markdown"]} code, error = result["code"], result["error"] if ( code not in TRY_UPROCK and not RETRY_ON_BRIGHT_DATA.match(code) and code != "request_rejected" ): raise RuntimeError(f"Bright Data: {error} ({code})") print(f"Bright Data: {error} ({code}); trying UpRock", file=sys.stderr) return {"via": "uprock", "markdown": via_uprock(url, country)}if __name__ == "__main__": # python read.py https://developer.mozilla.org/en-US/docs/Web/HTTP DE url = sys.argv[1] if len(sys.argv) > 1 else "https://developer.mozilla.org/en-US/docs/Web/HTTP" country = sys.argv[2] if len(sys.argv) > 2 else "US" page = read(url, country) print(page["via"], page["markdown"][:200])// npm i @modelcontextprotocol/sdk (Node 18+) env: BRIGHTDATA_API_KEY, BRIGHTDATA_ZONE, UPROCK_API_KEYimport { Client } from "@modelcontextprotocol/sdk/client/index.js";import { StreamableHTTPClientTransport } from "@modelcontextprotocol/sdk/client/streamableHttp.js";const uprock = new Client({ name: "brightdata-fallback", version: "1.0.0" });await uprock.connect( new StreamableHTTPClientTransport( new URL("https://mcp.uprock.ai/mcp"), { requestInit: { headers: { Authorization: `Bearer ${process.env.UPROCK_API_KEY}` } } }, ),);// Web Unlocker error codes (docs.brightdata.com, error codes, checked 21 Oct 2026).// Bright Data says these clear on a retry, since each attempt uses a different peer.const RETRY_ON_BRIGHT_DATA = /^(reject_block|resolve_failed_.*)$/;// The site refused, timed out or never loaded, or Bright Data throttled the target.const TRY_UPROCK = new Set([ "http_status", "navigation_timeout", "domcontentloaded_event_timeout", "expect_element", "document_load_failed", "sr_rate_limit",]);// Everything else, including ub_bad_endpoint_robots (the site's robots.txt says no),// policy blocks and broken TLS certificates, is an answer: don't route around it.const textOf = (content: unknown) => (content as { type: string; text?: string }[]).find((c) => c.type === "text")?.text ?? "";async function viaBrightData(url: string, country: string) { const res = await fetch("https://api.brightdata.com/request", { method: "POST", headers: { Authorization: `Bearer ${process.env.BRIGHTDATA_API_KEY}`, "Content-Type": "application/json", }, body: JSON.stringify({ zone: process.env.BRIGHTDATA_ZONE, // your Web Unlocker zone url, format: "raw", data_format: "markdown", country: country.toLowerCase(), }), signal: AbortSignal.timeout(180_000), // Bright Data asks for at least 180 s }); const code = res.headers.get("x-brd-error-code") ?? res.headers.get("x-brd-err-code"); const error = res.headers.get("x-brd-error"); // The outer status is 200 once the request reaches the unlocker; 401 and 400 mean a bad key, // zone or request body, with no x-brd headers. if (!res.ok) return { error: `HTTP ${res.status}: ${await res.text()}`, code: "request_rejected", }; if (error) return { error, code: code ?? "unknown" }; return { markdown: await res.text() };}async function viaUpRock(url: string, country: string) { const res = await uprock.callTool( { name: "crawl_fetch", arguments: { url, country } }, undefined, { timeout: 300_000 }, ); if (res.isError) throw new Error(`UpRock: ${textOf(res.content)}`); const uri = JSON.parse(textOf(res.content)).content?.markdown?.resource; if (!uri) throw new Error(`UpRock: no markdown for ${url}`); return textOf( (await uprock.callTool({ name: "resource_fetch", arguments: { uri } })).content, );}export async function read(url: string, country = "US") { let result = await viaBrightData(url, country); if (result.code && RETRY_ON_BRIGHT_DATA.test(result.code)) result = await viaBrightData(url, country); if (result.markdown !== undefined) return { via: "brightdata", markdown: result.markdown }; const { code, error } = result; if ( !TRY_UPROCK.has(code ?? "") && !RETRY_ON_BRIGHT_DATA.test(code ?? "") && code !== "request_rejected" ) { throw new Error(`Bright Data: ${error} (${code})`); } console.warn(`Bright Data: ${error} (${code}); trying UpRock`); return { via: "uprock", markdown: await viaUpRock(url, country) };}// node read.ts https://developer.mozilla.org/en-US/docs/Web/HTTP DEconst [url = "https://developer.mozilla.org/en-US/docs/Web/HTTP", country = "US"] = process.argv.slice(2);try { const page = await read(url, country); console.log(page.via, page.markdown.slice(0, 200));} finally { await uprock.close();}# pip install requests; npm i -g uprock env: BRIGHTDATA_API_KEY, BRIGHTDATA_ZONE, UPROCK_API_KEYimport jsonimport osimport reimport subprocessimport sysimport requests# Web Unlocker error codes (docs.brightdata.com, error codes, checked 21 Oct 2026).# Bright Data says these clear on a retry, since each attempt uses a different peer.RETRY_ON_BRIGHT_DATA = re.compile(r"^(reject_block|resolve_failed_.*)$")# The site refused, timed out or never loaded, or Bright Data throttled the target.TRY_UPROCK = { "http_status", "navigation_timeout", "domcontentloaded_event_timeout", "expect_element", "document_load_failed", "sr_rate_limit",}# Everything else, including ub_bad_endpoint_robots (the site's robots.txt says no),# policy blocks and broken TLS certificates, is an answer: don't route around it.def via_bright_data(url: str, country: str) -> dict: res = requests.post( "https://api.brightdata.com/request", headers={"Authorization": f"Bearer {os.environ.get('BRIGHTDATA_API_KEY')}"}, json={ "zone": os.environ.get("BRIGHTDATA_ZONE"), "url": url, "format": "raw", "data_format": "markdown", "country": country.lower(), }, timeout=180, # Bright Data asks for at least 180 s ) code = res.headers.get("x-brd-error-code") or res.headers.get("x-brd-err-code") error = res.headers.get("x-brd-error") # The outer status is 200 once the request reaches the unlocker; 401 and 400 mean a bad # key, zone or request body, with no x-brd headers. if not res.ok: return { "error": f"HTTP {res.status_code}: {res.text.strip()}", "code": "request_rejected", } if error: return {"error": error, "code": code or "unknown"} return {"markdown": res.text}def via_uprock(url: str, country: str) -> str: run = subprocess.run( ["uprock", "ai", "crawl", url, "--country", country, "--content"], capture_output=True, text=True, timeout=330, ) if run.returncode != 0: raise RuntimeError(f"UpRock: {run.stderr.strip()}") return json.loads(run.stdout).get("inlined_markdown", "")def read(url: str, country: str = "US") -> dict: result = via_bright_data(url, country) if RETRY_ON_BRIGHT_DATA.match(result.get("code") or ""): result = via_bright_data(url, country) if "markdown" in result: return {"via": "brightdata", "markdown": result["markdown"]} code, error = result["code"], result["error"] if ( code not in TRY_UPROCK and not RETRY_ON_BRIGHT_DATA.match(code) and code != "request_rejected" ): raise RuntimeError(f"Bright Data: {error} ({code})") print(f"Bright Data: {error} ({code}); trying UpRock", file=sys.stderr) return {"via": "uprock", "markdown": via_uprock(url, country)}if __name__ == "__main__": # python read.py https://developer.mozilla.org/en-US/docs/Web/HTTP DE url = sys.argv[1] if len(sys.argv) > 1 else "https://developer.mozilla.org/en-US/docs/Web/HTTP" country = sys.argv[2] if len(sys.argv) > 2 else "US" page = read(url, country) print(page["via"], page["markdown"][:200])Tested 21 Oct 2026 with an invalid Bright Data key: the API answered with the documented 401 Invalid token, and both versions fell back to UpRock. Without an account, the error-code routing follows Bright Data's published catalog rather than live failures. On 6 Nov 2026 we ran the HTTP and MCP versions again with an UpRock API key in an Authorization header: with an invalid Bright Data key, each fell back to UpRock and got the page.
FAQ
Doesn't Bright Data also use real people's devices?
For its residential proxies, yes. Bright Data says app owners add its SDK to their apps and users opt in, in exchange for an ad-free or upgraded app. UpRock's devices also belong to people who opted in, by installing the UpRock app or node and earning UPT for the work. The two are separate networks, which is the point of a backup.
Does Bright Data charge for a failed request?
Web Unlocker's pay-as-you-go price is "pay only for success", according to its pricing page. A fallback to UpRock adds one UpRock credit for the retry.
Should I retry a page Bright Data blocks for robots.txt?
No. The ub_bad_endpoint_robots error means the site's robots.txt disallows that path for Bright Data's no-verification mode. Treat it as the site's answer. UpRock fetches public pages only, and the fallback on this page leaves that error alone.
Which one covers more countries?
Bright Data, by a wide margin. It lists 195 countries for residential IPs; UpRock covers 63, plus regions such as EU or APAC.
Can I use Bright Data without verifying a company?
Web Unlocker, yes, with robots.txt honored in that mode. Residential proxies, no. Bright Data's docs say residential access is for verified companies only, with no self-serve path, and personal email addresses can't be approved. UpRock needs only an API key.
Other use cases to explore
- Cap what a single page can cost: one provider's credit table, from 1 to 75 credits a page, and how to stop before the expensive tiers.
- See a page as it looks in another country: one store page from five countries, with five prices, two languages and a cookie banner in one country only.
- Add a fallback to a crawler you run: hand the requests Crawlee gives up on to a device in the country you choose.
- Search public video by moment: find the second something is shown or said, across platforms.
References
- Bright Data, Web Unlocker pricing, checked 21 Oct 2026
- Bright Data, Residential proxy pricing, checked 21 Oct 2026
- Bright Data docs, Web Unlocker error codes, checked 21 Oct 2026
- Bright Data docs, Unlock a website (Direct API request), checked 21 Oct 2026
- Bright Data, How residential IPs are sourced, checked 21 Oct 2026
- Bright Data docs, Residential network access (company verification), checked 21 Oct 2026
- Bright Data docs, Free tier, checked 21 Oct 2026
- Bright Data docs, Acceptable use policy, checked 21 Oct 2026
- Bright Data, Video Search API (beta), checked 21 Oct 2026
- Bright Data MCP server on GitHub, checked 21 Oct 2026
- UpRock, everything in one file (API, CLI, MCP, pricing), checked 21 Oct 2026
Get started
Add UpRock in one paste.
Paste this into Claude Code, Codex, Cursor or any agent with a terminal. It reads our docs, sets up the MCP server or CLI, and tells you the cost before it spends anything.
Set up UpRock for me: read https://uprock.ai/agents.md, connect the UpRock MCP server, and run one test crawl. Tell me what it costs before you spend anything.