How do I convert a web page to markdown for an LLM?
Render the page if it needs JavaScript, then convert the HTML to markdown, keeping headings, lists, links, tables and code and dropping scripts, styles, menus and footers. On six public pages we measured on 25 Oct 2026, the markdown UpRock returned used 57% to 99% fewer tokens than the rendered HTML. Check the output before you trust it, though; on one docs page the code examples didn't survive.
Updated · Read as markdown
Why markdown, not HTML
A model reads tokens, and most of the tokens in a web page's HTML aren't the content: class names, inline styles and scripts, SVG icons, tracking tags, menus and footers. Markdown keeps the structure a model can use (headings, lists, links, tables, code) in a few characters each. Cloudflare's example: ## About Us costs about 3 tokens, while <h2 class="section-title" id="about">About Us</h2> costs 12 to 15. Its post measured itself at 16,180 tokens as HTML and 3,150 as markdown, an 80% cut (published 12 Feb 2026; the post doesn't say which tokenizer).
Fewer tokens means a lower bill, more pages in one context window, and cleaner chunks if you're building retrieval (RAG), because headings give you natural places to split.
HTML and markdown, measured on six pages
We fetched six varied public pages with UpRock's crawl_fetch on 25 Oct 2026, read both stored versions with resource_fetch, and counted tokens with o200k_base. For comparison, we also counted the HTML each site serves to a plain GET, before any JavaScript runs:
| Page | HTML as served | HTML after rendering | Markdown | Saved vs rendered HTML |
|---|---|---|---|---|
| Cloudflare blog, "Introducing Markdown for Agents" | 111,974 | 138,836 | 2,599 | 98% |
| Wikipedia, "Markdown" | 100,427 | 148,714 | 10,030 | 93% |
MDN, Array.prototype.map() | 48,141 | 25,352 | 1,823 | 93% (but see below) |
| Hacker News, front page | 11,725 | 11,773 | 5,083 | 57% |
| GOV.UK, "UK bank holidays" | 51,179 | 55,242 | 4,992 | 91% |
| GitHub, the openai/tiktoken repository page | 110,637 | 124,450 | 1,221 | 99% |
A few things stand out:
- Cloudflare's post is much heavier than when Cloudflare measured it. By our count, its served HTML is now 111,974 tokens, against the 16,180 the post reports. The markdown is still small: 2,599 tokens from UpRock, and 4,707 for the markdown Cloudflare itself serves for the post today when asked for
text/markdown(that version adds front matter and the title). - The rendered HTML moves around; the markdown doesn't. Two fetches of the Wikipedia article minutes apart gave 148,714 and 168,276 tokens of HTML, but 10,030 and 10,062 of markdown.
- Lean pages save less. Hacker News is mostly links, its HTML is already small, and markdown keeps every URL, so the cut is 57%.
- App pages save the most. On GitHub, the markdown is just the README; the file list, sidebar and menus are gone.
What the markdown kept and dropped
We read all six markdown files against the pages. What we saw:
| Element | What happened |
|---|---|
| Headings, paragraphs, lists, links | Kept on all six. Some headings carry an empty anchor link in front, such as ## [](…#why-markdown-is-important)Why markdown is important |
| Simple data tables | Kept. GOV.UK's bank holiday tables came through as clean pipe tables, 346 rows in all |
| Complex tables | Flattened. Wikipedia's infobox and its three-column syntax example each ran together into one line, cells pushed up against each other and the code samples' line breaks lost |
| Code blocks | Kept as fenced blocks on the Cloudflare post and the GitHub README, except a one-line pip install tiktoken block, which was dropped |
| Code inside web components | Lost. MDN moves its examples into <mdn-code-example> components in the browser, the captured HTML had none of the 18 <pre> blocks MDN serves, and the markdown kept the prose around each example with no code |
| Menus, footers, sidebars, scripts, styles | Dropped on all six |
| Page title | Kept on Wikipedia and GOV.UK; missing on the Cloudflare post (the markdown starts at the first paragraph) and on MDN (it starts at "Try it") |
| Images | Kept as image links on the Cloudflare post (its 4 diagrams and charts) and GitHub (a benchmark chart); Wikipedia's images, including the Markdown logo in the infobox, were dropped |
| Leftovers | Three raw <section> tags in the GOV.UK markdown, and Hacker News's vote arrows as empty links |
The pattern: prose, simple tables and ordinary code blocks convert well. Layout tables, content that a page draws inside web components, and very short blocks are where conversions go wrong. Spot-check a page or two from every site you rely on, whichever converter you use.
When to keep the HTML, or ask for markdown
- Ask the site first. Send
Accept: text/markdown. Sites on Cloudflare can turn on Markdown for Agents, which converts pages at the edge; Cloudflare's docs say it's available on Pro, Business and Enterprise plans at no extra cost, up to 6 MiB of HTML (checked 25 Oct 2026). The Cloudflare blog and docs answer that way today. Wikipedia, MDN and GOV.UK sent HTML when we asked on 25 Oct 2026. - Keep the HTML when the data is in the markup. Prices in a buy box, attributes such as
data-price, structured data in JSON-LD, and complex tables are easier to read from HTML. That's why the country comparison parses HTML. - Try the served HTML for static docs. On MDN, the HTML served to a plain GET had every code example, and the rendered page had none. For pages that build their content with JavaScript, it's the other way round, and you need the rendered version. UpRock renders by default.
- Keep both when you're not sure. One UpRock crawl stores both versions, so you can read the markdown and fall back to the HTML for the parts that didn't convert.
Count the tokens yourself
Every version take a country (or a region such as EU) and a list of URLs, fetch each page once, read the stored HTML and markdown, and print the token counts with o200k_base. UpRock gets one POST /v1/crawl request with your key in an Authorization: Bearer header; the MCP and CLI tabs do the same through UpRock's MCP server (TypeScript) or CLI (Python).
// npm i js-tiktoken (Node 18+) env: UPROCK_API_KEY// node tokens.ts EU https://developer.mozilla.org/en-US/docs/Web/HTTP https://news.ycombinator.comimport { getEncoding } from "js-tiktoken";const enc = getEncoding("o200k_base"); // GPT-4o and later OpenAI modelsconst count = (text: string) => enc.encode(text, [], []).length;// One crawl returns both versions: the rendered HTML and the markdown made from it.async function bothVersions(url: string, country: string) { // One call: UpRock loads the page on a real device in that country, tries a fresh device // every 7 s (up to 3 tries) and returns the rendered HTML next to the markdown. const res = await fetch("https://edge.uprock.com/v1/crawl", { method: "POST", headers: { Authorization: `Bearer ${process.env.UPROCK_API_KEY}`, "Content-Type": "application/json", }, body: JSON.stringify({ url, country, format: "html" }), signal: AbortSignal.timeout(90_000), }); const page = await res.json(); if (!res.ok) throw new Error(`UpRock ${res.status} ${page.code}: ${page.error}`); return { html: page.html ?? "", markdown: page.markdown ?? "" };}const [country = "EU", ...urls] = process.argv.slice(2);const results = await Promise.allSettled(urls.map((url) => bothVersions(url, country)));console.log("HTML tokens markdown tokens saved url");results.forEach((r, i) => { if (r.status === "rejected") return console.log(`failed: ${urls[i]} (${r.reason})`); const html = count(r.value.html); const md = count(r.value.markdown); if (!md) return console.log(`${html} no markdown ${urls[i]}`); const saved = Math.round(100 * (1 - md / html)); console.log(`${html} ${md} ${saved}% ${urls[i]}`);});# pip install tiktoken requests env: UPROCK_API_KEY# python tokens.py EU https://developer.mozilla.org/en-US/docs/Web/HTTP https://news.ycombinator.comimport osimport sysfrom concurrent.futures import ThreadPoolExecutorimport requestsimport tiktokenenc = tiktoken.get_encoding("o200k_base") # the tokenizer of GPT-4o and later modelsdef count(text: str) -> int: return len(enc.encode(text, disallowed_special=()))def both_versions(url: str, country: str) -> tuple[str, str]: """One crawl returns the rendered HTML and the markdown made from it.""" # One call: UpRock loads the page on a real device in that country, tries a fresh device # every 7 s (up to 3 tries) and returns the rendered HTML next to the markdown. res = requests.post( "https://edge.uprock.com/v1/crawl", headers={"Authorization": f"Bearer {os.environ['UPROCK_API_KEY']}"}, json={"url": url, "country": country, "format": "html"}, timeout=90, ) page = res.json() if not res.ok: raise RuntimeError(f"UpRock {res.status_code} {page.get('code')}: {page.get('error')}") return page.get("html", ""), page.get("markdown", "")if __name__ == "__main__": country, *urls = sys.argv[1:] or ["EU"] with ThreadPoolExecutor(max_workers=8) as pool: futures = {url: pool.submit(both_versions, url, country) for url in urls} print("HTML tokens markdown tokens saved url") for url, future in futures.items(): try: html, markdown = future.result() except Exception as err: print(f"failed: {url} ({err})") continue h, m = count(html), count(markdown) if not m: print(f"{h} no markdown {url}") continue print(f"{h} {m} {round(100 * (1 - m / h))}% {url}")// npm i @modelcontextprotocol/sdk js-tiktoken (Node 18+) env: UPROCK_API_KEY// node tokens.ts EU https://developer.mozilla.org/en-US/docs/Web/HTTP https://news.ycombinator.comimport { Client } from "@modelcontextprotocol/sdk/client/index.js";import { StreamableHTTPClientTransport } from "@modelcontextprotocol/sdk/client/streamableHttp.js";import { getEncoding } from "js-tiktoken";const enc = getEncoding("o200k_base"); // GPT-4o and later OpenAI modelsconst count = (text: string) => enc.encode(text, [], []).length;const uprock = new Client({ name: "html-vs-markdown", version: "1.0.0" });await uprock.connect( new StreamableHTTPClientTransport( new URL("https://mcp.uprock.ai/mcp"), { requestInit: { headers: { Authorization: `Bearer ${process.env.UPROCK_API_KEY}` } } }, ),);const textOf = (content: unknown) => (content as { type: string; text?: string }[]).find((c) => c.type === "text")?.text ?? "";const read = async (uri?: string) => uri ? textOf( (await uprock.callTool({ name: "resource_fetch", arguments: { uri } })).content, ) : "";// One crawl stores both versions: the rendered HTML and the markdown made from it.async function bothVersions(url: string, country: string) { const res = await uprock.callTool( { name: "crawl_fetch", arguments: { url, country } }, undefined, { timeout: 300_000 }, ); if (res.isError) throw new Error(textOf(res.content)); const { content } = JSON.parse(textOf(res.content)); return { html: await read(content.html?.resource), markdown: await read(content.markdown?.resource), // missing if extraction failed };}const [country = "EU", ...urls] = process.argv.slice(2);const results = await Promise.allSettled(urls.map((url) => bothVersions(url, country)));await uprock.close();console.log("HTML tokens markdown tokens saved url");results.forEach((r, i) => { if (r.status === "rejected") return console.log(`failed: ${urls[i]} (${r.reason})`); const html = count(r.value.html); const md = count(r.value.markdown); if (!md) return console.log(`${html} no markdown ${urls[i]}`); const saved = Math.round(100 * (1 - md / html)); console.log(`${html} ${md} ${saved}% ${urls[i]}`);});# pip install tiktoken; npm i -g uprock env: UPROCK_API_KEY# python tokens.py EU https://developer.mozilla.org/en-US/docs/Web/HTTP https://news.ycombinator.comimport jsonimport subprocessimport sysfrom concurrent.futures import ThreadPoolExecutorimport tiktokenenc = tiktoken.get_encoding("o200k_base") # the tokenizer of GPT-4o and later modelsdef count(text: str) -> int: return len(enc.encode(text, disallowed_special=()))def uprock(*args: str) -> str: run = subprocess.run( ["uprock", "ai", *args], capture_output=True, text=True, timeout=330, check=False, ) if run.returncode != 0: raise RuntimeError(run.stderr.strip()) return run.stdoutdef both_versions(url: str, country: str) -> tuple[str, str]: """One crawl stores the rendered HTML and the markdown made from it.""" page = json.loads(uprock("crawl", url, "--country", country, "--content")) html = uprock("fetch", page["content"]["html"]["resource"]) return html, page.get("inlined_markdown", "") # empty if extraction failedif __name__ == "__main__": country, *urls = sys.argv[1:] or ["EU"] with ThreadPoolExecutor(max_workers=8) as pool: futures = {url: pool.submit(both_versions, url, country) for url in urls} print("HTML tokens markdown tokens saved url") for url, future in futures.items(): try: html, markdown = future.result() except Exception as err: print(f"failed: {url} ({err})") continue h, m = count(html), count(markdown) if not m: print(f"{h} no markdown {url}") continue print(f"{h} {m} {round(100 * (1 - m / h))}% {url}")Run on the six pages above with the region set to EU, the TypeScript version printed the numbers in the table:
HTML tokens markdown tokens saved url138836 2599 98% https://blog.cloudflare.com/markdown-for-agents/148714 10030 93% https://en.wikipedia.org/wiki/Markdown25352 1823 93% https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/map11773 5083 57% https://news.ycombinator.com/55242 4992 91% https://www.gov.uk/bank-holidays124450 1221 99% https://github.com/openai/tiktokenTested 25 Oct 2026 against the live UpRock API. After that run we ran both code blocks from this page again, the TypeScript version (type-checked with tsc --strict) on GOV.UK, MDN and Hacker News and the Python version on example.com, the Cloudflare post and Hacker News, and the counts came within 3% of the table. During those runs, two fetches from the EU region failed after about 96 seconds with "context deadline exceeded" and worked when we ran them again; the code reports such a page as failed and carries on with the rest. On 6 Nov 2026 we ran the HTTP and MCP versions again with an UpRock API key in an Authorization header: MDN's HTTP overview page came out 92% smaller as markdown, Hacker News 57%.
FAQ
How many tokens does markdown save over HTML?
On the six pages we measured, 57% to 99% against the rendered HTML. Pages wrapped in heavy app chrome save the most (a GitHub repo page went from 124,450 tokens to 1,221), and lean pages that are mostly links save the least (Hacker News went from 11,773 to 5,083). Cloudflare reported 80% for its own blog post.
Which tokenizer did you use?
o200k_base, through tiktoken in Python and js-tiktoken in TypeScript, which gave identical counts on the same files. tiktoken maps GPT-4o, GPT-4.1, GPT-5 and the o-series models to it. Other models' tokenizers give other counts, so read the numbers as relative.
Does UpRock return markdown for every page?
For most pages. crawl_fetch stores the rendered HTML and, when extraction works, a markdown version made from it. One of our fetches the same day, a help article loaded from Japan, came back with HTML only, so handle a missing markdown resource in your code.
Should I ask the site for markdown first?
Yes, it's cheap and it's the site's own version. Send Accept text/markdown. The Cloudflare blog and Cloudflare's docs answer with markdown today; Wikipedia, MDN and GOV.UK answered with HTML when we tried on 25 Oct 2026.
What does it cost with UpRock?
One crawl gives you both versions, the rendered HTML and the markdown, for one credit, which is $0.50 per 1,000 pages by card or USDC ($0.40 with UPT). The price doesn't depend on how long the page is.
Other use cases to explore
- Estimate what 1,000 pages cost by the token: Reader's token counts for six other pages, and the length at which a flat per-page price is cheaper.
- See a page as people in another country see it: one store page from five countries, and why prices come from the HTML, not the markdown.
- Find out why your agent got a 403: a diagnosis table from status code to the legitimate fix, and when to stop.
- Keep agent web costs under control: where the money goes when an agent reads the web, and how to cut it.
References
- Cloudflare blog, Introducing Markdown for Agents (16,180 and 3,150 tokens), checked 25 Oct 2026
- Cloudflare docs, Markdown for Agents (plans and limits), checked 25 Oct 2026
- openai/tiktoken on GitHub (the o200k_base encoding), checked 25 Oct 2026
- js-tiktoken, a JavaScript port of tiktoken, checked 25 Oct 2026
- MDN, Content negotiation (the Accept header), checked 25 Oct 2026
- MDN, Using shadow DOM, checked 25 Oct 2026
- UpRock, everything in one file (API, CLI, MCP, pricing), checked 25 Oct 2026
Get started
Add UpRock in one paste.
Paste this into Claude Code, Codex, Cursor or any agent with a terminal. It reads our docs, sets up the MCP server or CLI, and tells you the cost before it spends anything.
Set up UpRock for me: read https://uprock.ai/agents.md, connect the UpRock MCP server, and run one test crawl. Tell me what it costs before you spend anything.