URL to Markdown API

The article. Not the article plus a cookie policy.

A URL-to-markdown API for RAG and agents. Extraction needs a real browser, and the moment you run one the consent dialog is in your document — hiding it with CSS keeps it out of a screenshot, not out of the DOM, so it comes back in the text. We detach the nodes before extracting. Measured on zoom.com: 51 fragments of consent copy from a leading browser-based extractor, none from this one.

display:none — still in the DOM detached before reading
93%renders succeedacross a 29-URL benchmark
69%are actually usablethe gap nobody quotes
1 / 29blocked by bot protection7 ruined by consent walls
~2smedian renderp50, viewport capture

Why a browser, and why it has to remove rather than hide

Modern extraction runs a real browser, because a plain fetch returns a shell that JavaScript was supposed to fill in. But once the browser has run, the consent dialog is in the document — and hiding it with display:none keeps it out of a screenshot, not out of the DOM. So it comes back in the text. Measured on zoom.com in August 2026: a leading browser-based markdown API returned 51 fragments of cookie-consent copy, including the full Cookie Preference Center. The same page through /v1/markdown returned none, because the nodes are detached before a word is read.

Reading https://www.zoom.com Fragments of consent copyWhat you actually get
A plain fetch + a readability parser0…and no article either: the page is a shell until JavaScript runs
A leading browser-based markdown API51including the full “Cookie Preference Center” and the OneTrust logo
rendershed /v1/markdown01 overlay node detached before extraction

Measured August 2026, and it will drift as everyone's pipelines change — the method is a regex for consent wording over each tool's own output, which is as reproducible as it is crude. Run it yourself before believing it.

Cosmetic is not enough for text

Suppression on the capture path is a stylesheet: the consent dialog stops being painted, which is the whole job for a screenshot. It is still a node, so page.content(), textContent and every HTML parser downstream still see it. These endpoints detach the tagged nodes first, and tell you how many in suppressed.stripped.

Where this does not help

Worth saying plainly. Some consent managers render in a cross-origin iframe — Sourcepoint, which fronts the Guardian and the BBC — and no extractor can read inside one, so their copy was never going to reach your text. And when main_only is on, the article-finding step already drops most page furniture. The gap this closes is inline consent managers, whole-document reads, and pages with no clear article body.

It never clicks Accept

Declining on your behalf for a throwaway read is defensible. Agreeing on your behalf is not. Reject buttons that are really subscription funnels are recognised by their own label and skipped in favour of hiding — the Guardian's is exactly that, which is why it comes back marked ~sourcepoint.

It sees what JavaScript built

A fetch returns what the server sent, which on most of the web is a shell. This runs the page, waits for the network to settle, and reads the document that resulted — the same navigation a screenshot gets, because it is the same fleet.

One render, one allowance

The same key, the same quota and the same subscription as a screenshot or a PDF. There is no separate plan for text, and a cached call is free because it cost us a disk read.

One call, no signup POST /v1/markdown text/markdown

Anonymous calls run at 40 an hour against your address. A free key raises that and works from anywhere.

# No key needed. Paste this as it is.
# response=text gives you text/markdown down the pipe; leave it off and
# you get the document plus what was taken off the page to produce it.
curl -X POST https://shots.rendershed.com/v1/markdown \
  -H "Content-Type: application/json" \
  -d '{"url": "theguardian.com/international", "response": "text"}'

# Compare it with the same page fetched raw — the consent copy is in there,
# because display:none keeps a banner out of a picture, not out of the DOM.
const res = await fetch("https://shots.rendershed.com/v1/markdown", {
  method: "POST",
  headers: {
    "Content-Type": "application/json",
    ...(process.env.RENDERSHED_KEY && {
      "X-API-Key": process.env.RENDERSHED_KEY,
    }),
  },
  body: JSON.stringify({
    url: "https://example.com/article",
    main_only: true,      // article body; false gives the whole document
    include_links: true,
    max_chars: 20000,     // cut on a line boundary, and say so
  }),
});

const doc = await res.json();
console.log(doc.title, doc.words, "words");
console.log(doc.suppressed.stripped, "overlay nodes detached first");
console.log(doc.markdown);
import os, requests

doc = requests.post(
    "https://shots.rendershed.com/v1/markdown",
    headers={"X-API-Key": os.environ["RENDERSHED_KEY"]},
    json={"url": "https://example.com/article", "main_only": True},
    timeout=90,
).json()

# Straight into a vector store, without the cookie policy riding along.
chunks = doc["markdown"].split("\n\n")
print(doc["title"], "-", doc["words"], "words,", len(chunks), "chunks")

What comes back

Trimmed from a real response through this API.

{
  "url": "https://www.theguardian.com/international",
  "title": "Latest news, sport and opinion from the Guardian",
  "root": "main#maincontent",
  "words": 90,
  "chars": 612,
  "truncated": false,
  "lang": "en",
  "status": 200,
  "ms": 3627,
  "suppressed": {
    "mode": "dismiss",
    "dismissed": ["~sourcepoint", "~sourcepoint", "~sourcepoint"],
    "hidden": 1,
    "swept": [],
    "stripped": 1
  },
  "markdown": "## News\n\n- ### Germany\n\n  Gun stash discovered in ..."
}

Parameters

NameTypeDefaultDescription
url stringrequiredThe page to read. http and https only.
main_only booltrueReturn just the article body. false returns the whole document, which is what you want for a landing page or a docs index.
include_links booltrueKeep links as markdown. false gives cleaner prose.
include_images boolfalseInclude images as markdown image syntax.
max_chars int0Truncate to roughly this many characters, on a line boundary. 0 means no limit beyond the service cap.
scroll boolfalseWalk to the bottom first so lazy-loaded content is present. Paid plans only — same unbounded work as full_page.
response enumjsonjson returns the document and its metadata; text returns text/markdown for piping straight into something else.
device enumiphone, android or ipad. Some sites serve substantially less content to phones.
overlay_mode enumdismissoff, hide or dismiss. Leave this on unless you want the banner text in your output.

Errors, response headers and the signed-URL scheme are the same for every product — they live in the reference.

On the same key

Five outputs, one account, one quota. Every call costs one render whichever path you use.

Get a key