switch402

How to Scrape a Website to Markdown in Python (Pay per Page, No API Key)

Fetch any public page as clean markdown from Python with one HTTP call. You pay $0.008 per successful page in USDC on Base over x402; there is no signup or API key.

Steps

  1. Install the x402 client. pip install "x402[requests,evm]". It wraps requests and pays HTTP 402 responses automatically.
  2. Fund a wallet on Base. Use a wallet that holds a little USDC on Base; export its private key as AGENT_KEY. Each page costs $0.008 (or $0.003 on lite).
  3. Check the price for free. GET https://api.switch402.com/v1/quote?url=... returns the price, healthy-provider count and estimated latency.
  4. Scrape the page. POST {"url": "..."} to /v1/scrape. The client pays the 402 and retries; the response JSON has url, title and markdown.

Code

Free quote (curl)

curl -s 'https://api.switch402.com/v1/quote?url=https://example.com' -H 'User-Agent: my-agent/1.0'

Python: page to markdown file

# pip install "x402[requests,evm]"
import os
from eth_account import Account
from x402 import x402ClientSync
from x402.http.clients import x402_requests
from x402.mechanisms.evm import EthAccountSigner
from x402.mechanisms.evm.exact import ExactEvmScheme

client = x402ClientSync()
client.register("eip155:8453", ExactEvmScheme(signer=EthAccountSigner(Account.from_key(os.environ["AGENT_KEY"]))))
session = x402_requests(client)
session.headers["User-Agent"] = "my-agent/1.0"

r = session.post("https://api.switch402.com/v1/scrape", json={"url": "https://example.com"})
r.raise_for_status()
page = r.json()  # {"url", "title", "markdown", "own_service", "ms", "attempts"}
open("page.md", "w").write(page["markdown"])
print(page["title"], len(page["markdown"]), "chars")

# Lower-cost tier: session.post("https://api.switch402.com/v1/scrape/lite", json={"url": ...})

You pay only for the pages you actually fetch: $0.008 per standard page on /v1/scrape, $0.003 per lite call on /v1/scrape/lite, in USDC on Base and charged only on success. No subscription or unused credits; optional prepaid packs and a $10 monthly plan.

FAQ

Do I need an API key?
No. Your code pays per call over x402 in USDC on Base; there is no account. An optional prepaid API key is available if you prefer a balance (see /docs#credits).
What does it cost?
$0.008 per successful page on /v1/scrape or $0.003 on /v1/scrape/lite. Failed calls return 502 and are not charged.
Does it render JavaScript pages?
switch402 routes to x402 scrape providers and returns their markdown; results depend on the page. Use the free quote endpoint to check availability first.

Related