Extract All URLs from a Website: Sitemap + Link API, $0.001 per Call

/v1/map reads a public page and the site’s sitemaps (robots.txt Sitemap lines or /sitemap.xml) and returns the same-site URLs it finds, deduplicated, up to 1,000 per call. $0.001 USDC on Base per call, paid over x402, charged only when URLs come back.

• Page links + sitemap URLs

  One list, deduplicated. sources says how many came from the page and which sitemap files were read (up to 4 per call).

• Up to 1,000 URLs

  limit defaults to 200 and goes up to 1,000. truncated is true when there were more.

• Limits

  Public http(s) URLs on default ports only, at most 5 redirects, 8 s per fetch and 20 s per call, 2 MB per page and 5 MB per sitemap.

• Not charged when empty

  Not an HTML page or no URLs found returns 422; a fetch failure returns 502. Neither is charged.

Cost: $0.001 per call. Mapping 500 sites once costs $0.50.

Code examples

curl -i -G https://api.switch402.com/v1/map \
  -H 'User-Agent: my-agent/1.0' \
  --data-urlencode 'url=https://example.com' \
  --data-urlencode 'limit=500'
# 200 after payment: {"url", "final_url", "page_status", "count", "truncated", "limit", "urls": [...],
#   "sources": {"page_links", "sitemap_urls", "sitemaps": [...]}, "ms"}

# pip install "x402[requests,evm]"
import os
from eth_account import Account
from x402 import x402ClientSync
from x402.http.clients import x402_requests
from x402.mechanisms.evm import EthAccountSigner
from x402.mechanisms.evm.exact import ExactEvmScheme

client = x402ClientSync()
client.register("eip155:8453", ExactEvmScheme(signer=EthAccountSigner(Account.from_key(os.environ["AGENT_KEY"]))))
session = x402_requests(client)
session.headers["User-Agent"] = "my-agent/1.0"

import csv

r = session.post("https://api.switch402.com/v1/map", json={"url": "https://example.com", "limit": 1000})  # $0.001
r.raise_for_status()
site = r.json()
with open("urls.csv", "w", newline="") as f:
    csv.writer(f).writerows([[u] for u in site["urls"]])
print(site["count"], "URLs", "(truncated)" if site["truncated"] else "")

curl -s -X POST https://api.switch402.com/mcp -H 'Content-Type: application/json' -H 'Accept: application/json, text/event-stream' -H 'User-Agent: my-agent/1.0' \
  -d '{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"map","arguments":{"url":"https://example.com","limit":500}}}'