SCRAPE API

Extract the page your product actually needs.

Use scrape when you know the URL. Basecrawl renders modern pages, removes recurring chrome, and returns one normalized document.

ENDPOINT REFERENCE

Extract a single page

The scrape endpoint is optimized for precision. Send a full URL and ask for markdown, structured JSON, or the rendered HTML your downstream system needs.

ENDPOINT

POST/v1/scrape

Render and extract a single page.

Parameters

Configure the request with only the controls your workflow needs.

Parameter
url
Type
string
REQUIRED
Description
Absolute URL for the page to fetch and extract.
Parameter
formats
Type
string[]
DEFAULT
[markdown]
Description
Requested output fields: markdown, json, or html.
Parameter
onlyMainContent
Type
boolean
DEFAULT
true
Description
Preserve the primary article or product content only.
Parameter
headers
Type
Record<string, string>
Description
Additional request headers for the target origin.

Example request

POST /v1/scrape
const response = await fetch("https://api.basecrawl.dev/v1/scrape", {
  method: "POST",
  headers: {
    Authorization: "Bearer bc_live_your_key",
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    url: "https://example.com/research",
    formats: ["markdown", "html"],
    onlyMainContent: true,
    waitFor: 500,
  }),
});

const document = await response.json();

Predictable output for downstream systems

Markdown is ideal for retrieval and LLM context, JSON preserves structured fields, and HTML supports custom transforms. Request several formats during a migration without fetching the target twice.

Normalized document
{
  "success": true,
  "data": {
    "url": "https://docs.example.com/guide",
    "markdown": "# Getting started\n\n...",
    "html": "<main>...</main>",
    "metadata": {
      "title": "Getting started",
      "statusCode": 200
    }
  }
}

KEEP EXPLORING

OverviewChoose a focused path into the Basecrawl platform.QuickstartMake your first request in under two minutes.API referenceRequest conventions, authentication, and response shapes.CrawlDiscover and extract an entire site.SDKsTyped clients for JavaScript and Python.