API REFERENCE

A small API surface for a very large web.

Use a consistent request shape for one page or an entire site. Basecrawl normalizes rendering, extraction, and output behind two focused endpoints.

Base URL and authentication

All API requests use https://api.basecrawl.dev/v1. Authenticate with a project key in the Authorization header. Keep the key in your server environment, never in a browser bundle.

Authorization header
Authorization: Bearer bc_live_your_key
Content-Type: application/json

ENDPOINT REFERENCE

Send a scrape request

These shared fields apply to extraction requests. Crawl-specific discovery controls are documented on the crawl endpoint.

ENDPOINT

POST/v1/scrape

Extract one normalized page with a consistent request shape.

Parameters

Configure the request with only the controls your workflow needs.

Parameter
url
Type
string
REQUIRED
Description
Absolute HTTP(S) URL to scrape, or the root URL to crawl.
Parameter
formats
Type
string[]
DEFAULT
[markdown]
Description
Requested output fields: markdown, json, or html.
Parameter
onlyMainContent
Type
boolean
DEFAULT
true
Description
Remove navigation, cookie banners, and page chrome before extraction.
Parameter
waitFor
Type
number
DEFAULT
0
Description
Extra milliseconds to wait after rendering JavaScript.

Example request

HTTP request
POST /v1/scrape HTTP/1.1
Authorization: Bearer bc_live_your_key
Content-Type: application/json

{"url":"https://docs.example.com/guide","formats":["markdown"]}

Response envelope

A successful response carries a data object and the requested formats. Validation errors include a stable code, a message, and the failing parameter path.

200 OK
{
  "success": true,
  "data": {
    "url": "https://docs.example.com/guide",
    "markdown": "# Getting started\n\n...",
    "html": "<main>...</main>",
    "metadata": {
      "title": "Getting started",
      "statusCode": 200
    }
  }
}

KEEP EXPLORING

OverviewChoose a focused path into the Basecrawl platform.QuickstartMake your first request in under two minutes.CrawlDiscover and extract an entire site.ScrapeExtract clean content from a single page.SDKsTyped clients for JavaScript and Python.