Live web context for AI agents

The live web,
ready for your agents.

Search, scrape, crawl, and transform the web into clean, source-linked context.

Start building Read the docs

Open source core · Hosted API · Agent ready

Agent runLive

taskBuild a cited brief on edge AI

Understand requestComplete

Discover sourcesRanking relevant sources

Extract contextQueued

Return grounded outputQueued

12sources2.4selapsedgrounded

Result · JSONStreaming

"summary": "Edge AI moves inference closer to data, reducing latency and bandwidth.",
"confidence": 0.94,
"sources": [
  "docs.nvidia.com/edge-ai",
  "cloud.google.com/edge"
],
"status": "12 sources discovered"

Source URLs preserved8 primary · 4 supporting

Built for teams shipping web-aware AI

ResearchSupportSalesAgentsRAGDocumentationKnowledge basesAssistantsResearchSupportSalesAgentsRAGDocumentationKnowledge basesAssistants

Interactive preview

Preview the API. Before signup.

This local simulation shows the response shape. Create an account when you are ready to run real requests.

Pipeline

1Validate request

2Fetch source

3Extract content

4Format output

…

# News

News turns the live web into clean, structured context for AI agents.

## Highlights
- Clean Markdown and JSON
- Robots-respecting crawl
- URL metadata preserved

// URL discovery

Map a site before your
agent dives in.

Qrawl checks the sitemap first, then falls back to breadth-first link discovery to return a deduplicated URL inventory.

Sitemap firstBFS fallbackDomain controls

[ 01 / 05 ] · MAIN FEATURES

01 / Product

Compose web context
your way.

A modular toolkit for searching, scraping, mapping, and crawling public web content.

Search

Find high-signal sources across the open web.

Scrape

Turn any page into clean Markdown, text, HTML, or JSON.

Crawl

Traverse sites and keep linked context intact.

One API. Predictable output.

import { QrawlClient } from "qrawl";

const qrawl = new QrawlClient({
  apiKey: process.env.QRAWL_API_KEY!,
});

const result = await qrawl.scrape("https://example.com", {
  format: "markdown",
});

Structured result● 200 OK · 1.42s

"title": "A practical guide to edge AI"
"content": "# Edge AI..."
"statusCode": 200
"format": "markdown"

01example.com/edge-ai02docs.example.com/deploy

[ 02 / 05 ] · AGENT READY

02 / Agent ready

Built for agents.
Observable by humans.

Inspect every source, step, retry, and transformation without slowing the agent down.

Run in progressrun_9f2b7c3e

Understand requestIntent and constraints

Discover sourcesRanked 12 pages

Extract contentParsing page 8 of 12

Validate schemaWaiting

Return contextSource-linked

Started10:42:18 AMRetries0 / 2Duration3.1sOutputSource-linked

MCP Â· REST Â· SDK

Plug context into
any agent stack.

MCPClaudeCursor

Connect any agent in one command.

Drop the MCP server into Claude, Cursor, or Windsurf — no glue code, no backend.

npx -y qrawl-mcp

MCP setup

Node.jsTypeScript

Use the cloud API from your app.

Install the typed Node SDK, create a client with your API key, and call the same four core endpoints.

npm install qrawl

SDK quickstart

Integrations

Drops into your stack.

Connect through MCP, the Node SDK, raw HTTP, or the self-hosted core.

ClaudeMCP

CursorMCP

WindsurfMCP

VS CodeMCP

Node.jsSDK

TypeScriptSDK

REST APIHTTP

cURLHTTP

qrawl-coreSelf-host

DockerSelf-host

SearchAPI

Crawl jobsAPI

[ 03 / 05 ] · CORE

03 / Infrastructure

Reliable by default.
Powerful when it matters.

Robots-aware crawling

Respect robots.txt and enforce polite per-domain delays.

Predictable formats

Return Markdown, text, HTML, or JSON through one interface.

Durable crawl jobs

Poll consistent job state backed by the database.

Open source

Transparent core, built in the open and self-hostable.

Crawl visibilitySee every discovered URL

QSeed URL

example.com/pricingdocs.example.com/featuresblog.example.com/release

pricing · JSONfeatures · JSONchanges · JSON

Explore the open-source core ↗

Core API endpoints

MCP

Agent-native access

Node

Typed cloud SDK

MIT

Open-source core

[ 04 / 05 ] · USE CASES

04 / Use cases

One context layer.
Many workflows.

Research across the live web

Plan the query, rank sources, extract evidence, and synthesize with citations.

1Understand the objective2Discover high-signal sources3Map and crawl4Synthesize grounded context

Agent is working…Reading 24 sourcesStructuring outputKeeping source URLs

[ 05 / 05 ] · FAQ

05 / FAQ

Questions,
answered.

The essentials before your first request.

What is Qrawl?

Qrawl is an API and open-source engine for searching, scraping, mapping, and crawling public web content for AI systems.

What can I build with Qrawl?

Research agents, RAG ingestion, documentation discovery, support tools, and other workflows that need current web content.

Does Qrawl support AI agents and MCP?

Yes. Connect through the REST API, typed Node SDK, or the qrawl-mcp server for Claude, Cursor, Windsurf, and VS Code.

Is Qrawl open source?

Yes. The crawling engine is MIT licensed and self-hostable; the managed cloud adds authentication, quotas, jobs, and billing.

Does Qrawl respect robots.txt?

Yes. Crawls respect robots.txt by default and use per-domain delays to avoid hammering sites.

Give your agent a
live context layer.

Start with one endpoint. Search, scrape, map, and crawl through the API, SDK, or MCP.

Start building Explore the API

The live web,ready for your agents.