Crawlee

Crawlee JavaScript SDK

The Crawlee JavaScript SDK is a Node.js/TypeScript library for building reliable web scrapers and crawlers. It provides a family of crawler classes - BasicCrawler, HttpCrawler, CheerioCrawler, JSDOMCrawler, LinkeDOMCrawler, PuppeteerCrawler, PlaywrightCrawler, and AdaptivePlaywrightCrawler - along with shared infrastructure for AutoscaledPool resource management, proxy rotation, session pooling, RequestQueue task queuing, Dataset result storage, and KeyValueStore unstructured data persistence. Crawlee handles retries, error recovery, request fingerprinting, and statistics tracking out of the box, allowing developers to focus on extraction logic.

GitHub

Documentation

📖
Documentation
https://crawlee.dev/js
📖
GettingStarted
https://crawlee.dev/js/docs/quick-start

SDKs

📦
GitHubRepository
https://github.com/apify/crawlee

Other Resources

🔗
Reference
https://crawlee.dev/js/api
🔗
NpmPackage
https://www.npmjs.com/package/crawlee

API entry from apis.yml

apis.yml Raw ↑ 
aid: crawlee:crawlee-javascript-sdk
name: Crawlee JavaScript SDK
description: The Crawlee JavaScript SDK is a Node.js/TypeScript library for building reliable web scrapers
  and crawlers. It provides a family of crawler classes - BasicCrawler, HttpCrawler, CheerioCrawler, JSDOMCrawler,
  LinkeDOMCrawler, PuppeteerCrawler, PlaywrightCrawler, and AdaptivePlaywrightCrawler - along with shared
  infrastructure for AutoscaledPool resource management, proxy rotation, session pooling, RequestQueue
  task queuing, Dataset result storage, and KeyValueStore unstructured data persistence. Crawlee handles
  retries, error recovery, request fingerprinting, and statistics tracking out of the box, allowing developers
  to focus on extraction logic.
humanURL: https://crawlee.dev/js
properties:
- type: Documentation
  url: https://crawlee.dev/js
- type: Reference
  url: https://crawlee.dev/js/api
- type: GettingStarted
  url: https://crawlee.dev/js/docs/quick-start
- type: GitHubRepository
  url: https://github.com/apify/crawlee
- type: NpmPackage
  url: https://www.npmjs.com/package/crawlee
tags:
- Browser Automation
- Cheerio
- JavaScript
- Node.js
- Playwright
- Puppeteer
- Scraping
- TypeScript