Screaming Frog SEO Spider
The Screaming Frog SEO Spider is a desktop website crawler for Windows, macOS, and Ubuntu that performs comprehensive technical SEO audits. It crawls websites to find broken links, analyze page titles and meta data, discover duplicate content, generate XML sitemaps, and integrate with third-party APIs including Google Analytics, Google Search Console, PageSpeed Insights, Ahrefs, Majestic, Moz, and AI services (OpenAI, Gemini, Ollama, Anthropic). The Spider acts as an API consumer, pulling external metrics and data into crawl reports.
Documentation
Examples
Schemas & Data
JSONSchema
https://raw.githubusercontent.com/api-evangelist/screaming-frog/refs/heads/main/json-schema/screaming-frog-crawl-result-schema.json
JSONStructure
https://raw.githubusercontent.com/api-evangelist/screaming-frog/refs/heads/main/json-structure/screaming-frog-crawl-result-structure.json
Other Resources
Pricing
https://www.screamingfrog.co.uk/seo-spider/pricing/
ReleaseHistory
https://www.screamingfrog.co.uk/seo-spider/release-history/
JSONLDContext
https://raw.githubusercontent.com/api-evangelist/screaming-frog/refs/heads/main/json-ld/screaming-frog-context.jsonld
Vocabulary
https://raw.githubusercontent.com/api-evangelist/screaming-frog/refs/heads/main/vocabulary/screaming-frog-vocabulary.yml