Algolia Crawler API

Algolia Crawler API allows developers to scrape and index data from websites in order to make it searchable and discoverable through Algolia's search engine. The API enables users to specify which pages and data they want to crawl, set up custom rules for how the data should be processed, and monitor the crawling process in real-time.