Find file Copy path
Fetching contributors…
Cannot retrieve contributors at this time
56 lines (40 sloc) 1.67 KB
title tags keywords last_update summary sidebar permalink folder fa_icon complexity toc
Getting Started
guide getting-started
October, 27, 2016
Help you get on track for using OnCrawl API.

3rd Party Integration

If you want users of your service take benefits of their OnCrawl data within your service, OnCrawl API supports OAuth2. But first you need to register your application.

Register your application

OnCrawl API is still in beta. To register your application, please fill the form.

We will then provide you a consumer key and consumer secret. Those keys are required to perform an OAuth2 challenge to retrieve a session token, described in the OAuth2 page.

Make request like an OnCrawl user

If it possible to authenticate using the username and password of OnCrawl users to obtain a token that allows to request our API.

If you want to either tests our API or simply request your own data, this is the simplest solution.

Data Model

OnCrawl API allows to deal with objects and operations.

The top-level object is the Project. A Project is made of:

  • Attributes: Name, Start URL, User agent, ...
  • List of Crawl Configurations
  • List of Crawls

A CrawlConfiguration is only made of attributes that belong to a Crawl.

A Crawl is made of attributes like the Crawl period, the CrawlConfiguration that has been use. But the most important is its identifier, which allows you to search for pages, or perform data aggregation.

More information about the available requests in the API Reference.