WebScrapingApi NodeJS SDK
WebScrapingApi is an API that allows scraping websites while using rotating proxies to prevent bans. This SDK for NodeJS makes the usage of the API easier to implement in any project you have.
Run the following command in the main folder of your project:
npm i webscrapingapi --save
To use the API and the SDK you will need a API Key. You can get one by registering at WebScrapingApi
Using the SDK it's quite easy. An example of a GET call to the API is the following:
For a better understanding of the parameters, please check out our documentation
- Added support for
- Switched from
got was required in order to implement the
http.maxHeaderSize property. This property allows us to overwrite NodeJS' default maximum allowed http header size (
--max-http-header-size) and overcome scenarios in which certain scraped websites (i.e. Wallmart), would result in a
Header overflow error, becasue the scraped headers size would exceed the maximum 16KiB allowed by NodeJS.
Important! Please note that the response structure in
got response structure and the Object received is differeent compared to previous versions! If you update
webscrapingapi in your current project, please also consider updating the way you're handling the response!