No description, website, or topics provided.
Branch: master
Clone or download
Latest commit 366641e Apr 18, 2017
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
samples initial commit Feb 24, 2017
.editorconfig initial commit Feb 24, 2017
.gitignore initial commit Feb 24, 2017
License.txt Create License.txt Mar 20, 2017
package.json updated version Apr 3, 2017
readme.md Update readme.md Apr 18, 2017
webhoseio.js initial commit Feb 24, 2017

readme.md

webhose.io client for Node.js

npm version npm downloads

A simple way to access the Webhose.io API from your Node.js code

const webhoseio = require('webhoseio');

const client = webhoseio.config({token: 'YOUR_API_KEY'});
client.query('filterWebContent', {q: 'github'})
  .then(output => {
    console.log(output['posts'][0]['text']); // Print the text of the first post
    console.log(output['posts'][0]['published']); // Print the text of the first post publication date
});

// Get the next batch of posts
client.getNext()
  .then(output => {
    console.log(output['posts'][0]['thread']['site']); // Print the site of the first post
  });

API Key

To make use of the webhose.io API, you need to obtain a token that would be used on every request. To obtain an API key, create an account at https://webhose.io/auth/signup, and then go into https://webhose.io/dashboard to see your token.

Installing

You can install using npm:

$ npm install webhoseio

Use the API

To get started, you need to import the library, and set your access token. (Replace YOUR_API_KEY with your actual API key).

const webhoseio = require('webhoseio');

const client = webhoseio.config({token: 'YOUR_API_KEY'});

API Endpoints

The first parameter the query() function accepts is the API endpoint string. Available endpoints:

  • filterWebContent - access to the news/blogs/forums/reviews API
  • productFilter - access to data about eCommerce products/services
  • darkFilter - access to the dark web (coming soon)

Now you can make a request and inspect the results:

client.query('filterWebContent', {q: 'github'})
  .then(output => {
    console.log(output['totalResults']);
    // 15565094

    console.log(output['posts'].length);
    // 100

    console.log(output['posts'][0]['language']);
    // english

    console.log(output['posts'][0]['title']);
    // Putting quotes around dictionary keys in JS
});

For your convenience, functions query and getNext both return Promise with one argument - the response JSON, so you can loop over it and get all the results of this batch (up to 100).

client.query('filterWebContent', {q: 'github'})
  .then(output => {
    let totalWords = output['posts'].reduce((sum, post) => {
      return sum + post['text'].split(' ').length}, 0);
    console.log(totalWords);
    // 8822
  });

Full documentation

  • config({token})

    • token - your API key
  • query(end_point_str, params)

    • end_point_str:
      • filterWebContent - access to the news/blogs/forums/reviews API
      • productFilter - access to data about eCommerce products/services
      • darkFilter - access to the dark web (coming soon)
    • params: A key value dictionary. The most common key is the "q" parameter that hold the filters Boolean query. Read about the available filters.
  • getNext() - a method to fetch the next page of results.

Polling

If you want to make repeated searches, performing an action whenever there are new results, use code like this:

const client = webhoseio.config({token: 'YOUR_API_KEY'});
let r = client.query('filterWebContent', {q: 'github'});

setInterval(() => {
  r.then(output => {
    output['posts'].map(post => performAction(post));
    return client.getNext();
  });
}, 300);