Scrape a bunch of URLs for tags, get an array of strings back
This is to be run server side, not client side. Do it to pull down some fun text from websites, to use in twitter bots or wherever.
Install via npm:
$ npm i node-scraper-to-text --save
As a module:
const NodeScraperToText = require('node-scraper-to-text');
const options = {};
const nodeScraperToText = new NodeScraperToText(options);
You can pass in some actual options, too :)
const NodeScraperToText = require('node-scraper-to-text');
const options = {
shouldSplit: true,
urls: ['https://reddit.com'],
tags: ['a.title']
};
const nodeScraperToText = new NodeScraperToText(options);
Instantiation options.
@param {boolean} shouldSplit
- Whether to split the text by periods or not.@param {array} urls
- An array of URL strings@param {array} tags
- An array of tag strings (jQuery type strings, they will be searched like$(tag)
)@param {Boolean | String} save
- Whether to save the file (pass a location for the file if you want it to save, e.g. "./my-dumb-file.js"). Will save the file asmodule.exports = {"sentences": [... your scraped sentences array]}
If you notice anything wrong, or want me to add something, feel free to file an issue here.