@scrapinghub

Scrapinghub

Loading…

aduana

Frontera backend to guide a crawl using PageRank, HITS or other ranking algorithms based on the link structure of the web graph, even when making big crawls (one billion pages).

Updated

Shell 0 95

docker-redmine

forked from sameersbn/docker-redmine

Dockerized redmine app server with a couple of pre-installed themes and plugins

Updated

Python 307 59

splash

Lightweight, scriptable browser as a service with an HTTP API

Updated

otp

forked from erlang/otp

Erlang/OTP

Updated

JavaScript 3,165 421

portia

Visual scraping for Scrapy

Updated

Python 39 14

frontera

A flexible frontier for web crawlers

Updated

Python 218 70

scrapyjs

Scrapy+JavaScript integration

Updated

Shell 0 1

pkg-opengrok

Ubuntu packaging for OpenGrok

Updated

Python 0 100

py-trello

forked from rocioar/py-trello

Python API wrapper around Trello's API

Updated

Python 0 0

doc.scrapinghub.com

Scrapinghub Documentation

Updated

Python 353 28

dateparser

python parser for human readable dates

Updated

Python 11 17

python-hubstorage

HubStorage client library

Updated

Python 156 60

scrapylib

Collection of Scrapy utilities (extensions, middlewares, pipelines, etc)

Updated

Python 27 4

adblockparser

Python parser for Adblock Plus filters

Updated

Python 123 16

scrapyrt

Scrapy realtime

Updated

Python 11 9

crawlera-tools

Crawlera tools

Updated

Python 70 37

testspiders

Useful test spiders for Scrapy

Updated

Python 0 46

python-intercom

forked from maiiku/python-intercom

Python wrapper for the Intercom API.

Updated

Python 7 11

shub

Scrapinghub Command Line Client

Updated

Python 16 1

skinfer

Simple tool to infer and/or merge JSON schemas

Updated