@scrapinghub

Scrapinghub

Loading…

Python 362 32

dateparser

python parser for human readable dates

Updated

otp

forked from erlang/otp

Erlang/OTP

Updated

aduana

Frontera backend to guide a crawl using PageRank, HITS or other ranking algorithms based on the link structure of the web graph, even when making big crawls (one billion pages).

Updated

Python 12 9

crawlera-tools

Crawlera tools

Updated

JavaScript 3,249 433

portia

Visual scraping for Scrapy

Updated

Python 46 15

frontera

A flexible frontier for web crawlers

Updated

docker-redmine

forked from sameersbn/docker-redmine

Dockerized redmine app server with a couple of pre-installed themes and plugins

Updated

Python 0 252

kafka-python

forked from mumrah/kafka-python

Python client for Apache Kafka

Updated

Python 0 0

doc.scrapinghub.com

Scrapinghub Documentation

Updated

Python 9 11

shub

Scrapinghub Command Line Client

Updated

Python 159 60

scrapylib

Collection of Scrapy utilities (extensions, middlewares, pipelines, etc)

Updated

Python 330 66

splash

Lightweight, scriptable browser as a service with an HTTP API

Updated

Shell 0 28

docker-kibana

forked from balsamiq/docker-kibana

Balsamiq kibana webapp docker container

Updated

Python 1 0

scrapinghub-entrypoint-scrapy

Scrapy entrypoint for Scrapinghub job runner

Updated

Erlang 0 331

mochiweb

forked from shaneaevans/mochiweb

MochiWeb is an Erlang library for building lightweight HTTP servers.

Updated

Python 229 72

scrapyjs

Scrapy+JavaScript integration

Updated

Python 7 2

python-cld2

Python bindings for CLD2.

Updated

Python 71 38

testspiders

Useful test spiders for Scrapy

Updated

Python 129 16

scrapyrt

Scrapy realtime

Updated

Shell 0 1

pkg-opengrok

Ubuntu packaging for OpenGrok

Updated