Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
HubStorage client library
Python

README.rst

HubStorage service client library

https://badge.fury.io/py/hubstorage.png https://secure.travis-ci.org/scrapinghub/python-hubstorage.png?branch=master

Note

This module is experimental and its API may change without previous notice.

Overview

This is the HubStorage client library, which contains:

  • Full client api trough hubstorage.HubstorageClient

Requirements

Basic API

Example creating a new job:

>>> from hubstorage import HubstorageClient
>>> hs = HubstorageClient(auth=apikey)
>>> job = hs.push_job(projectid='1111111', spidername='foo')
>>> job.key
'1111111/1/1'

>>> job.metadata['state']
'pending'

>>> job.items.write({'title': 'my first item'})
>>> job.logs.info('lorem impsum message are cool')
>>> job.logs.error('but sometimes s**t happens')
>>> job.finished()

Example getting job data later:

>> job = hs.get_job('1111111/1/1')
>> job.metadata['state']
'finished'

>> list(job.items.list(count=1))
[{'title': 'my first item'}]

...
Something went wrong with that request. Please try again.