Skip to content

gbif/thumbor_hbase

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

HBase Storage Module for Thumbor

Build Status

Introduction

Thumbor is a smart imaging service. It enables on-demand crop, resizing and flipping of images.

HBase is a column oriented database from the Hadoop ecosystem.

This module provide support for Hadoop HBase as a large auto replicant key/value backend storage for images in Thumbor.

Installation

The current version of the module is 0.20.

In order to install the HBase Storage Module for Thumbor, you have to install Hadoop / HBase ecosystem first.

Hadoop / HBase installation

The HBase Storage Module for Thumbor was originally developed and tested on Cloudera CDH3 Hadoop on a Debian system.

The migration to Python 3 and Thumbor 7 was tested on Cloudera CDH5 Hadoop on a CentOS system.

You will need to install HBase and HBase-Thrift.

Thumbor installation

You have to install Thumbor following the Thumbor Installation Guide.

HBase Storage Module installation

Finally, install the HBase Storage Module :

pip install git+https://github.com/gbif/thumbor_hbase

Usage

Using it is simple, just change your configuration in thumbor.conf:

HBASE_STORAGE_SERVER_HOST = 'localhost'
HBASE_STORAGE_SERVER_PORT = 9090
HBASE_STORAGE_TABLE = b'storage-table'
HBASE_STORAGE_FAMILY = b'storage-family'

and create a suitable table in HBase. (You may well want to investigate Medium Object Storage and adjust this command accordingly):

create 'thumbor', 'image'

If you want to use thumbor_hbase for loading original images, change your thumbor.conf to read:

LOADER = "thumbor_hbase.loader"

If you want to use thumbor_hbase for storage of original images, change your thumbor.conf to read:

STORAGE = "thumbor_hbase.storage"

Testing

In order to execute pyvows tests, you have to install pyvows :

pip install pyvows

and run tests with :

pyvows -c -l thumbor_hbase

License

Licensed under the MIT license:
https://opensource.org/license/mit/
Copyright © 2011 globo.com timehome@corp.globo.com

About

HBase storage module for Thumbor

Resources

Stars

Watchers

Forks

Packages

No packages published

Languages

  • Python 100.0%