A fast Python implementation of locality sensitive hashing.
Python
Switch branches/tags
Nothing to show
Pull request Compare This branch is 7 commits ahead of kayzhu:master.
Latest commit c7b02f8 Mar 13, 2015 @simonemainardi Merge pull request #1 from wuan/python3compatibility
added python 3 compatibility and travis tests
Permalink
Failed to load latest commit information.
lshash added python 3 compatibility and travis tests Mar 12, 2015
tests added python 3 compatibility and travis tests Mar 12, 2015
.gitignore
.travis.yml added python 3 compatibility and travis tests Mar 12, 2015
CHANGES.rst fixes docs for PyPi, increment version to 0.0.3dev Dec 28, 2012
CONTRIBUTORS.txt
LICENSE Initialized README, added MIT license Dec 1, 2012
MANIFEST.in
README.rst Bumped up version to 0.0.4dev Apr 27, 2013
requirements.txt added python 3 compatibility and travis tests Mar 12, 2015
setup.py Fix imports and setup.py Dec 28, 2012

README.rst

LSHash

Version: 0.0.4dev

A fast Python implementation of locality sensitive hashing with persistance support.

Highlights

  • Fast hash calculation for large amount of high dimensional data through the use of numpy arrays.
  • Built-in support for persistency through Redis.
  • Multiple hash indexes support.
  • Built-in support for common distance/objective functions for ranking outputs.

Installation

LSHash depends on the following libraries:

  • numpy
  • redis (if persistency through Redis is needed)
  • bitarray (if hamming distance is used as distance function)

To install:

$ pip install lshash

Quickstart

To create 6-bit hashes for input data of 8 dimensions:

>>> from lshash import LSHash

>>> lsh = LSHash(6, 8)
>>> lsh.index([1,2,3,4,5,6,7,8])
>>> lsh.index([2,3,4,5,6,7,8,9])
>>> lsh.index([10,12,99,1,5,31,2,3])
>>> lsh.query([1,2,3,4,5,6,7,7])
[((1, 2, 3, 4, 5, 6, 7, 8), 1.0),
 ((2, 3, 4, 5, 6, 7, 8, 9), 11)]

Main Interface

  • To initialize a LSHash instance:
LSHash(hash_size, input_dim, num_of_hashtables=1, storage=None, matrices_filename=None, overwrite=False)

parameters:

hash_size:
The length of the resulting binary hash.
input_dim:
The dimension of the input vector.
num_hashtables = 1:
(optional) The number of hash tables used for multiple lookups.
storage = None:
(optional) Specify the name of the storage to be used for the index storage. Options include "redis".
matrices_filename = None:
(optional) Specify the path to the .npz file random matrices are stored or to be stored if the file does not exist yet
overwrite = False:
(optional) Whether to overwrite the matrices file if it already exist
  • To index a data point of a given LSHash instance, e.g., lsh:
lsh.index(input_point, extra_data=None):

parameters:

input_point:
The input data point is an array or tuple of numbers of input_dim.
extra_data = None:
(optional) Extra data to be added along with the input_point.
  • To query a data point against a given LSHash instance, e.g., lsh:
lsh.query(query_point, num_results=None, distance_func="euclidean"):

parameters:

query_point:
The query data point is an array or tuple of numbers of input_dim.
num_results = None:
(optional) The number of query results to return in ranked order. By default all results will be returned.
distance_func = "euclidean":
(optional) Distance function to use to rank the candidates. By default euclidean distance function will be used.