Skip to content

Commit

Permalink
Merge branch 'release/1.1.1'
Browse files Browse the repository at this point in the history
  • Loading branch information
Fabian Gieseke committed Dec 14, 2015
2 parents 8d6e5f3 + c178585 commit b40a277
Show file tree
Hide file tree
Showing 5 changed files with 31 additions and 28 deletions.
24 changes: 12 additions & 12 deletions README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -4,14 +4,24 @@ bufferkdtree

The bufferkdtree package is a Python library that aims at accelerating nearest neighbor computations using both k-d trees and modern many-core devices such as graphics processing units (GPUs). The implementation is based on `OpenCL <https://www.khronos.org/opencl/OpenCL>`_.

The implementation can be seen as an intermediate version between a standard parallel k-d tree traversal (on multi-core systems) and a massively-parallel brute-force implementation for nearest neigbhor search. In particular, it makes use of the top of a standard k-d tree (which induces a spatial subdivision of the space) and resorts to a simple yet efficient brute-force implementation for processing chunks of "big" leaves. The implementation is well-suited for data sets with a large reference set (e.g., 1,000,000 points) and a huge query set (e.g., 10,000,000 points) given a moderate dimensionality of the search space (e.g., from d=5 to d=25).
The buffer k-d tree technique can be seen as an intermediate version between a standard parallel k-d tree traversal (on multi-core systems) and a massively-parallel brute-force implementation for nearest neighbor search. In particular, it makes use of the top of a standard k-d tree (which induces a spatial subdivision of the space) and resorts to a simple yet efficient brute-force implementation for processing chunks of "big" leaves. The implementation is well-suited for data sets with a large reference set (e.g., 1,000,000 points) and a huge query set (e.g., 10,000,000 points) given a moderate dimensionality of the search space (e.g., from d=5 to d=50).

=============
Documentation
=============

See the `documentation <http://bufferkdtree.readthedocs.org>`_ for details and examples.

============
Dependencies
============

The bufferkdtree package is tested under Python 2.6 and Python 2.7. The required Python dependencies are:

- NumPy >= 1.6.1

Further, `Swig <http://www.swig.org>`_, `OpenCL <https://www.khronos.org/opencl/OpenCL>`_, `setuptools <https://pypi.python.org/pypi/setuptools>`_, and a working C/C++ compiler need to be available. See the `documentation <http://bufferkdtree.readthedocs.org>`_ for more details.

==========
Quickstart
==========
Expand All @@ -20,7 +30,7 @@ The package can easily be installed via pip via::

pip install bufferkdtree

To install the package from the sources, first get the current version via::
To install the package from the sources, first get the current stable release via::

git clone https://github.com/gieseke/bufferkdtree.git

Expand All @@ -33,16 +43,6 @@ On Debian/Ubuntu systems, the package can be installed globally for all users vi
python setup.py build
sudo python setup.py install

============
Dependencies
============

The bufferkdtree package is tested under Python 2.6 and Python 2.7. The required Python dependencies are:

- NumPy >= 1.6.1

Further, `Swig <http://www.swig.org>`_, `OpenCL <https://www.khronos.org/opencl/OpenCL>`_, `setuptools <https://pypi.python.org/pypi/setuptools>`_, and a working C/C++ compiler need to be available. See the `documentation <http://bufferkdtree.readthedocs.org>`_ for more details.

==========
Disclaimer
==========
Expand Down
2 changes: 1 addition & 1 deletion bufferkdtree/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,4 +6,4 @@

import os

__version__ = '1.1'
__version__ = '1.1.1'
4 changes: 4 additions & 0 deletions docs/changes.rst
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,10 @@
Changes
=======

Release 1.1.1 (Dezember 2015)
-----------------------------
* Updated documentation

Release 1.1 (Dezember 2015)
-----------------------------
* Fixed wrong parameter assignment in 'kneighbors' method of both neighbors/kd_tree/base.py and neighbors/buffer_kdtree/base.py
Expand Down
12 changes: 6 additions & 6 deletions docs/examples.rst
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
Examples
========

The following two examples sketch the use of the different implementations and can be found in the *examples* directory of the bufferkdtree package.
The following two examples sketch the use of the different implementations and can both be found in the *examples* subdirectory of the bufferkdtree package.

Toy Example
-----------
Expand All @@ -12,7 +12,7 @@ Toy Example
:start-after: # Licence: GNU GPL (v2)
:end-before: X = numpy.random.uniform(low=-1, high=1, size=(10000,10))

All implementations are provided via the ``NearestNeighbors`` class, which exhibits a similar layout as the corresponding class of the `scikit-learn <http://scikit-learn.org/stable/modules/generated/sklearn.neighbors.NearestNeighbors.html>`_ package. The parameter ``n_jobs`` determines the number of threads that shall be used by the standard k-d tree implementation (CPU). The parameter ``plat_dev_ids`` determines the OpenCL devices that shall be used by the buffer k-d tree implementation (OpenCL): Each key of the dictionary corresponds to a OpenCL platform id and for each platform id, a list of associated device ids can be provided. For this example, the first platform and its first device are used.
All implementations are provided via the ``NearestNeighbors`` class, which exhibits a similar layout as the corresponding class of the `scikit-learn <http://scikit-learn.org/stable/modules/generated/sklearn.neighbors.NearestNeighbors.html>`_ package. The parameter ``n_jobs`` determines the number of threads that shall be used by the standard k-d tree implementation (CPU). The parameter ``plat_dev_ids`` determines the OpenCL devices that shall be used by the buffer k-d tree implementation (OpenCL): Each key of the dictionary corresponds to a OpenCL platform id and for each platform id, a list of associated device ids can be provided. For instance, the first platform (with id 0) and its first device (with id 0) is used for the current example.

Next, a small artificial data set is generated, where ``X`` contains the points, one row per point:

Expand Down Expand Up @@ -49,7 +49,7 @@ For a detailed description of the remaining keywords, see the description of the

.. admonition:: Brute-Force

The brute-force implementatation is only used for comparison in relatively low-dimensional spaces; the performance is suboptimal for higher dimensional feature spaces (but superior over other matrix based implementations making use e.g., CUBLAS, for low-dimensional spaces).
Note that the brute-force implementatation is only used for comparison purposes given data sets in relatively low-dimensional search spaces. Its performance is suboptimal for high-dimensional feature spaces compared to matrix-based implementations that make use of e.g. CUBLAS (but superior to such implementations given low-dimensional search spaces).

Large-Scale Querying
--------------------
Expand All @@ -60,7 +60,7 @@ The main purpose of the buffer k-d tree implementation is to speed up the queryi
:start-after: # Licence: GNU GPL (v2)
:end-before: def run_algorithm(algorithm="buffer_kd_tree", tree_depth=None, leaf_size=None):

Note that four devices of the first platform are used now (0,1,2,3). The helper function defined next is used to time the runtimes needed for the training and testing phases of each method:
Note that four devices (with ids 0,1,2,3) of the first platform (with id 0) are used in this case. The helper function defined next is used to time the runtimes needed for the training and testing phases of each method:

.. literalinclude:: ../examples/astronomy.py
:start-after: n_neighbors=10
Expand All @@ -70,7 +70,7 @@ Note that either ``tree_depth`` or ``leaf_size`` is used to determine the final

.. literalinclude:: ../examples/astronomy.py
:start-after: # get/download data
:end-before: print "----------------------------------------------------------------------"
:end-before: print("----------------------------------------------------------------------")

Loading the data this way should yield an output like::

Expand All @@ -89,7 +89,7 @@ Loading the data this way should yield an output like::
Finally, both implementations are invoked to compute the 10 nearest neighbors for each query point:

.. literalinclude:: ../examples/astronomy.py
:start-after: print "----------------------------------------------------------------------"
:start-after: print("----------------------------------------------------------------------")

The above code yields the folling output on an *Ubuntu 14.04* system (64 bit) with an *Intel(R) Core(TM) i7-4790K* running at 4.00GHz (4 cores, 8 hardware threads), 32GB RAM, two *Geforce Titan Z* GPUs (with two devices each), CUDA 6.5 and Nvidia driver version 340.76::

Expand Down
17 changes: 8 additions & 9 deletions docs/install.rst
Original file line number Diff line number Diff line change
Expand Up @@ -3,18 +3,18 @@
Installation
============

Quick Installation
------------------

.. warning::

The authors are not responsible for any implications that stem from the use of this software.

Quick Installation
------------------

The package is available on `PyPI <https://pypi.python.org/pypi>`_, but can also be installed from the sources. For instance, to install the package via `PyPI <https://pypi.python.org/pypi>`_ on Linux machines, type::

$ sudo pip install bufferkdtree

To install the package from the sources, first get the current version via::
To install the package from the sources, first get the current stable release via::

$ git clone https://github.com/gieseke/bufferkdtree.git

Expand Down Expand Up @@ -50,24 +50,23 @@ On an OpenSUSE system, the corresponding command is::
OpenCL
------

OpenCL needs to be installed correctly. Make sure that the OpenCL header files are available, for example by setting the C_INCLUDE_PATH environment variable in the .bashrc file on Linux systems. For instance, in case CUDA is installed with header files being located in ``/usr/local/cuda/include``, then the following command should update the environment variable::
OpenCL needs to be installed correctly on the system. In addition, make sure that the OpenCL header files are available as well and accessible during the installation process, e.g., by setting the C_INCLUDE_PATH environment variable in the .bashrc file on Linux-based systems. For instance, given CUDA along with OpenCL, the header files are probably located in ``/usr/local/cuda/include``. Hence, the following command would update the environment variable accordingly (if needed)::

export C_INCLUDE_PATH=$C_INCLUDE_PATH:/usr/local/cuda/include

Virtualenv & Pip
----------------

We recommend to install the package via virtualenv and pip. On Ubuntu 12.04/14.04, for instance, the following commands can be used to install virtualenv and pip::
As for most Python packages, we recommend to make use of `virtualenv <https://pypi.python.org/pypi/virtualenv>`_ to install the package. To install virtualenv on recent Debian/Ubuntu-based systems, the following commands can be used to install virtualenv and pip::

$ sudo apt-get install python-virtualenv python-pip

Afterwards, create a new virtual environment and install the Numpy package::
Afterwards, a new virtual environment can be created to install the Numpy and the bufferkdtree package::

$ mkdir ~/.virtualenvs
$ cd ~/.virtualenvs
$ virtualenv bufferkdtree
$ source bufferkdtree/bin/activate
$ pip install numpy==1.6.1

Given the activated virtual environment, follow the instructions above to install the bufferkdtree package (Quick Installation).
$ pip install bufferkdtree

0 comments on commit b40a277

Please sign in to comment.