Merge pull request #137 from fbcotter/tf

TF Functionality

.travis.yml

@@ -7,10 +7,12 @@ before_install:
     - sudo apt-get install -y software-properties-common
     - sudo add-apt-repository -y "deb http://us.archive.ubuntu.com/ubuntu/ trusty universe multiverse restricted"
     - sudo apt-get update -qq
-    - sudo apt-get install -y opencl-headers fglrx
+    - sudo apt-get install -y opencl-headers fglrx ocl-icd-opencl-dev
 env:
     - TOX_ENV=py3
     - TOX_ENV=py27
+    - TOX_ENV=py27-tf
+    - TOX_ENV=py3-tf
     - TOX_ENV=docs
 install:
     - pip install --upgrade pip

docs/backends.rst

@@ -1,10 +1,12 @@
 Multiple Backend Support
 ========================
 
-The ``dtcwt`` library currently provides two backends for computing the wavelet
-transform: a `NumPy <http://www.numpy.org/>`_ based implementation and an OpenCL
+
+The ``dtcwt`` library currently provides three backends for computing the wavelet
+transform: a `NumPy <http://www.numpy.org/>`_ based implementation, an OpenCL
 implementation which uses the `PyOpenCL <http://mathema.tician.de/software/pyopencl/>`_
-bindings for Python.
+bindings for Python, and a Tensorflow implementation which uses the 
+`Tensorflow <https://www.tensorflow.org>`_ bindings for Python.
 
 NumPy
 '''''
@@ -26,27 +28,59 @@ may not be full-featured.
 OpenCL support depends on the `PyOpenCL
 <http://mathema.tician.de/software/pyopencl/>`_ package being installed and an
 OpenCL implementation being installed on your machine. Attempting to use an
-OpenCL backen without both of these being present will result in a runtime (but
+OpenCL backend without both of these being present will result in a runtime (but
 not import-time) exception.
 
+Tensorflow
+''''''''''
+
+If you want to take advantage of having a GPU on your machine, 
+some transforms and algorithms have been implemented with a Tensorflow backend.
+This backend will provide an identical API to the NumPy backend.
+I.e. NumPy-based input may be passed to a tensorflow backend in the same manner
+as it was passed to the NumPy backend. In which case it
+will be converted to a tensorflow variable, the transform performed, and then
+converted back to a NumPy variable afterwards. This conversion between types can
+be avoided if a tensorflow variable is passed to the dtcwt Transforms.
+
+The real speedup gained from using GPUs is obtained by parallel processing. For
+this reason, when using the tensorflow backend, the Transforms can accept
+batches of images. To do this, see the `forward_channels` and `inverse_channels`
+methods. More information is in the :ref:`tensorflowbackend` section.
+
+Tensorflow support depends on the 
+`Tensorflow <https://www.tensorflow.org/install/>`_ python package being installed in the
+current python environment, as well as the necessary CUDA + CUDNN libraries
+installed). Attempting to use a Tensorflow backend without the python package
+available will result in a runtime (but not import-time) exception. Attempting
+to use the Tensorflow backend without the CUDA and CUDNN libraries properly
+installed and linked will result in the Tensorflow backend being used, but
+operations will be run on the CPU rather than the GPU.
+
+If you do not have a GPU, some speedup can still be seen for using Tensorflow with
+the CPU vs the plain NumPy backend, as tensorflow will naturally use multiple
+processors.  
+
 Which backend should I use?
 '''''''''''''''''''''''''''
 
-The top-level transform routines, such as :py:class`dtcwt.Transform2d`, will
+The top-level transform routines, such as :py:class:`dtcwt.Transform2d`, will
 automatically use the NumPy backend. If you are not primarily focussed on
 speed, this is the correct choice since the NumPy backend has the fullest
 feature support, is the best tested and behaves correctly given single- and
 double-precision input.
 
 If you care about speed and need only single-precision calculations, the OpenCL
-backend can provide significant speed-up. On the author's system, the 2D
-transform sees around a times 10 speed improvement.
+or Tensorflow backends can provide significant speed-up. 
+On the author's system, the 2D transform sees around a times 10 speed
+improvement for the OpenCL backend, and a 8-10 times speed up for the Tensorflow
+backend.
 
 Using a backend
 '''''''''''''''
 
-The NumPy and OpenCL backends live in the :py:mod:`dtcwt.numpy`
-and :py:mod:`dtcwt.opencl` modules respectively. Both provide
+The NumPy, OpenCL and Tensorflow backends live in the :py:mod:`dtcwt.numpy`,
+:py:mod:`dtcwt.opencl`, and :py:mod:`dtcwt.tf` modules respectively. All provide
 implementations of some subset of the DTCWT library functionality.
 
 Access to the 2D transform is via a :py:class:`dtcwt.Transform2d` instance. For
@@ -72,10 +106,19 @@ switch to the OpenCL backend
 .. code-block:: python
 
     dtcwt.push_backend('opencl')
+    xfm = Transform2d()
     # ... Transform2d, etc now use OpenCL ...
 
-As is suggested by the name, changing the backend manipulates a stack behind
-the scenes and so one can temporarily switch backend using
+and to switch to the Tensorflow backend
+
+.. code-block:: python
+    
+    dtcwt.push_backend('tf')
+    xfm = Transform2d()
+    # ... Transform2d, etc now use Tensorflow ...
+
+As is suggested by the name, changing the backend manipulates a stack behind the
+scenes and so one can temporarily switch backend using
 :py:func:`dtcwt.push_backend` and :py:func:`dtcwt.pop_backend`
 
 .. code-block:: python

docs/reference.rst

@@ -74,3 +74,23 @@ OpenCL
 .. automodule:: dtcwt.opencl.lowlevel
     :members:
 
+.. _tensorflowbackend:
+
+Tensorflow
+''''''''''
+Currently the Tensorflow backend only supports single precision operations, and
+only has functionality for the Transform1d() and Transform2d() classes (i.e. 
+changing the backend to 'tf' will still use the numpy Transform3d() class).
+
+To preserve functionality, the Transform1d() and Transform2d() classes have
+a `forward` method which behaves identically to the NumPy backend. However, to
+get speedups with tensorflow, we want to feed our transform batches of images.
+For this reason, the 1-D and 2-D transforms also have `forward_channels` and
+`inverse_channels` methods. See the below documentation for how to use these.
+
+.. automodule:: dtcwt.tf
+    :members:
+    :inherited-members:
+
+.. automodule:: dtcwt.tf.lowlevel
+    :members:

dtcwt/__init__.py

@@ -19,6 +19,7 @@
 
 import dtcwt.numpy
 import dtcwt.opencl
+import dtcwt.tf
 
 # An array of dictionaries. Each dictionary stores the top-level module
 # variables for that backend.
@@ -38,6 +39,12 @@
         'Transform3d': dtcwt.numpy.Transform3d,
         'Pyramid': dtcwt.opencl.Pyramid,
     },
+    'tf': {
+        'Transform1d': dtcwt.tf.Transform1d,
+        'Transform2d': dtcwt.tf.Transform2d,
+        'Transform3d': dtcwt.numpy.Transform3d,
+        'Pyramid': dtcwt.tf.Pyramid,
+    },
 }
 
 def _update_from_current_backend():

dtcwt/_version.py

@@ -1,2 +1,2 @@
 # IMPORTANT: before release, remove the 'devN' tag from the release name
-__version__ = '0.12.0dev1'
+__version__ = '0.13.0dev1'

dtcwt/numpy/transform2d.py

@@ -74,8 +74,10 @@ def forward(self, X, nlevels=3, include_scale=False):
         original_size = X.shape
 
         if len(X.shape) >= 3:
-            raise ValueError('The entered image is {0}, please enter each image slice separately.'.
-                    format('x'.join(list(str(s) for s in X.shape))))
+            raise ValueError('The entered image is {0}, which is invalid '.
+                             format('x'.join(list(str(s) for s in X.shape))) +
+                             'for the 2D transform in a numpy backend. ' +
+                             'Please enter each image slice separately.')
 
         # The next few lines of code check to see if the image is odd in size, if so an extra ...
         # row/column will be added to the bottom/right of the image
@@ -150,9 +152,9 @@ def forward(self, X, nlevels=3, include_scale=False):
             Yh[level][:,:,0:6:5] = q2c(coldfilt(Hi,h0b,h0a).T)  # Horizontal
             Yh[level][:,:,2:4:1] = q2c(coldfilt(Lo,h1b,h1a).T)  # Vertical
             if len(self.qshift) >= 12:
-                Yh[level][:,:,1:5:3] = q2c(coldfilt(Ba,h2b,h2a).T)  # Diagonal   
+                Yh[level][:,:,1:5:3] = q2c(coldfilt(Ba,h2b,h2a).T)  # Diagonal
             else:
-                Yh[level][:,:,1:5:3] = q2c(coldfilt(Hi,h1b,h1a).T)  # Diagonal   
+                Yh[level][:,:,1:5:3] = q2c(coldfilt(Hi,h1b,h1a).T)  # Diagonal
 
             if include_scale:
                 Yscale[level] = LoLo
@@ -267,7 +269,7 @@ def inverse(self, pyramid, gain_mask=None):
 
             if np.any(np.array(Z.shape) != S[:2]):
                 raise ValueError('Sizes of highpasses are not valid for DTWAVEIFM2')
-            
+
             current_level = current_level - 1
 
         if current_level == 1:
@@ -300,7 +302,7 @@ def q2c(y):
     """
     Convert from quads in y to complex numbers in z.
     """
-    
+
     j2 = (np.sqrt(0.5) * np.array([1, 1j])).astype(appropriate_complex_type_for(y))
 
     # Arrange pixels from the corners of the quads into
@@ -310,7 +312,7 @@ def q2c(y):
     #  |    |
     #  c----d
 
-    # Combine (a,b) and (d,c) to form two complex subimages. 
+    # Combine (a,b) and (d,c) to form two complex subimages.
     p = y[0::2, 0::2]*j2[0] + y[0::2, 1::2]*j2[1] # p = (a + jb) / sqrt(2)
     q = y[1::2, 1::2]*j2[0] - y[1::2, 0::2]*j2[1] # q = (d - jc) / sqrt(2)
 

dtcwt/tf/__init__.py

@@ -0,0 +1,16 @@
+"""
+Provide low-level Tensorflow accelerated operations. This backend requires that
+Tensorflow be installed. Works best with a GPU but still offers good
+improvements with a CPU.
+
+"""
+
+from .common import Pyramid
+from .transform1d import Transform1d
+from .transform2d import Transform2d
+
+__all__ = [
+    'Pyramid',
+    'Transform1d',
+    'Transform2d',
+]

dtcwt/tf/common.py

@@ -0,0 +1,75 @@
+from __future__ import absolute_import
+
+try:
+    import tensorflow as tf
+except ImportError:
+    # The lack of tensorflow will be caught by the low-level routines.
+    pass
+
+
+class Pyramid(object):
+    """A tensorflow representation of a transform domain signal.
+
+    An interface-compatible version of
+    :py:class:`dtcwt.Pyramid` where the initialiser
+    arguments are assumed to be :py:class:`tf.Variable` instances.
+
+    The attributes defined in :py:class:`dtcwt.Pyramid`
+    are implemented via properties. The original tf arrays may be accessed
+    via the ``..._op(s)`` attributes.
+
+    .. py:attribute:: lowpass_op
+
+        A tensorflow tensor that can be evaluated in a session to return
+        the coarsest scale lowpass signal for the input, X.
+
+    .. py:attribute:: highpasses_op
+
+        A tuple of tensorflow tensors, where each element is the complex
+        subband coefficients for corresponding scales finest to coarsest.
+
+    .. py:attribute:: scales_ops
+
+        *(optional)* A tuple where each element is a tensorflow tensor
+        containing the lowpass signal for corresponding scales finest to
+        coarsest. This is not required for the inverse and may be *None*.
+    """
+    def __init__(self, lowpass, highpasses, scales=None, numpy=False):
+        self.lowpass_op = lowpass
+        self.highpasses_ops = highpasses
+        self.scales_ops = scales
+        self.numpy = numpy
+
+    @property
+    def lowpass(self):
+        if not hasattr(self, '_lowpass'):
+            if self.lowpass_op is None:
+                self._lowpass = None
+            else:
+                with tf.Session() as sess:
+                    sess.run(tf.global_variables_initializer())
+                    self._lowpass = sess.run(self.lowpass_op)
+        return self._lowpass
+
+    @property
+    def highpasses(self):
+        if not hasattr(self, '_highpasses'):
+            if self.highpasses_ops is None:
+                self._highpasses = None
+            else:
+                with tf.Session() as sess:
+                    sess.run(tf.global_variables_initializer())
+                    self._highpasses = \
+                        tuple(sess.run(x) for x in self.highpasses_ops)
+        return self._highpasses
+
+    @property
+    def scales(self):
+        if not hasattr(self, '_scales'):
+            if self.scales_ops is None:
+                self._scales = None
+            else:
+                with tf.Session() as sess:
+                    sess.run(tf.global_variables_initializer())
+                    self._scales = tuple(sess.run(x) for x in self.scales_ops)
+        return self._scales