python 3 compatible. fixed some more pytest problems.

.gitignore

@@ -51,7 +51,7 @@ coverage.xml
 *.log
 
 # Sphinx documentation
-docs/_build/
+docs/build/
 
 # PyBuilder
 target/

CHANGELOG.rst

@@ -1,7 +1,10 @@
-Change log
-----------
+What's new in cuvarbase
+***********************
+
 * **0.1.9**
 	* Added Sphinx documentation
+	* **Now Python 3 compatible!**
+    * Miscillaneous bug fixes
 	* CE
 		* Run functions for ``ConditionalEntropyAsyncProcess`` now allow for a ``balanced_magbins`` argument to set the magnitude bins to have widths that vary with the distribution of magnitude values. This is more robust to outliers, but performance comparisons between the usual CE algorithm indicate that you should use care.
 		* Added ``precompute`` function to ``ConditionalEntropyAsyncProcess`` that allows you to speed up computations without resorting to the ``batched_run_constant_nfreq`` function. Currently it still assumes that the frequencies used will be the same for all lightcurves.

INSTALL.rst

@@ -1,63 +1,137 @@
 Install instructions
 ********************
 
+These installation instructions are for Linux/BSD-based systems (OS X/macOS, Ubuntu, etc.). Windows users, your suggestions and feedback is welcome if we can make your life easier!
+
+Installing the Nvidia Toolkit
+-----------------------------
+
+``cuvarbase`` requires PyCUDA and scikit-cuda, which both require the Nvidia toolkit for access to the Nvidia compiler, drivers, and runtime libraries.
+
+Go to the `NVIDIA Download page <https://developer.nvidia.com/cuda-downloads>`_ and select the distribution for your operating system. Everything has been developed and tested using **version 8.0**, so it may be best to stick with that version for now until we verify that later versions are OK.
+
+.. warning::
+
+	Make sure that your ``$PATH`` environment variable contains the location of the ``CUDA`` binaries. You can test this by trying
+	``which nvcc`` from your terminal. If nothing is printed, you'll have to amend your ``~/.bashrc`` file: 
+
+	``echo "export PATH=/usr/local/cuda/bin:${PATH}" >> ~/.bashrc && . ~/.bashrc``
+
+	The ``>>`` is not a typo -- using one ``>`` will *overwrite* the ``~/.bashrc`` file. Make sure you change ``/usr/local/cuda`` to the appropriate location of your Nvidia install.
+
+	**Also important**
+
+	Make sure your ``$LD_LIBRARY_PATH`` and ``$DYLD_LIBRARY_PATH`` are also similarly modified to include the ``/lib`` directory of the CUDA install:
+
+	``echo "export LD_LIBRARY_PATH=/usr/local/cuda/lib:${LD_LIBRARY_PATH}" >> ~/.bashrc && . ~/.bashrc``
+	``echo "export DYLD_LIBRARY_PATH=/usr/local/cuda/lib:${DYLD_LIBRARY_PATH}" >> ~/.bashrc && . ~/.bashrc``
+
+
 Using conda
 -----------
 
 `Conda <https://www.continuum.io/downloads>`_ is a great way to do this in a safe, isolated environment.
 
-First create a new conda environment (named ``pycu`` here) that uses Python 2.7 (future versions should be compatible with 3.x), and install some required dependencies (``numpy``, ``astropy`` and ``matplotlib``).
+First create a new conda environment (named ``pycu`` here) that will use Python 2.7 (python 2.7, 3.4, 3.5, and 3.6
+have been tested), with the numpy library installed. 
 
 .. code:: bash
 
-	conda create -n pycu python=2.7 numpy astropy matplotlib
+	conda create -n pycu python=2.7 numpy
+
+.. note::
+
+	The numpy library *has* to be installed *before* PyCUDA is installed with pip. 
+	The PyCUDA setup needs to be able to access the numpy library for building against it. You can do this with
+	the above command, or alternatively just do ``pip install numpy && pip install cuvarbase``
 
 Then activate the virtual environment
 
 .. code:: bash
 
 	source activate pycu
 
-and use ``pip`` to install the other dependencies.
+and then use ``pip`` to install ``cuvarbase``
 
 .. code:: bash
 
-	pip install nfft scikit-cuda pycuda
+	pip install cuvarbase
 
-You should test if ``pycuda`` is working correctly.
+
+Installing with just ``pip``
+----------------------------
+
+**If you don't want to use conda** the following should work with just pip
 
 .. code:: bash
 
-	python -c "import pycuda.autoinit"
+	pip install numpy 
+	pip install cuvarbase
 
-If everything works up until now, we should be ready to install ``cuvarbase``
 
-.. code:: bash
+Troubleshooting PyCUDA installation problems
+--------------------------------------------
 
+The ``PyCUDA`` installation step may be a hiccup in this otherwise orderly process. If you run into problems installing ``PyCUDA`` with pip, you may have to install PyCUDA from source yourself. It's not too bad, but if you experience any problems, please submit an `Issue <https://github.com/johnh2o2/cuvarbase/issues>`_ at the ``cuvarbase`` Github page and I'll amend this documentation.
+
+Below is a small bash script that (hopefully) automates the process of installing PyCUDA in the event of any problems you've encountered at this point.
+
+.. code-block:: bash
+	
+	PYCUDA="pycuda-2017.1.1"
+	PYCUDA_URL="https://pypi.python.org/packages/b3/30/9e1c0a4c10e90b4c59ca7aa3c518e96f37aabcac73ffe6b5d9658f6ef843/pycuda-2017.1.1.tar.gz#md5=9e509f53a23e062b31049eb8220b2e3d"
+	CUDA_ROOT=/usr/local/cuda
+
+	# Download
+	wget $PYCUDA_URL
+
+	# Unpack
+	tar xvf ${PYCUDA}.tar.gz
+	cd $PYCUDA
+
+	# Configure with current python exe
+	./configure.py --python-exe=`which python` --cuda-root=$CUDA_ROOT
+	python setup.py build
 	python setup.py install
 
-and run the unit tests
+If everything goes smoothly, you should now test if ``pycuda`` is working correctly.
 
 .. code:: bash
 
-	py.test cuvarbase
+	python -c "import pycuda.autoinit; print 'Hurray!'"
+
+If everything works up until now, we should be ready to install ``cuvarbase``
+
+.. code:: bash
 
+	pip install cuvarbase
 
-Using pip
----------
+Installing from source
+----------------------
 
-**If you don't want to use conda** the following should work with just pip (assuming you're using Python 2.7):
+You can also install directly from the repository. Clone the ``git`` repository on your machine:
 
 .. code:: bash
+	
+	git clone https://github.com/johnh2o2/cuvarbase
 
-	pip install numpy scikit-cuda pycuda astropy nfft matplotlib
+Then install!
+
+.. code:: bash
+
+	cd cuvarbase
 	python setup.py install
-	py.test cuvarbase
 
+The last command can also be done with pip:
+
+.. code:: bash
+
+	pip install -e .
 
 
-Installing on a Mac
--------------------
+
+Troubleshooting on a Mac
+------------------------
 
 Nvidia offers `CUDA for Mac OSX <https://developer.nvidia.com/cuda-downloads>`_. After installing the
 package via downloading and running the ``.dmg`` file, you'll have to make a couple of edits to your
@@ -73,7 +147,7 @@ and then source these changes in your current shell by running ``. ~/.bash_profi
 Another important note: **nvcc (8.0.61) does not appear to support the latest clang compiler**. If this is
 the case, running ``python example.py`` should produce the following error:
 
-.. code::
+.. code:: bash
 
     nvcc fatal   : The version ('80100') of the host compiler ('Apple clang') is not supported
 

README.rst

@@ -1,9 +1,10 @@
 cuvarbase 0.1.9
 ===============
 
-John Hoffman -- 2017
+John Hoffman
+(c) 2017
 
-``cuvarbase`` is a Python (2.7) library that uses `PyCUDA <https://mathema.tician.de/software/pycuda/>`_ to implement several time series tools used in astronomy to GPUs.
+``cuvarbase`` is a Python library that uses `PyCUDA <https://mathema.tician.de/software/pycuda/>`_ to implement several time series tools used in astronomy to GPUs.
 
 This project is under active development, and currently includes implementations of
 

cuvarbase/bls.py

@@ -5,7 +5,10 @@
 .. [K2002] `Kovacs et al. 2002 <http://adsabs.harvard.edu/abs/2002A%26A...391..369K>`_
 
 """
+from __future__ import print_function, division
 
+from builtins import zip
+from builtins import range
 import sys
 import pycuda.driver as cuda
 import pycuda.gpuarray as gpuarray
@@ -370,7 +373,7 @@ def eebls_gpu_custom(t, y, dy, freqs, q_values, phi_values,
     best_q = bls_best_q.get()
     best_phi = bls_best_phi.get()
 
-    qphi_sols = zip(best_q, best_phi)
+    qphi_sols = list(zip(best_q, best_phi))
 
     return bls_g.get()/YY, qphi_sols
 
@@ -575,7 +578,7 @@ def locext(ext, arr, imin=None, imax=None):
     best_q = bls_best_q.get()
     best_phi = bls_best_phi.get()
 
-    qphi_sols = zip(best_q, best_phi)
+    qphi_sols = list(zip(best_q, best_phi))
 
     return bls_g.get()/YY, qphi_sols
 
@@ -669,7 +672,6 @@ def hone_solution(t, y, dy, f0, df0, q0, dlogq0, phi0, stop=1e-5,
         nf = int((fmax - fmin)/df)
         freqs = np.linspace(fmin, fmax + df, nf)
 
-        print (phimin, phimax, qmin, qmax, q, dphi, nphi)
         powers, sols = eebls_gpu_custom(t, y, dy, freqs, q_values, phi_values,
                                         batch_size=5, nstreams=5,
                                         functions=functions, **kwargs)
@@ -679,7 +681,6 @@ def hone_solution(t, y, dy, f0, df0, q0, dlogq0, phi0, stop=1e-5,
         q, phi = sols[ibest]
         pn = powers[ibest]
         i += 1
-        print i, (pn - p0) / p0, q, phi, pn
     return f, pn, i, (q, phi)
 
 

cuvarbase/ce.py

@@ -2,7 +2,11 @@
 Implementation of Graham et al. 2013's Conditional Entropy
 period finding algorithm
 """
+from __future__ import print_function, division
 
+from builtins import zip
+from builtins import range
+from builtins import object
 from .core import GPUAsyncProcess
 import numpy as np
 import pycuda.driver as cuda
@@ -47,7 +51,6 @@ def __init__(self, **kwargs):
         self.y_g = None
         self.dy_g = None
 
-        self.set_bins = False
         self.bins_g = None
         self.ce_c = None
         self.ce_g = None
@@ -122,12 +125,11 @@ def allocate_bins(self, **kwargs):
         assert(nf is not None)
 
         self.nbins = nf * self.phase_bins * self.mag_bins
-        
+
         if self.weighted:
             self.bins_g = gpuarray.zeros(self.nbins, dtype=self.real_type)
         else:
             self.bins_g = gpuarray.zeros(self.nbins, dtype=np.uint32)
-            self.set_bins=True
 
         if self.balanced_magbins:
             self.mag_bwf_g = gpuarray.zeros(self.mag_bins,
@@ -215,7 +217,6 @@ def balance_magbins(self, y, **kwargs):
 
         mag_bwf /= (max(y) - min(y))
 
-        # print self.balanced_magbins, ybins
         return ybins, mag_bwf.astype(self.real_type)
 
     def setdata(self, t, y, **kwargs):
@@ -272,11 +273,11 @@ def setdata(self, t, y, **kwargs):
         return self
 
     def set_gpu_arrays_to_zero(self, **kwargs):
-        for x in [self.t_g, self.y_g, self.dy_g]:
-            if x is not None:
-                x.fill(self.real_type(0), stream=self.stream)
+        self.t_g.fill(self.real_type(0), stream=self.stream)
+        self.y_g.fill(self.ytype(0), stream=self.stream)
         if self.weighted:
             self.bins_g.fill(self.real_type(0), stream=self.stream)
+            self.dy_g.fill(self.real_type(0), stream=self.stream)
         else:
             self.bins_g.fill(np.uint32(0), stream=self.stream)
 
@@ -318,14 +319,12 @@ def conditional_entropy(memory, functions, block_size=256,
             memory.transfer_ce_to_cpu()
         return memory.ce_c
 
-    # print memory.bins_g, memory.set_bins
     args = (grid, block, memory.stream)
     args += (memory.t_g.ptr, memory.y_g.ptr)
     args += (memory.bins_g.ptr, memory.freqs_g.ptr)
     args += (np.int32(memory.nf), np.int32(memory.n0))
     hist_count.prepared_async_call(*args)
 
-    # print memory.bins_g, memory.set_bins
     grid = (int(np.ceil(memory.nf / float(block_size))), 1)
     args = (grid, block, memory.stream)
     args += (memory.bins_g.ptr, np.int32(memory.nf), memory.ce_g.ptr)
@@ -340,9 +339,7 @@ def conditional_entropy(memory, functions, block_size=256,
         ce_std.prepared_async_call(*args)
 
     if transfer_to_host:
-        memory.stream.synchronize()
         memory.transfer_ce_to_cpu()
-        memory.stream.synchronize()
 
     return memory.ce_c
 
@@ -439,7 +436,7 @@ def _compile_and_prepare_functions(self, **kwargs):
             standard_ce=[np.intp, np.int32, np.intp],
             weighted_ce=[np.intp, np.int32, np.intp]
         )
-        for fname, dtype in self.dtypes.iteritems():
+        for fname, dtype in self.dtypes.items():
             func = self.module.get_function(fname)
             self.prepared_functions[fname] = func.prepare(dtype)
         self.function_tuple = tuple(self.prepared_functions[fname]
@@ -760,5 +757,5 @@ def batched_run_const_nfreq(self, data, batch_size=10,
 
             for i, (f, ce) in enumerate(results):
                 ces.append(np.copy(ce))
-        
+
         return [(freqs, ce) for ce in ces]

cuvarbase/core.py

@@ -1,3 +1,9 @@
+from __future__ import absolute_import
+from __future__ import division
+from __future__ import print_function
+
+from builtins import range
+from builtins import object
 import numpy as np
 from .utils import gaussian_window, tophat_window, get_autofreqs
 import pycuda.driver as cuda

cuvarbase/cunfft.py

@@ -1,5 +1,9 @@
 #!/usr/bin/env python
+from __future__ import absolute_import
+from __future__ import division
+from __future__ import print_function
 
+from builtins import object
 import sys
 import pycuda.driver as cuda
 import pycuda.gpuarray as gpuarray
@@ -88,11 +92,9 @@ def allocate_pinned_cpu(self, **kwargs):
         self.nf = kwargs.get('nf', self.nf)
 
         assert(self.nf is not None)
-        #self.ghat_c = cuda.aligned_zeros(shape=(self.nf,),
-        #                                 dtype=self.complex_type,
-        #                                 alignment=resource.getpagesize())
-        #self.ghat_c = cuda.register_host_memory(self.ghat_c)
-        self.ghat_c = np.zeros(self.nf, dtype=self.complex_type)
+        self.ghat_c = cuda.aligned_zeros(shape=(self.nf,),
+                                         dtype=self.complex_type,
+                                         alignment=resource.getpagesize())
         self.ghat_c = cuda.register_host_memory(self.ghat_c)
 
         return self
@@ -454,7 +456,7 @@ def _compile_and_prepare_functions(self, **kwargs):
                         self.real_type, self.real_type, self.real_type]
         )
 
-        for function, dtype in self.dtypes.iteritems():
+        for function, dtype in self.dtypes.items():
             func = self.module.get_function(function)
             self.prepared_functions[function] = func.prepare(dtype)