Join GitHub today
GitHub is home to over 50 million developers working together to host and review code, manage projects, and build software together.Sign up
GitHub is where the world builds software
Millions of developers and companies build, ship, and maintain their software on GitHub — the largest and most advanced development platform in the world.
===================================================================================== MRS3D - MultiResolution in Spherical 3D Library and tools v1.0b 09/2013 ===================================================================================== MRS3D is a C++ package providing a library and a set of tools allowing the manipulation of Spherical Fourier-Bessel Coefficients and the implementation of MultiResolution algorithms adapted to 3D data in spherical coordinates. In particular, MRS3D implements a Spherical 3D Isotropic Wavelet Decomposition. As a part of MRS3D, this package also contains the FastDSBT C++ library wich implements a Discrete Spherical Bessel Transform based on a matrix formalism. MRS3D is distributed under the CeCILL License (see LICENSE.CeCILL) in the hope that it will be usefull but without any warranty. The following softwares are used in MRS3D: - HEALPix (http://healpix.jpl.nasa.gov/) distributed under the GNU GPL - 3DEX (https://github.com/ixkael/3DEX) developed by Boris Leistedt and released under the CeCiLL License. See the ArXiV paper http://arxiv.org/abs/1111.3591. Author: François Lanusse (email@example.com) Copyright CEA 2013 =================== Requirements =================== In order to be able to compile the MRS3D package, the following softwares are required: - cmake (http://cmake.org/cmake/resources/software.html) - HEALPix (http://sourceforge.net/projects/healpix/files/) - cfitsio (http://heasarc.gsfc.nasa.gov/fitsio/) - OpenMPI or other MPI implementation - gfortran and gcc The following softwares are optional: - Doxygen - IDL =================== Building =================== Before building the MRS3D package, it is necessary to build and install the HEALPix package (Fortran AND CXX versions). First, unpack the tar.gz archive: $ tar -xvzf MRS3D.tar.gz $ cd MRS3D Set the CFITSIO_DIR environment variable to the directory containing libcfitsio.a: $ export CFITSIO_DIR=/usr/lib If IDL is installed on your system, you can build the MRS3D IDL interface by setting the environment variable IDL_DIR to the IDL root directory: $ export IDL_DIR=/Applications/itt/idl By default, the binary tools, headers and libraries will be installed in the MRS3D directory. To install them elsewhere on your sytem (e.g. /usr/local), you can set the INSTALL_DIR environment variable: $ export INSTALL_DIR=/usr/local [ Optional ] Compile the package: $ mkdir build $ cd build $ cmake .. $ make all $ make install To build the documentation: $ make doc The html documentation will be available at doc/html/index.html. If LaTeX is installed on your system, you can also build the pdf documentation by going to the doc/latex subdirectory and typing make. =================== Running the code =================== The transformations implemented in MRS3D require the values of zeros of Bessel functions. Instead of behing calculated each time, these values are tabulated inside a binary FITS file. This file, referred to as qlnTable, must be provided as first argument of all MRS3D commands. A precomputed qlnTable is provided with MRS3D and contains the first 3000 zeros of Bessel functions up to the order 2000. This file should be sufficient for most applications. To compute a larger qlnTable, the gen_qln tool is provided. The following command will generate a table containing the first 2000 zeros of Bessel functions up to order 4000. $ gen_qln qlnTable.fits 4000 2000 MRS3D provides two main binary tools almn2wavelet and waveletThresholding: > almn2wavelet : Computes an Isotropic Undecimated Spherical 3D Wavelet decomposition from an almn .fits file. > waveletThresholding : Applies wavelet hard thresholding to the input almn .fits file. These tools work on Spherical Fourier-Bessel coefficients stored in "almn" FITS files. In order to compute almn coefficients to test these functions, two additional tools are provided field2almn and almn2field: > field2almn : Computes the Discrete Spherical Fourier-Bessel Transform from a 3D cartesian field .fits file. > almn2field : Reconstructs a cartesian 3D field from an almn .fits file. As an example, to decompose a field into wavelet scales from the MRS3D root directory run: $ bin/field2almn test/qlnTable.fits test/VirgoField256.fits test/VirgoAlmn.fits 512 250.0 256 256 256 Will store in "VirgoAlmn.fits" coefficients computed from the field "VirgoField256.fits" using a HEALPix resolution of nside=512, a boundary condition rmax=250.0 Mpc/h, up to the orders lmax=256, mmax=256 and nmax=256. $ bin/almn2wavelet test/qlnTable.fits test/VirgoAlmn.fits wavelet 512 4 3.5 This command will produce 5 FITS files (wavelet_0.fits wavelet_1.fits...), one for each wavelet scale, plus one for the smoothed density. The cutoff frequency of the low pass filter is set to 3.5 $ bin/almn2field test/qlnTable.fits test/wavelet_0.fits test/waveletField_0.fits 256 479.0 512 Reconstructs the density field of the first wavelet scale inside a cube of size 256x256x256 and physical size 479.0 Mpc/h using a Healpix resolution of nside=512. Now, to visualize the density fields, the IDL interface can be used, see the next section. =================== IDL Interface =================== To work on spherical Fourier-Bessel coefficients and 3D fields from IDL, MRS3D comes with an IDL interface. The IDL procedures are in src/IDL. In order to work these procedures load libMRS3DIDL and by default expect the library to be in the same directory. To setup the IDL procedures, go to src/IDL and create a symlink to the libMRS3DIDL library: $ ln -s ../../lib/libmrs3d_IDL.dylib ./libmrs3d_IDL.dylib To setup the FastDSBT IDL interface use as well: $ ln -s ../../lib/libfastDSBT_IDL.dylib ./libfastDSBT_IDL.dylib (under linux, the libraries in /lib will be .so and not .dylib but keep the name in .dylib for the symlinks) In the /src/IDL directory, launch IDL. Available features: - Compute power spectrum from almn - Save/Load power spectrum to/from FITS files - Save/Load almn to/from FITS files - Save/Load cartesian density fiels to/from FITS files - Convert almn to field and field to almn (The use of the binary tools is prefered) Example of how to visualize the wavelet scale computed at the previous section: IDL> fieldWavelet = mrs3d_load_field('/path-to-MRS3D/test/waveletField_0.fits') IDL> ivolume, fieldWavelet To get help on the mrs3d_* commands, set the help keyword (e.g. mrs3d_save_field,/help) =================== Parallel execution =================== MRS3D is designed to take full advantage of OpenMP and MPI. All the tools provided with MRS3D are parallelized using OpenMP and will by default run using a thread by processor core. To manually set the number of threads used by the command set the environment variable OMP_NUM_THREADS: export OMP_NUM_THREADS=4 (bash) or setenv OMP_NUM_THREADS 4 (tcsh) To launch a command using several MPI processes: mpirun -np 4 [command followed by arguments] !!!!!!!!!!!!!!!!!!!!!!!!! WARNING !!!!!!!!!!!!!!!!!!!!! When using MPI, be sure to use powers of 2 for the number of l,m and n coefficients. Since l goes from 0 to lmax, you must set l=2^i -1 !!!! The same goes for m. Since n varies from 1 to nmax you can directly set nmax= 2^l A valid set of (lmax,mmax,nmax) is : 1023 1023 512 !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! An example of a PBS script to run a command on a computer cluster is provided below: -----------------------------------waveletThresholding.job---------------------------------- #!/bin/sh #PBS -l pmem=15G #PBS -l pvmem=16G #PBS -l mem=120G #PBS -l vmem=128G #PBS -l nodes=8:ppn=1 #PBS -W x=NACCESSPOLICY:SINGLEJOB #PBS -l walltime=100:00:00 #PBS -M [your email address] #PBS -m abe export OMP_NUM_THREADS=8 cd [path to working directory] cat $PBS_NODEFILE | sort | uniq > machinefile.txt echo "----Begining job----" > log date >> log echo "--------------------" >> log mpiexec -machinefile machinefile.txt -np 8 ./waveletThresholding qlnTable_2000_3000.fits Almn.fits ThresholdedAlmn.fits 2048 250.0 5 5 9.0 0.75 >> log echo "----Job done--------" >> log date >> log ---------------------------------------------------------------------------------------------- To launch the job : qsub waveletThresholding.job