PyWI CTA - A CTA wrapper for PyWI

Copyright (c) 2016-2018 Jeremie DECOCK (www.jdhp.org)

• Web site: http://cta.pywi.org/
• Online documentation: http://cta.pywi.org/docs/
• Source code: https://github.com/jeremiedecock/pywi-cta
• Issue tracker: https://github.com/jeremiedecock/pywi-cta/issues
• PyWI-CTA on PyPI: https://pypi.org/project/pywicta/
• PyWI-CTA on Anaconda Cloud: https://anaconda.org/jdhp/pywicta

Description

PyWI-CTA is a ctapipe wrapper for PyWI.

Warning

This project is in beta stage.

Features

The PyWI and PyWI-CTA libraries contain:

• wavelet transform and wavelet filtering functions for image multiresolution analysis and filtering;
• additional filter to remove some image components (non-significant pixels clusters);
• a set of generic filtering performance estimators (MSE, angular precision, energy loss, contamination rate, ...), some relying on the scikit-image Python library (supplementary estimators can be easily added to meet particular needs);
• a graphical user interface to visualize the filtering process in the wavelet transformed space;
• an Evolution Strategies (ES) algorithm known in the mathematical optimization community for its good convergence rate on generic derivative-free continuous global optimization problems (Beyer, H. G. (2013) "The theory of evolution strategies", Springer Science & Business Media);
• additional tools to manage and monitor the parameter optimization.

Dependencies

PyWI-CTA has the following strict requirements:

• Python 3.5 or 3.6
• Numpy
• ctapipe 0.6.1
• pyhessio 2.0.1

PyWI-CTA also depends on other packages for optional features:

• Scipy
• Scikit-image
• Pillow (a.k.a. PIL) to read and write many image formats (PNG, JPEG, TIFF, ...)
• Astropy to provide Fits file format
• Pandas
• Matplotlib 1.5 or later to provide plotting functionality
• PyWI
• Cosmostat iSAP Sparce2D

However, note that these only need to be installed if those particular features are needed. pywicta will import even if these dependencies are not installed.

Installation

PyWI-CTA and its dependencies may be installed using the Anaconda or Miniconda package system. We recommend creating a conda virtual environment first, to isolate the installed version and dependencies from your master environment (this is optional).

The following command will set up a conda virtual environment, add the necessary package channels, and download PyWI-CTA and its dependencies. The file environment.yml can be found in this repository. Note this is beta stage software and is not yet stable enough for end-users (expect large API changes until the first stable 1.0 release).

conda env create -n pywi-cta -f environment.yml
source activate pywi-cta
pip install pywicta --no-deps

If you have already installed ctapipe following the official installation procedure, you can add PyWI-CTA to the cta virtual environment like this:

source activate cta
pip install pywicta --no-deps

Developers should follow the development install instructions found in the documentation.

Note

As ctapipe is not tested to work on Microsoft Windows systems, PyWI-CTA does not officially support these systems neither.

Note

The --no-deps flag is optional, but highly recommended otherwise pip will sometimes try to "help" you by upgrading PyWI-CTA dependencies like Numpy, which may not always be desired.

Cosmostat iSAP Sparce2D installation (optional)

1. Download http://www.cosmostat.org/wp-content/uploads/2014/12/ISAP_V3.1.tgz (see http://www.cosmostat.org/software/isap/)

2. Unzip this archive, go to the "sparse2d" directory and compile the sparse2d library. It should generate two executables named mr_transform and mr_filter:

   tar -xzvf ISAP_V3.1.tgz
   cd ISAP_V3.1/cxx
   tar -xzvf sparse2d_V1.1.tgz
   cd sparse2d
   compile the content of this directory

An automated compilation and installation script for Linux is available here (author: Tino Michael).

Getting started: tutorial Notebooks

PyWI-CTA provides some Jupyter notebooks that can be used as examples or tutorials: https://github.com/jeremiedecock/pywi-cta-notebooks. New users should check them in the following order:

• Tutorial #0: Check PyWI-CTA install
• Tutorial #1a: Load and plot a FITS image using PyWI-CTA
• Tutorial #1b: Load and plot a Simtel event using PyHESSIO
• Tutorial #1c: Load and plot a Simtel image using PyWI-CTA
• Tutorial #1d: Plot Hillas parameters using PyWI-CTA
• Tutorial #2a: Tailcut cleaning with PyWI-CTA
• Tutorial #2b: Plot Starlet planes with PyWI-CTA
• Tutorial #2c: Starlet cleaning with PyWI-CTA
• Tutorial #3a: Tailcut interactive notebook
• Tutorial #3b: Starlet cleaning interactive notebook
• Tutorial #3c: Wavelet Sparce2D MrTransform interactive notebook
• Tutoriel #4a: Optimize Tailcut thresholds with exhaustive search
• Tutoriel #4b: Optimize Starlet thresholds with exhaustive search
• Tutoriel #4c: Optimize Tailcut thresholds with Differential Evolution
• Tutoriel #4d: Optimize Starlet thresholds with Differential Evolution

Recommended JupyterLab extensions for these tutorials (type the following commands in a terminal within the right conda environment):

jupyter labextension install @jupyter-widgets/jupyterlab-manager
jupyter labextension install @jupyterlab/toc
jupyter labextension install @ijmbarr/jupyterlab_spellchecker

Console usage example

PyWI-CTA can also be used through console commands. The following is an usage example:

1. Get a simtel file (e.g. from there)

2. In your system terminal, from the directory that contains the sample image, type the following commands (where SIMTEL_FILE is the path to your simtel file):

   pywicta-starlet -f common_hard_filtering -t 13.,1.5 -L mask --camid LSTCam --max-images 1 --plot SIMTEL_FILE
   pywicta-mrtransform -f common_hard_filtering -t 13.,1.5 -L mask --camid LSTCam --max-images 1 --plot SIMTEL_FILE
   pywicta-mrfilter -K -k -C1 -m3 -n4 -s2,4.5,3.5,3 --kill-isolated-pixels --camid LSTCam --max-images 1 --plot SIMTEL_FILE

3. Type pywicta-starlet -h, pywicta-mrtransform -h or pywicta-mrfilter -h to display the list of available options and their documentation.

Analysis results

Notebooks containing the analysis results can be found there: https://gitlab.com/jdhp/pywi-cta-analysis

• LST analysis (essential plots)
• LST analysis (very detailed)

Bug reports

To search for bugs or report them, please use the PyWI Bug Tracker at:

https://github.com/jeremiedecock/pywi-cta/issues