diff --git a/README.md b/README.md index 143098d2..1a17c2d3 100644 --- a/README.md +++ b/README.md @@ -32,8 +32,8 @@ If you wish to know more about the instrument please check the If you are having trouble operating the Goodman Pipeline we suggest the following procedure. -* Check [existing issues](https://github.com/soar-telescope/goodman/issues) or -open a [new Issue](https://github.com/soar-telescope/goodman/issues/new) on GitHub. +* Check [existing issues](https://github.com/soar-telescope/goodman_pipeline/issues) or +open a [new Issue](https://github.com/soar-telescope/goodman_pipeline/issues/new) on GitHub. ## Development Team diff --git a/docs/_install_dcr.rst b/docs/_install_dcr.rst new file mode 100644 index 00000000..c29d1e80 --- /dev/null +++ b/docs/_install_dcr.rst @@ -0,0 +1,93 @@ +.. _dcr: + +Installing DCR +************** + +.. admonition:: Acknowledgement Note + + Please cite: Pych, W., 2004, PASP, 116, 148 + +In terms of cosmic ray rejection we shifted to a non-python package because the +results were much better compared to LACosmic's implementation in Astropy. +LACosmic was not designed to work with spectroscopy. Though since version +:ref:`1.1.0 ` we shifted from Astropy to Astroscrappy's implementation +of LACosmic. + +The latest version of the Goodman Spectroscopic Pipeline uses a modified version +of ``dcr`` to help with the pipeline's workflow. It is included under + + ``/goodman_pipeline/goodman_pipeline/data/dcr-source/dcr/`` + +``goodman_pipeline-`` is the folder that will be created once you untar or unzip the latest +release of the |pipeline name|. + +.. important:: + + The changes we made to DCR include deletion of all ``HISTORY`` and ``COMMENT`` keywords, + which we don't use in the pipeline. And addition of a couple of custom + keywords, such as: ``GSP_FNAM``, which stores the name of the file being + created. ``GSP_DCRR`` which stores the reference to the paper to cite. + + +You are still encouraged to visit the official +`Link `_. We remind again that users of the +Goodman Pipeline should cite the DCR paper with the reference indicated above. + +Compiling DCR +^^^^^^^^^^^^^ + +Compiling ``dcr`` is actually very simple. + + ``cd /goodman_pipeline/goodman_pipeline/data/dcr-source/dcr/`` + +Then simply type: + + ``make`` + +This will compile `dcr` and also it will create other files. The executable +binary here is ``dcr``. + +We have successfully compiled *dcr* right out the box in several platforms, such as: + +- Ubuntu 16.04 +- Centos 7.1, 7.4 +- MacOS Sierra +- Solaris 11 + + +Installing the DCR binary +^^^^^^^^^^^^^^^^^^^^^^^^^ + +This is a suggested method. If you are not so sure what you are doing, we +recommend you follow the steps shown below. If you are a more advanced user and +you want to do it your own way, all you have to achieve is to have the ``dcr`` +executable binary in your ``$PATH`` variable. + +1. Open a terminal +2. In your home directory create a hidden directory ``.bin`` (Home directory + should be the default when you open a new terminal window) + + ``mkdir ~/.bin`` + +3. Move the binary of your choice and rename it ``dcr``. If you compiled it, + most likely it's already called ``dcr`` so you can ignore the renaming part of + this step. + + ``mv dcr.Ubuntu16.04 ~/.bin/dcr`` + + Or + + ``mv dcr ~/.bin/dcr`` + +4. Add your ``$HOME/.bin`` directory to your ``$PATH`` variable. Open the file + ``.bashrc`` and add the following line. + + ``export PATH=$PATH:/home/myusername/.bin`` + + Where ``/home/myusername`` is of course your home directory. + +5. Close and reopen the terminal or load the ``.bashrc`` file. + + ``source ~/.bashrc`` + +.. include:: _shortcuts.rst \ No newline at end of file diff --git a/docs/img/cvso166_400m2_gg455_2.png b/docs/_static/cvso166_400m2_gg455_2.png similarity index 100% rename from docs/img/cvso166_400m2_gg455_2.png rename to docs/_static/cvso166_400m2_gg455_2.png diff --git a/docs/img/cvso2016_Ha_auto_wl_solution2.png b/docs/_static/cvso2016_Ha_auto_wl_solution2.png similarity index 100% rename from docs/img/cvso2016_Ha_auto_wl_solution2.png rename to docs/_static/cvso2016_Ha_auto_wl_solution2.png diff --git a/docs/img/cvso2016_Ha_auto_wl_solution_1.png b/docs/_static/cvso2016_Ha_auto_wl_solution_1.png similarity index 100% rename from docs/img/cvso2016_Ha_auto_wl_solution_1.png rename to docs/_static/cvso2016_Ha_auto_wl_solution_1.png diff --git a/docs/img/cvso2016_Ha_auto_wl_solution_3.png b/docs/_static/cvso2016_Ha_auto_wl_solution_3.png similarity index 100% rename from docs/img/cvso2016_Ha_auto_wl_solution_3.png rename to docs/_static/cvso2016_Ha_auto_wl_solution_3.png diff --git a/docs/img/eg21_1200M5_GG455.png b/docs/_static/eg21_1200M5_GG455.png similarity index 100% rename from docs/img/eg21_1200M5_GG455.png rename to docs/_static/eg21_1200M5_GG455.png diff --git a/docs/img/lamp1D_example.png b/docs/_static/lamp1D_example.png similarity index 100% rename from docs/img/lamp1D_example.png rename to docs/_static/lamp1D_example.png diff --git a/docs/img/lst b/docs/_static/lst similarity index 100% rename from docs/img/lst rename to docs/_static/lst diff --git a/docs/img/realvnc.png b/docs/_static/realvnc.png similarity index 100% rename from docs/img/realvnc.png rename to docs/_static/realvnc.png diff --git a/docs/img/realvnc1.png b/docs/_static/realvnc1.png similarity index 100% rename from docs/img/realvnc1.png rename to docs/_static/realvnc1.png diff --git a/docs/img/realvnc_login.png b/docs/_static/realvnc_login.png similarity index 100% rename from docs/img/realvnc_login.png rename to docs/_static/realvnc_login.png diff --git a/docs/img/reminavnc.png b/docs/_static/reminavnc.png similarity index 100% rename from docs/img/reminavnc.png rename to docs/_static/reminavnc.png diff --git a/docs/img/screenshot_1.png b/docs/_static/screenshot_1.png similarity index 100% rename from docs/img/screenshot_1.png rename to docs/_static/screenshot_1.png diff --git a/docs/_static/soar_logo.png b/docs/_static/soar_logo.png new file mode 100644 index 00000000..a5fe2923 Binary files /dev/null and b/docs/_static/soar_logo.png differ diff --git a/docs/img/soardata3-terminal.png b/docs/_static/soardata3-terminal.png similarity index 100% rename from docs/img/soardata3-terminal.png rename to docs/_static/soardata3-terminal.png diff --git a/docs/img/std_dev_Ha_automated_solution_GSPEC_2018-03-22.png b/docs/_static/std_dev_Ha_automated_solution_GSPEC_2018-03-22.png similarity index 100% rename from docs/img/std_dev_Ha_automated_solution_GSPEC_2018-03-22.png rename to docs/_static/std_dev_Ha_automated_solution_GSPEC_2018-03-22.png diff --git a/docs/_working_with_virtualenv.rst b/docs/_working_with_virtualenv.rst index 0260151b..0961797e 100644 --- a/docs/_working_with_virtualenv.rst +++ b/docs/_working_with_virtualenv.rst @@ -11,16 +11,18 @@ If you know nothing about them we recommend you to start in the `Conda site `_ for information. +Here is a quick overview of the basic operations with virtual environments. + Discover what environments exist in your system. ``conda env list`` Will print a list where the first column is the name. -Activate (enter) the virtual Environment. +Activate the virtual Environment. ``source activate `` Where ```` is the name of your virtual environment. Your shell's @@ -29,7 +31,7 @@ Activate (enter) the virtual Environment. ``() [user@hostname folder-name]$`` -Deactivate (leave) the virtual environment. +Deactivate the virtual environment. ``source deactivate`` This time the prompt will change again to: diff --git a/docs/conf.py b/docs/conf.py index 6d95558b..7fd23cb0 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -111,6 +111,8 @@ html_theme = 'sphinx_rtd_theme' +html_logo = '_static/soar_logo.png' + html_context = {'license': 'BSD 3-Clause License'} # Theme options are theme-specific and customize the look and feel of a theme diff --git a/docs/install.rst b/docs/install.rst index 9b27b7ee..9694d9bc 100644 --- a/docs/install.rst +++ b/docs/install.rst @@ -2,200 +2,102 @@ Install ####### -Using the pipeline remotely is the recommended method, in which case you don't need -to worry about software requirements. -However, for users who wish to go ahead with a local installation, we provide -simple instructions in the current section. +We do not have the resources to provide installation support, thus we provide +a server with the latest and older versions installed that users with access rights +can use, however, the installation process is simple. -Requirements -************ - -The |pipeline name| is completely written in Python 3.x and relies on several -libraries like: - -* NumPy -* SciPy -* MatPlotLib -* Pandas -* AstroPy -* AstroPy/ccdproc -* AstroPy/astroplan -* DCR - -Using Conda -*********** - -We **do not** recommend the installation of these libraries or the -|pipeline name| in your system since updates and upgrades may ruin it. We rather -recommend the use of Virtual Environments. If you are not familiar with this -term, please check the official documentation by visiting the links below: - - https://docs.python.org/3/tutorial/venv.html - - or - - http://docs.python-guide.org/en/latest/dev/virtualenvs/ - -Another option is to install **Conda**, a Virtual Environment Manager, or -**AstroConda**, the same but for astronomers. Everything you need to know -about installing both can be found in the link below: - - https://astroconda.readthedocs.io/ - - -.. include:: _working_with_virtualenv.rst - -Using PIP -********* - -.. warning:: - - You may find that `ccdproc` and `astroplan` do not come with Astroconda. - They are not available on any Conda channel either. That means that you will - have to install them separately. You can do so by downloading the source files - and installing them by hand, or simply - `activate your Virtual Environment `_ and - then install these two packages using pip with - - ``pip install ccdproc astroplan`` - - - -Setup for local installation -**************************** -System installation is not recommended because it can mess things up specially in -Linux and Mac OS. Before you proceed, make sure that your system has all -the required libraries, as described in `Requirements`_. - -Once you have Python running and all the libraries installed either using -Conda/AstroConda or not, you may download the last version available in the -following address: +Installation Overview +********************* - https://github.com/soar-telescope/goodman/releases/latest +The required steps for a successful installation may vary, but here is a general +list. -Before continuing, make sure that your Virtual Environment is active if this is -the case. There are several ways of doing this but normally the command below -should work: +Install *astroconda*. + Visit `AstroConda site `_ and use + their instructions on how to install and basic setup. Make sure you add + the `astroconda channel `_ - ``$ source activate `` +Create a *virtual environment*. + This will depend on the method of installation but here is the official + documentation on `virtual environment management `_. -Where ```` is the name of your Virtual Environment (e.g. -astroconda). +Install all requirements. + This can be done along with the virtual environment creation, but it might be + necessary to manually install dependencies. -Now you can finally install the |pipeline name|. Download the file, decompress -it, and enter the directory created during the file decompression. Test the -installation by typing: +Run test code. + This is optional only possible if you are installing from source code. - ``$ python setup.py test`` +Install the goodman pipeline. + The most important step (of course). -If you have any errors, check the traceback. If you find difficulties carring -on at this poing, you may contact us by `opening a new issue `_ or using the e-mail -`goodman-pipeline@ctio.noao.edu`. +Getting the code +**************** -If no error messages start popping up in your screen, you are good to carry -on with the installation. +There are several ways to get the goodman pipeline code on your computer. - ``$ python setup.py install`` +1. Clone the repository: Cloning the `repository `_ will + get you the latest version of the code, this could be a very unstable version, + so is not recommended. Also is very likely that you will download things you don't need. -.. note:: +2. Get a release: A release is like a snapshot of a very special stage of development. + Though is not easy to guarantee that the code will not break a released version should + be stable in the sense that no experimental code should exist. Also, is a + special packaging of the necessary parts for operation. The releases are listed + `here `_. make sure + it has a green tag that says **Latest Release**. - This will install the pipeline in the currently active Python version. - If you have Virtual Environments, make sure that it is active. If not, - you can add the ``--user`` option to install only for your user and avoid - needing root access. +3. Using pip: Since version :ref:`1.1.2 ` the goodman pipeline can be installed using + pip: ``pip install goodman-pipeline`` however this does not install astroconda + neither creates the virtual environment. -.. _dcr: -Installing DCR -************** - -.. admonition:: Acknowledgement Note - - Please cite: Pych, W., 2004, PASP, 116, 148 - -In terms of cosmic ray rejection we shifted to a non-python package because the -results were much better compared to LACosmic's implementation in Astropy. -LACosmic was not designed to work with spectroscopy. Though since version -:ref:`1.1.0 ` we shifted from Astropy to Astroscrappy's implementation -of LACosmic. - -The latest version of the Goodman Spectroscopic Pipeline uses a modified version -of ``dcr`` to help with the pipeline's workflow. It is included under - - ``/goodman/pipeline/data/dcr-source/dcr/`` - -``goodman`` is the folder that will be created once you untar or unzip the latest -release of the |pipeline name|. - -.. important:: - - The changes we made to DCR include deletion of all ``HISTORY`` and ``COMMENT`` keywords, - which we don't use in the pipeline. And addition of a couple of custom - keywords, such as: ``GSP_FNAM``, which stores the name of the file being - created. ``GSP_DCRR`` which stores the reference to the paper to cite. - - -You are still encouraged to visit the official -`Link `_. We remind again that users of the -Goodman Pipeline should cite the DCR paper with the reference indicated above. - -Compiling DCR -^^^^^^^^^^^^^ - -Compiling ``dcr`` is actually very simple. - - ``cd /goodman/pipeline/data/dcr-source/dcr/`` - -Then simply type: - - ``make`` - -This will compile `dcr` and also it will create other files. The executable -binary here is ``dcr``. - -We have successfully compiled *dcr* right out the box in several platforms, such as: - -- Ubuntu 16.04 -- Centos 7.1, 7.4 -- MacOS Sierra -- Solaris 11 +Dependencies +************ +We have made an enormous effort to maintain dependencies under control, with +one exception, but don't worry that we have :ref:`detailed instruccions ` for +dealing with it. -Installing the DCR binary -^^^^^^^^^^^^^^^^^^^^^^^^^ +For the ordinary dependencies we make use of the great tools provided by astroconda. +With one step we can create a virtual environment with all the dependencies on it. +This does not work for the pip installation, but either by cloning the repository or +by downloading a release you get a file called `environment.yml` that can be used by +conda in the following way: -This is a suggested method. If you are not so sure what you are doing, we -recommend you follow the steps shown below. If you are a more advanced user and -you want to do it your own way, all you have to achieve is to have the ``dcr`` -executable binary in your ``$PATH`` variable. + ``conda env create -f environment.yml`` -1. Open a terminal -2. In your home directory create a hidden directory ``.bin`` (Home directory - should be the default when you open a new terminal window) +This will create a virtual environment called ``goodman_pipeline`` that you can start +using by running. - ``mkdir ~/.bin`` + ``source activate goodman_pipeline`` -3. Move the binary of your choice and rename it ``dcr``. If you compiled it, - most likely it's already called ``dcr`` so you can ignore the renaming part of - this step. +.. include:: _install_dcr.rst - ``mv dcr.Ubuntu16.04 ~/.bin/dcr`` +Running tests +************* - Or +*Test code is code to test the code*, yes, that's right, and the |pipeline full name| +comes with a lot of it. Ideally the 100% of the code should be covered, we are not +there yet but we are getting closer. Running the test code is not a requirement +but it helps a lot to identify problems. Go to the extraction folder +(where the file ``setup.py`` exist) and execute the following instruction: - ``mv dcr ~/.bin/dcr`` + ``python setup.py test`` -4. Add your ``$HOME/.bin`` directory to your ``$PATH`` variable. Open the file - ``.bashrc`` and add the following line. +This will detect the test code and execute it. Failure or success will be informed. - ``export PATH=$PATH:/home/myusername/.bin`` +Installation +************ - Where ``/home/myusername`` is of course your home directory. +If the previous steps ended successfully this should be easy. Go to the extraction +folder and execute the following instruction: -5. Close and reopen the terminal or load the ``.bashrc`` file. + ``python setup.py install`` - ``source ~/.bashrc`` +Alternatively, after the creation of the virtual environment you can install the pipeline +using pip. -.. include:: _shortcuts.rst \ No newline at end of file + ``pip install goodman-pipeline`` \ No newline at end of file diff --git a/docs/overview.rst b/docs/overview.rst index 5fdec44c..2a5657c0 100644 --- a/docs/overview.rst +++ b/docs/overview.rst @@ -12,7 +12,7 @@ spectra can be produced shortly after the shutter closes. The pipeline is primarily intended to be run on a data reduction dedicated computer though it is available for local installation. The |pipeline full name| project is hosted at GitHub at -`it's GitHub Repository `_. +`it's GitHub Repository `_. Instructions for running the software are provided in the :ref:`usage` section of this guide. How to access the the data reduction server is on diff --git a/goodman_pipeline/images/goodman_ccd.py b/goodman_pipeline/images/goodman_ccd.py index 8f6145c3..58a3262e 100755 --- a/goodman_pipeline/images/goodman_ccd.py +++ b/goodman_pipeline/images/goodman_ccd.py @@ -216,7 +216,7 @@ def __call__(self, args=None): "not found inside headers. This keyword contains what " "is the Goodman Wavelength Configuration that was used" " for this observation. Please add it manually (see " - "https://github.com/soar-telescope/goodman/blob/development/goodman_modes.md) " + "https://github.com/soar-telescope/goodman_pipeline/blob/development/goodman_modes.md) " "and run again. Leaving the program now.") sys.exit(1)