Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DM-10636: Update installation docs for v14 (in progress) #54

Merged
merged 14 commits into from
Sep 15, 2017
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@
sys.setrecursionlimit(2000)

# master variable for current Stack release
CURRENT_RELEASE = '13.0'
CURRENT_RELEASE = '14.0'

# If extensions (or modules to document with autodoc) are in another directory,
# add these directories to sys.path here. If the directory is relative to the
Expand Down
57 changes: 42 additions & 15 deletions index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -5,30 +5,57 @@ The LSST Science Pipelines
The LSST Science Pipelines enable optical and near-infrared astronomy in the big data era.
We are building the Science Pipelines for the `Large Synoptic Survey Telescope (LSST) <http://lsst.org>`_, but our command line task and Python API can be extended for any optical or near-infrared dataset.

- Join us on the `LSST Community forum, community.lsst.org <http://community.lsst.org>`_.
- Fork our code on GitHub at https://github.com/lsst.
- Report issues in `JIRA <https://jira.lsstcorp.org/projects/DM/issues/>`_.
- API documentation is currently published with `Doxygen <http://doxygen.lsst.codes/stack/doxygen/x_masterDoxyDoc/>`_.
- DM Developer guidance is at https://developer.lsst.io.
- Learn more about LSST Data Management by visiting http://dm.lsst.org.
- Help us improve our documentation! This guide is on GitHub at `lsst/pipelines_lsst_io <https://github.com/lsst/pipelines_lsst_io>`_.
The latest release is |current-release|: :doc:`learn what's new <releases/index>`.

Latest release
==============
.. _part-installation:

The latest release is |current-release|: :doc:`learn what's new <releases/index>`.
Installation
============

Recommended installation path:

- :doc:`Installing with newinstall.sh <install/newinstall>`
- :doc:`install/setup`
- :doc:`install/top-level-packages`

Alternative distributions and installation methods:

- :doc:`install/docker`
- :doc:`Installing from source with lsstsw <install/lsstsw>`
- `CernVM FS <https://github.com/airnandez/lsst-cvmfs>`_ (contributed by IN2P3)

To install the LSST Simulation software, such as MAF, follow the `LSST Simulations documentation <https://confluence.lsstcorp.org/display/SIM/Catalogs+and+MAF>`_.

.. This toctree is hidden to let us curate the section above, but still add the install/ pages to the Sphinx toctree

.. toctree::
:hidden:
:caption: Installation

releases/index
install/index

.. _part-release-details:

.. toctree::
:maxdepth: 1
:caption: Getting Started
:name: part-getting-started
:caption: Release details

install/index
known-issues
releases/notes
known-issues
metrics/index

.. toctree::
:hidden:

releases/index

More info
=========

- Join us on the `LSST Community forum, community.lsst.org <http://community.lsst.org>`_.
- Fork our code on GitHub at https://github.com/lsst.
- Report issues in `JIRA <https://jira.lsstcorp.org/projects/DM/issues/>`_.
- API documentation is currently published with `Doxygen <http://doxygen.lsst.codes/stack/doxygen/x_masterDoxyDoc/>`_.
- DM Developer guidance is at https://developer.lsst.io.
- Learn more about LSST Data Management by visiting http://dm.lsst.org.
- Contribute to our documentation. This guide is on GitHub at `lsst/pipelines_lsst_io <https://github.com/lsst/pipelines_lsst_io>`_.
113 changes: 0 additions & 113 deletions install/conda.rst

This file was deleted.

26 changes: 13 additions & 13 deletions install/demo.rst
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
######################################################
Testing the Science Pipelines Installation with a Demo
Testing the Science Pipelines installation with a demo
######################################################

This demo will allow you to quickly test your LSST Science Pipelines installation, :doc:`regardless of your installation method <index>`.
Expand All @@ -10,15 +10,14 @@ This demo will allow you to quickly test your LSST Science Pipelines installatio
Remember to first load the LSST Science Pipelines into your shell's environment.
The method depends on how the Science Pipelines were installed:

- :ref:`Conda <conda-install-activate>`
- :ref:`newinstall.sh <install-from-source-loadlsst>`
- :doc:`newinstall.sh <setup>`
- :ref:`lsstsw <lsstsw-setup>`

2. Download the Demo Project
2. Download the demo project
============================

Choose a directory to run the demo in.
We'll call this directory :file:`demo_data`.
For example:

.. code-block:: bash

Expand All @@ -29,13 +28,13 @@ Then download the demo's data (if you aren't running the current stable release,

.. code-block:: bash

curl -L https://github.com/lsst/lsst_dm_stack_demo/archive/13.0.tar.gz | tar xvzf -
cd lsst_dm_stack_demo-13.0
curl -L https://github.com/lsst/lsst_dm_stack_demo/archive/14.0.tar.gz | tar xvzf -
cd lsst_dm_stack_demo-14.0

.. note::

The demo's version should match your installed software.
If you installed from source (with :doc:`lsstsw <lsstsw>`), clone the demo repository instead of downloading a release:
If you installed from source (with :doc:`lsstsw <lsstsw>`) or with a :ref`newer tag <newinstall-other-tags>`, clone the demo repository instead of downloading a release:

.. code-block:: bash

Expand All @@ -54,7 +53,7 @@ run camcol field filters

*Filters in parentheses are not processed if run with the* ``--small`` *option, see below*

3. Run the Demo
3. Run the demo
===============

Now setup the processing package and run the demo:
Expand All @@ -66,11 +65,12 @@ Now setup the processing package and run the demo:

For each input image the script performs the following operations:

* generate a subset of basic image characterization (e.g., determine photometric zero-point, detect sources, and measures positions, shapes, brightness with a variety of techniques),
* creates a :command:`./output` subdirectory containing subdirectories of configuration files, processing metadata, calibrated images, FITS tables of detected sources. These "raw" outputs are readable by other parts of the LSST pipeline, and
* generates a master comparison catalog in the working directory from the band-specific source catalogs in the ``output/sci-results/`` subdirectories.
- Generates a subset of basic image characterizations (such photometric zero-point estimation, source detection, and measurement).
- Creates a :file:`./output` directory containing subdirectories of configuration files, processing metadata, calibrated images, and FITS tables of detected sources.
These ``raw`` outputs are readable by other parts of the LSST pipeline.
- Generates a master comparison catalog in the working directory from the band-specific source catalogs in the ``output/sci-results/`` subdirectories.

4. Check the Demo Results
4. Check the demo results
=========================

The demo will take a minute or two to execute (depending upon your machine), and will generate a large number of status messages.
Expand Down
77 changes: 77 additions & 0 deletions install/docker.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,77 @@
.. _docker:

###################
Running with Docker
###################

LSST provides versioned Docker images containing the Science Pipelines software.
With Docker, you can quickly install download and run the LSST Science Pipelines on any platform without compiling from source.

If you have issues, reach out on the `LSST Community support forum <https://community.lsst.org/c/support>`_.

.. _docker-prereqs:

Prerequisites
=============

To download Docker images and run containers, you need Docker's software.
The `Docker Community Edition <https://store.docker.com>`_ is freely available for most platforms, including macOS, Linux, and Windows.

If you haven't used Docker before, you might want to learn more about Docker, images, and containers.
Docker's `Getting Started <https://docs.docker.com/get-started/>`_ documentation is a good resource.

.. _docker-quick-start:

Quick start
===========

This command downloads a weekly build of the LSST Science Pipelines, starts a container, and opens a prompt:

.. code-block:: bash

docker run -ti lsstsqre/centos:7-stack-lsst_distrib-w_2017_35

Then in the container's shell, load the LSST environment and set up a :doc:`top-level package <top-level-packages>` (``lsst_distrib`` in this case):

.. code-block:: bash

source /opt/lsst/software/stack/loadLSST.bash
setup lsst_distrib

This step is equivalent to the :doc:`setup instructions <setup>` for a :doc:`newinstall.sh <newinstall>`\ -based installation.
In fact, the images are internally based on :command:`newinstall.sh`.

Next, learn more with these topics:

- :ref:`docker-tags`

.. _docker-tags:

Finding images for different LSST Science Pipelines releases
============================================================

LSST Science Pipelines Docker images are published as `lsstsqre/centos`_ on Docker Hub.
These images are based on the CentOS base image.

Docker images are versioned with tags, allowing you to run any release of the LSST Science Pipelines software.
The schema of these tags is:

.. code-block:: text

<centos major version>-stack-<EUPS product>-<EUPS distrib tag>

For example:

.. code-block:: text

7-stack-lsst_distrib-w_2017_35

This tag corresponds to:

- CentOS 7 operating system.
- ``lsst_distrib`` :doc:`top-level package <top-level-packages>`.
- ``w_2017_35`` EUPS tag. See :ref:`newinstall-other-tags` for an overview of LSST's EUPS tag schema.

You can see what tags are available by browsing `lsstsqre/centos on Docker Hub <https://hub.docker.com/r/lsstsqre/centos/tags/>`_.

.. _`lsstsqre/centos`: https://hub.docker.com/r/lsstsqre/centos/
44 changes: 8 additions & 36 deletions install/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -2,43 +2,15 @@
Installing the LSST Science Pipelines
#####################################

.. This page acts as an index.html for the install/ path, but the currated contents are on the homepage.

.. toctree::
:hidden:
conda
:maxdepth: 1

prereqs/index
newinstall
setup
top-level-packages
docker
lsstsw
demo

We offer a few ways of installing the LSST Science Pipelines.
Choose an option below to get started.

.. note::

To install the LSST Simulation software, such as MAF, please follow the `LSST Simulations documentation <https://confluence.lsstcorp.org/display/SIM/Catalogs+and+MAF>`_ instead of these guides.

:doc:`Conda installation <conda>`
Install the Pipelines as an `Anaconda/Miniconda <https://www.continuum.io/why-anaconda>`__ binary package.
This is the easiest installation option since no source compilation is required.

:doc:`Source installation (newinstall.sh) <newinstall>`
``newinstall.sh`` allows you to build and install the LSST Science Pipelines from source.
You can use your existing Python 2.7, or opt to use a built-in `Miniconda <http://conda.pydata.org/docs/>`__.

`CernVM FS (external link) <https://github.com/airnandez/lsst-cvmfs>`__
CernVM FS is a virtual machine that makes it easy to run the LSST Science Pipelines without compiling code.
This distribution is supported by Fabio Hernandez of IN2P3.

`Docker Containers (external link) <https://hub.docker.com/r/lsstsqre/centos/>`__
These images are convenient for using the LSST Science Pipelines in distributed or cloud-based computing workflows, or for one-off installations without modifying your environment. You can get them from our `DockerHub repo <https://hub.docker.com/r/lsstsqre/centos/>`__

:doc:`lsstsw installation <lsstsw>`
``lsstsw`` (and ``lsst-build``) are the tools we use internally to build and test the LSST Science Pipelines.
Use this installation option if you're interested in developing the Science Pipelines since ``lsstsw`` presents the Pipelines as a directory of cloned repositories from `github.com/lsst <https://github.com/lsst>`__.

Next, :doc:`try out your Science Pipelines installation by running a demo <demo>`.

If you have difficulty installing LSST software:

- review the :ref:`known installation issues for your platform <installation-issues>`.
- reach out on the `Support forum at community.lsst.org <https://community.lsst.org/c/support>`_.
Loading