Permalink
Fetching contributors…
Cannot retrieve contributors at this time
259 lines (156 sloc) 8.17 KB

Installation with lsstsw and lsst-build

This page guides you through installing the LSST Science Pipelines from source with lsstsw and lsst-build. These are the same tools LSST Data Management uses to build and test the Science Pipelines.

Since lsstsw presents the Science Pipelines as a directory of Git repositories cloned from github.com/lsst, this installation method can be very convenient for developing Science Pipelines code. Other methods of installing LSST Science Pipelines software are :doc:`newinstall.sh <newinstall>` and :doc:`Docker <docker>`.

If you have issues using lsstsw, here are two ways to get help:

1. Prerequisites

The LSST Science Pipelines can generally be compiled on CentOS, Debian, and macOS platforms. See :ref:`prereq-platforms` for information about LSST's official reference platform and build reports with other platforms.

Before you begin:

2. Deploy lsstsw

Begin by choosing a working directory, then deploy lsstsw into it:

git clone https://github.com/lsst/lsstsw.git
cd lsstsw
./bin/deploy
source bin/setup.sh

For more information about the :command:`deploy` command, see :ref:`lsstsw-about-deploy`.

3. Build the Science Pipelines packages

From the :file:`lsstsw` directory, run:

rebuild lsst_distrib

Once the rebuild step finishes, note the build number printed on screen. It is formatted as "bNNNN." Tag this build as current so that EUPS can set it up by default:

eups tags --clone bNNNN current

Finally, set up the packages with EUPS:

setup lsst_distrib

See :doc:`setup` for more information.

Note

You can do more with the :command:`build` command, including building from branches of GitHub repositories. For more information:

4. Testing Your installation (optional)

Once the LSST Science Pipelines are installed, you can verify that it works by :doc:`running a demo project <demo>`. This demo processes a small amount of SDSS data.

Sourcing the Pipelines in a new shell

In every new shell session you will need to set up the Science Pipelines environment and EUPS package stack.

Run these two steps:

  1. Activate the lsstsw software environment by sourcing the :file:`setup.sh` script in lsstsw's :file:`bin` directory:

    source bin/setup.sh

    If you are running in a :command:`csh` or :command:`tcsh`, run this set up script instead:

    source bin/setup.csh
  2. Set up a :doc:`top-level package <top-level-packages>`:

    setup lsst_distrib

    Instead of lsst_distrib, you can set up a different top-level package like lsst_apps or any individual EUPS package you previously installed. See :doc:`top-level-packages`.

Next steps and advanced topics

About the lsstsw deploy script

The deploy script automates several things to prepare an LSST development environment:

  1. Installs Git.
  2. Installs Git LFS (you are still responsible for :doc:`configuring it <git-lfs>`).
  3. Installs a Miniconda Python 3 environment specific to this lsstsw workspace.
  4. Installs EUPS into :file:`eups/current/`.
  5. Clones lsst-build, which runs the build process.
  6. Clones versiondb, a robot-managed Git repository of package dependency information.
  7. Creates an empty stack installation directory, :file:`stack/`.

This environment, including the EUPS, Miniconda, Git, and Git LFS software, is only activated when you source the :file:`bin/setup.sh` or :file:`bin/setup.csh` scripts in a shell. Otherwise, lsstsw does not affect the software installed on your computer.

See also: :ref:`lsstsw-deploy-ref`.

About the lsstsw rebuild command

The :command:`rebuild` command accomplishes the following:

  1. Clones all Science Pipelines packages from github.com/lsst. The repos.yaml file in the https://github.com/lsst/repos repository maps package names to GitHub repositories.
  2. Runs the Scons-based build process to compile C++, make Pybind11 bindings, and ultimately create the :lmod:`lsst` Python package. The stack is built and installed into the :file:`stack/` directory inside your :file:`lsstsw/` work directory.

lsstsw clones repositories using HTTPS (see repos.yaml). Our guide to Setting up a Git credential helper will allow you to push new commits up to GitHub without repeatedly entering your GitHub credentials.

See also: :ref:`lsstsw-rebuild-ref`.

Building from branches

lsstsw's :command:`rebuild` command enables you to clone and build development branches.

To build lsst_distrb, but use the Git branch my-feature when it's available in a package's repository:

rebuild -r my-feature lsst_distrib

Multiple ticket branches across multiple products can be built in order of priority:

rebuild -r feature-1 -r feature-2 lsst_distrib

In this example, a feature-1 branch will be used in any package's Git repository. A feature-2 branch will be used secondarily in repositories where feature-1 doesn't exist. Finally, lsstsw falls back to using the master branch for repositories that lack both feature-1 and feature-2.

lsstsw deploy command reference

.. program:: deploy

usage: deploy.sh [-2|-3] [-b] [-h]
.. option:: -2

   Install a Python 2-based Miniconda_ (not supported for ``v16_0`` of the LSST Science Pipelines and later)

.. option:: -3

   Use a Python 3-based Miniconda_ (default).

.. option:: -b

   Use bleeding-edge conda packages.

.. option:: -h

   Print the help message.

lsstsw rebuild command reference

.. program:: rebuild

rebuild [-p] [-n] [-u] [-r <ref> [-r <ref2> [...]]] [-t <eupstag>] [product1 [product2 [...]]]
.. option:: -p

   Prep only.

.. option:: -n

   Do not run :command:`git fetch` in already-downloaded repositories.

.. option:: -u

   Update the :file:`repos.yaml` package index to the ``master`` branch on GitHub of https://github.com/lsst/repos.

.. option:: -r <git ref>

   Rebuild using the Git ref.
   A Git ref can be a branch name, tag, or commit SHA.
   Multiple ``-r`` arguments can be given, in order or priority.

.. option:: -t

   EUPS tag.