Installation with lsstsw and lsst-build
This page guides you through installing the LSST Science Pipelines from source with lsstsw and lsst-build. These are the same tools LSST Data Management uses to build and test the Science Pipelines.
Since lsstsw presents the Science Pipelines as a directory of Git repositories cloned from github.com/lsst, this installation method can be very convenient for developing Science Pipelines code. Other methods of installing LSST Science Pipelines software are :doc:`newinstall.sh <newinstall>` and :doc:`Docker <docker>`.
If you have issues using lsstsw, here are two ways to get help:
- Review the :ref:`known installation issues <installation-issues>`.
- Ask a question on the LSST Community support forum.
The LSST Science Pipelines can generally be compiled on CentOS, Debian, and macOS platforms. See :ref:`prereq-platforms` for information about LSST's official reference platform and build reports with other platforms.
Before you begin:
- Install prerequisites for your platform: :doc:`CentOS / RedHat <prereqs/centos>`, :doc:`Debian / Ubuntu <prereqs/debian>`, or :doc:`macOS <prereqs/macos>`.
- If you intend to use a Git LFS repository, like ci_hsc or afwdata, :doc:`install and configure Git LFS <git-lfs>`.
- If you opt not to use the default Python environment provided by
lsstswyou will need to :ref:`install these Python dependencies <python-deps>`.
2. Deploy lsstsw
Begin by choosing a working directory, then deploy
lsstsw into it:
git clone https://github.com/lsst/lsstsw.git cd lsstsw ./bin/deploy source bin/setup.sh
3. Build the Science Pipelines packages
From the :file:`lsstsw` directory, run:
rebuild step finishes, note the build number printed on screen.
It is formatted as "
Tag this build as
current so that EUPS can set it up by default:
eups tags --clone bNNNN current
Finally, set up the packages with EUPS:
See :doc:`setup` for more information.
You can do more with the :command:`build` command, including building from branches of GitHub repositories. For more information:
4. Testing Your installation (optional)
Once the LSST Science Pipelines are installed, you can verify that it works by :doc:`running a demo project <demo>`. This demo processes a small amount of SDSS data.
Sourcing the Pipelines in a new shell
In every new shell session you will need to set up the Science Pipelines environment and EUPS package stack.
Run these two steps:
lsst_distrib, you can set up a different top-level package like
lsst_appsor any individual EUPS package you previously installed. See :doc:`top-level-packages`.
Next steps and advanced topics
About the lsstsw deploy script
deploy script automates several things to prepare an LSST development environment:
- Installs Git.
- Installs Git LFS (you are still responsible for :doc:`configuring it <git-lfs>`).
- Installs a Miniconda Python 3 environment specific to this lsstsw workspace.
- Installs EUPS into :file:`eups/current/`.
- Clones lsst-build, which runs the build process.
- Clones versiondb, a robot-managed Git repository of package dependency information.
- Creates an empty stack installation directory, :file:`stack/`.
This environment, including the EUPS, Miniconda, Git, and Git LFS software, is only activated when you source the :file:`bin/setup.sh` or :file:`bin/setup.csh` scripts in a shell. Otherwise, lsstsw does not affect the software installed on your computer.
See also: :ref:`lsstsw-deploy-ref`.
About the lsstsw rebuild command
The :command:`rebuild` command accomplishes the following:
- Clones all Science Pipelines packages from github.com/lsst. The repos.yaml file in the https://github.com/lsst/repos repository maps package names to GitHub repositories.
- Runs the Scons-based build process to compile C++, make Pybind11 bindings, and ultimately create the :lmod:`lsst` Python package. The stack is built and installed into the :file:`stack/` directory inside your :file:`lsstsw/` work directory.
lsstsw clones repositories using HTTPS (see repos.yaml). Our guide to Setting up a Git credential helper will allow you to push new commits up to GitHub without repeatedly entering your GitHub credentials.
See also: :ref:`lsstsw-rebuild-ref`.
Building from branches
lsstsw's :command:`rebuild` command enables you to clone and build development branches.
lsst_distrb, but use the Git branch
my-feature when it's available in a package's repository:
rebuild -r my-feature lsst_distrib
Multiple ticket branches across multiple products can be built in order of priority:
rebuild -r feature-1 -r feature-2 lsst_distrib
In this example, a
feature-1 branch will be used in any package's Git repository.
feature-2 branch will be used secondarily in repositories where
feature-1 doesn't exist.
lsstsw falls back to using the
master branch for repositories that lack both
lsstsw deploy command reference
.. program:: deploy
usage: deploy.sh [-2|-3] [-b] [-h]
.. option:: -2 Install a Python 2-based Miniconda_ (not supported for ``v16_0`` of the LSST Science Pipelines and later)
.. option:: -3 Use a Python 3-based Miniconda_ (default).
.. option:: -b Use bleeding-edge conda packages.
.. option:: -h Print the help message.
lsstsw rebuild command reference
.. program:: rebuild
rebuild [-p] [-n] [-u] [-r <ref> [-r <ref2> [...]]] [-t <eupstag>] [product1 [product2 [...]]]
.. option:: -p Prep only.
.. option:: -n Do not run :command:`git fetch` in already-downloaded repositories.
.. option:: -u Update the :file:`repos.yaml` package index to the ``master`` branch on GitHub of https://github.com/lsst/repos.
.. option:: -r <git ref> Rebuild using the Git ref. A Git ref can be a branch name, tag, or commit SHA. Multiple ``-r`` arguments can be given, in order or priority.
.. option:: -t EUPS tag.