Update readme and readthedocs (#107)

* Updating the readme and readthedocs to include conda install.
* Adding new instructions to install using conda
* Adding a new readthedocs theme
* Adding details about MPI on NERSC

.gitignore

@@ -64,7 +64,7 @@ instance/
 .scrapy
 
 # Sphinx documentation
-docs/_build/
+doc/_build/
 
 # PyBuilder
 target/

README.md

@@ -1,3 +1,4 @@
+[![Conda Version](https://anaconda.org/conda-forge/tjpcov/badges/version.svg)](https://anaconda.org/conda-forge/tjpcov/)
 [![PyPI version](https://badge.fury.io/py/tjpcov.svg)](https://badge.fury.io/py/tjpcov)
 [![Coverage Status](https://coveralls.io/repos/github/LSSTDESC/TJPCov/badge.svg?branch=master)](https://coveralls.io/github/LSSTDESC/TJPCov?branch=master)
 [![Documentation Status](https://readthedocs.org/projects/tjpcov/badge/?version=latest)](https://tjpcov.readthedocs.io/en/latest/?badge=latest)
@@ -9,23 +10,26 @@ TJPCov is a general covariance calculator interface to be used within LSST DESC.
 
 ## Installation
 
-### Non-developer Installation
-To use TJPCov, we recommend installing with pip:
+### Quickstart
+The easiest and recommended way to install TJPCov is to install it via conda:
+```
+conda install -c conda-forge tjpcov
+```
 
- - `python -m pip install .` will install tjpcov and the minimal dependencies.
- - `python -m pip install .\[doc\]` will install tjpcov, the minimal
-     dependencies and the dependencies needed to build the documentation.
- - `python -m pip install .\[nmt\]` will install tjpcov, the minimal
-     dependencies and the dependencies needed to use NaMaster.
- - `python -m pip install .\[mpi4py\]` will install, the minimal
-     dependencies and the mpi4py library to use MPI parallelization. **Note** This will not work on NERSC; see [the NERSC docs](https://docs.nersc.gov/development/languages/python/parallel-python/#mpi4py-in-your-custom-conda-environment) for how to install there.
- - `python -m pip install .\[full\]` will install tjpcov and all dependencies
+Alternatively you may also install tjpcov via PyPi:
+```
+pip install tjpcov 
+```
+will install TJPCov with minimal dependencies, and
+```
+pip install 'tjpcov[full]'
+```
+will include all dependencies (for details, see Optional dependencies (PyPi only) section)
 
-Note that due to a bug in the NaMaster installation, one needs to make sure numpy is installed before trying to install NaMaster. If you are doing a fresh install, run `python -m pip install .` first, and then `python -m pip install .\[nmt\]`
+> **_NOTE:_** If you plan to use TJPCov with MPI on NERSC, extra work must be done to get MPI running in your conda environment.  See [the NERSC docs](https://docs.nersc.gov/development/languages/python/parallel-python/#mpi4py-in-your-custom-conda-environment) for how to install MPI there.
 
 ### Developer Installation
-
-If you want to contribute to TJPCov, run the following steps to set up your development environment.
+If you wish to contribute to TJPCov, follow the steps below to set up your development environment.
 
 1. Clone the repository
 2. Create the conda environment with `conda env create --file environment.yml`
@@ -34,6 +38,23 @@ If you want to contribute to TJPCov, run the following steps to set up your deve
 5. Run `pytest -vv tests/`
 
 
+### Optional dependencies (PyPi only)
+Because TJPCov relies on some packages that may not be necessary for every user, we have added different installation options to accommodate different use cases. For example, if a user has no plans to use MPI with TJPCov, they do not need `mpi4py`. Below we list the different installation options available on PyPi.
+
+ - `pip install tjpcov` will install tjpcov and the minimal dependencies.
+ - `pip install tjpcov'[doc]'` will install tjpcov, the minimal
+     dependencies and the dependencies needed to build the documentation.
+ - `pip install 'tjpcov[nmt]'` will install tjpcov, the minimal
+     dependencies and the dependencies needed to use NaMaster.
+ - `pip install 'tjpcov[mpi4py]'` will install, the minimal
+     dependencies and the mpi4py library to use MPI parallelization. **Does not work on NERSC** (see above)
+ - `pip install 'tjpcov[full]'` will install tjpcov and all dependencies
+
+
+#### Developer installation (PyPi only)
+If you are using PyPi to set up your development environment (we recommend using conda instead), due to a bug in the NaMaster installation, one needs to make sure numpy is installed before trying to install NaMaster. For a fresh install, run `python -m pip install .` first, and then `python -m pip install .\[nmt\]`
+
+
 ## Planning & development
 
 Ask @felipeaoli or @mattkwiecien for access to the repository and join the #desc-mcp-cov channel on the LSST DESC slack to contribute.
@@ -57,7 +78,7 @@ flake8 .
 Furthermore, we are following GitHub's recommendation of using [Semantic Versioning](https://semver.org/) in our releases.
 
 
-## Dependencies and versioning
+## Supported Python Versions
 TJPCov currently runs on python 3.8, but python 3.9, 3.10 and 3.11 are supported.
 
 TJPCov also has a few specific software versions hardcoded.  Please check the `pyproject.toml` file to see version requirements.

doc/README.md

@@ -0,0 +1,10 @@
+# Building the Docs
+
+We use sphinx to build the documentation.  To build, make sure you have the sphinx dependencies installed, then run:
+
+```
+sphinx-build -b html source/. _build/html
+```
+
+This will output the generated html into the `_build/html` directory which you can then open in your browser.
+

doc/requirements.txt

@@ -1,3 +1,4 @@
 sphinx
 sphinx-autoapi
 sphinx_rtd_theme
+sphinx_book_theme

doc/source/conf.py

@@ -92,9 +92,14 @@
 
 # The theme to use for HTML and HTML Help pages.  See the documentation for
 # a list of builtin themes.
-html_theme = "sphinx_rtd_theme"
+html_theme = "sphinx_book_theme"
+html_title = "TJPCov Documentation"
 
 # Add any paths that contain custom static files (such as style sheets) here,
 # relative to this directory. They are copied after the builtin static files,
 # so a file named "default.css" will overwrite the builtin "default.css".
 # html_static_path = ['_static']
+html_theme_options = {
+    "repository_url": "https://github.com/LSSTDESC/TJPCov",
+    "use_repository_button": True,
+}

doc/source/contributing.rst

@@ -0,0 +1,29 @@
+Contributing
+------------
+
+Planning & development
+^^^^^^^^^^^^^^^^^^^^^^
+
+Ask @felipeaoli or @mattkwiecien for access to the repository and join the #desc-mcp-cov channel on the LSST DESC slack to contribute.
+
+We have adopted the following style convention (which are enforced in each PR):
+
+- `Google-style docstrings <https://google.github.io/styleguide/pyguide.html>`_
+- `Black code style <https://github.com/psf/black>`_ (with 79 characters line-width)
+- PEP8 except for E203 (for better compatibility with black)
+
+For a general idea of TJPCov's scientific scope, see also the :ref:`Terms of Reference`.
+
+Code Quality Tools
+^^^^^^^^^^^^^^^^^^
+
+We use ``black`` and ``flake8`` configuration files so that code follows a unified coding style and remains PEP8 compliant.
+
+This means before submitting your PR you must run the following in the root directory::
+
+    black .
+    flake8 .
+
+Furthermore, we are following GitHub's recommendation of using Semantic Versioning <https://semver.org/> in our releases.
+
+

doc/source/dev_installation.rst

@@ -0,0 +1,25 @@
+Developer Installation
+----------------------
+
+Using conda (Recommended)
+^^^^^^^^^^^^^^^^^^^^^^^^^
+If you wish to contribute to TJPCov, follow the steps below to set up your development environment.
+
+1. Clone the repository
+2. Create the conda environment with ``conda env create --file environment.yml``
+3. Activate the environment with ``conda activate tjpcov``
+4. Run ``pip install -e .``
+5. Run ``pytest -vv tests/``
+
+Using pip
+^^^^^^^^^
+1. Clone the repository
+2. Run ``pip install -e .``
+3. Run ``pytest -vv tests/``
+
+.. warning::
+    NaMaster installation (pip only).  
+
+    If you are using PyPi to set up your development environment (we recommend using conda instead), due to a bug in the NaMaster installation, one needs to make sure numpy is installed before trying to install NaMaster. 
+
+    For a fresh install, run ``pip install .`` first, and then ``pip install .[nmt]``

doc/source/example.rst

@@ -1,5 +1,5 @@
 Example: Basic covariance computation
-=====================================
+-------------------------------------
 
 Users should use the :code:`tjpcov.covariance_calculator.CovarianceCalculator`
 class with a configuration file or dictionary input.
@@ -15,7 +15,7 @@ Extra sections (e.g. :code:`GaussianFsky`) are ignored.
 In order to generate the covariance there are two options:
 
 1. Using the :code:`run_tjpcov.py` script.
-------------------------------------------
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 .. code-block:: bash
 
@@ -27,7 +27,7 @@ covariance in the output directory defined in the configuration file (i.e.
 to use a different file name (the path will be the same).
 
 2. Interactively
-----------------
+^^^^^^^^^^^^^^^^
 
 If you just want to get the final covariance:
 

doc/source/index.rst

@@ -6,15 +6,38 @@
 Welcome to TJPCov's documentation!
 ==================================
 
-TJPCov is a general covariance calculator interface to be used within LSST DESC.
+Welcome to the documentation page for TJPCov! Here, you will find detailed instructions on how to install TJPCov, whether you're looking to use it for your projects or contribute to its development. Additionally, we provide examples using TJPCov to help users get started. 
+
+For those interested in contributing, we have included guidelines to help you understand how to join the TJPCov development team. You can also find the terms of reference which outlines the goals that drive the development of TJPCov. Lastly, the API documentation section can be used as a reference for TJPCov's interfaces.
 
 .. toctree::
    :maxdepth: 2
-   :caption: Contents:
+   :caption: Getting Started
+   :name: getting_started
 
    installation
+   Developer Installation <dev_installation.rst>
+
+.. toctree::
+   :maxdepth: 1
+   :caption: User Guide
+   :name: using_tjpcov
+
    example
-   autoapi/index
+   Contributing <contributing.rst>
+   Terms of Reference <terms_of_reference.rst>
+
+.. toctree::
+   :maxdepth: 1
+   :caption: License
+   :name: license
+
+   license
+
+.. toctree::
+   :maxdepth: 1
+   :caption: Reference
+   :name: reference
 
 
 Indices and tables

doc/source/installation.rst

@@ -1,13 +1,37 @@
-************
 Installation
-************
-TJPCov can be installed with `pip`. There are five different flavors of TJPCov at the moment:
- - :code:`python -m pip install .` will install tjpcov and the minimal dependencies.
- - :code:`python -m pip install .\[doc\]` will install tjpcov, the minimal dependencies and the dependencies needed to build the documentation.
- - :code:`python -m pip install .\[nmt\]` will install tjpcov, the minimal dependencies and the dependencies needed to use NaMaster.
- - :code:`python -m pip install .\[mpi4py\]` will install, the minimal dependencies and the mpi4py library to use MPI parallelization.
- - :code:`python -m pip install .\[full\]` will install tjpcov and all dependencies
-
-Note that due to a bug in the NaMaster installation, one needs to make sure
-numpy is installed before trying to install NaMaster. If you are doing a fresh
-install, run :code:`python -m pip install .` first, and then :code:`python -m pip install .\[nmt\]`
+------------
+
+Quickstart
+^^^^^^^^^^
+The easiest and recommended way to install TJPCov is to install it via conda::
+
+    conda install -c conda-forge tjpcov
+
+Alternatively, you may also install TJPCov via PyPi::
+
+    pip install tjpcov
+
+will install TJPCov with minimal dependencies, and::
+
+    pip install 'tjpcov[full]'
+
+will include all dependencies (for details, see Optional dependencies (PyPi only) section).
+
+.. note:: 
+    If you plan to use TJPCov with MPI on NERSC, extra work must be done to get MPI running in your conda environment.  See `the NERSC docs <https://docs.nersc.gov/development/languages/python/parallel-python/#mpi4py-in-your-custom-conda-environment>`_ for how to install MPI there.
+
+Supported Python Versions
+^^^^^^^^^^^^^^^^^^^^^^^^^
+TJPCov currently runs on python 3.8, but python 3.9, 3.10, and 3.11 are supported.
+
+TJPCov also has a few specific software versions hardcoded. Please check the ``pyproject.toml`` file to see version requirements.
+
+Optional dependencies (PyPi only)
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+Because TJPCov relies on some packages that may not be necessary for every user, we have added different installation options to accommodate different use cases. For example, if a user has no plans to use MPI with TJPCov, they do not need ``mpi4py``. Below we list the different installation options available on PyPi.
+
+- ``pip install tjpcov`` will install tjpcov and the minimal dependencies.
+- ``pip install tjpcov'[doc]'`` will install tjpcov, the minimal dependencies, and the dependencies needed to build the documentation.
+- ``pip install 'tjpcov[nmt]'`` will install tjpcov, the minimal dependencies, and the dependencies needed to use NaMaster.
+- ``pip install 'tjpcov[mpi4py]'`` will install the minimal dependencies and the mpi4py library to use MPI parallelization. **Does not work on NERSC** (see note above)
+- ``pip install 'tjpcov[full]'`` will install tjpcov and all dependencies.

doc/source/license.rst

@@ -0,0 +1,6 @@
+License
+-------
+
+TJPCov is free software made available under the MIT License. For details see the `LICENSE <https://github.com/LSSTDESC/TJPCov?tab=MIT-1-ov-file#>`_.
+
+If you make use of any of the ideas or software in this package in your own research, please cite them as “(LSST DESC, in preparation)” and provide a link to this repository: https://github.com/LSSTDESC/tjpcov. 

doc/source/terms_of_reference.rst

@@ -0,0 +1,49 @@
+.. _Terms of Reference:
+
+Terms of Reference
+------------------
+
+Background
+^^^^^^^^^^
+
+- This document lays out terms of reference for TJPCov developers and contributors in the wider MCP and LSST DESC context.
+
+Scope
+^^^^^
+
+- TJPCov provides a set of APIs for calculation of covariance (and cross-covariance) matrices (or other descriptions of measurement uncertainty as required) for all main canonical large scale structure cosmological probes in DESC:
+
+  - weak lensing shear
+  - galaxy clustering
+  - galaxy clusters clustering
+  - galaxy clusters number counts
+
+- It provides flexibility in how its functionality is accessed:
+
+  - it is called from TXPipe and APIs are designed to work with TXPipe
+  - it can be called as a standalone library
+  - it can be called from the command line through driver routines
+
+- Interface support is as follows:
+
+  - Public APIs are maintained in Python
+  - Documentation should be maintained in the main README, readthedocs, and the benchmarks folder.
+  - Command-line driver routines are maintained in the bin folder
+  - Examples will be maintained in a separate repository. [AS?]
+
+- In the basic model, covariance matrices are factorised as follows:
+
+  - Disconnected part of the Gaussian covariance matrix
+  - Connected part of the Gaussian covariance matrix arising from mode coupling
+  - Additional covariance due to super-survey modes
+
+- External libraries:
+
+  - TJPCov aims to implement basic functionality internally but supports an external back-end for versatility and cross-checking
+  - External back-ends can be linked as external libraries (on top of a thin wrapper inside TJPCov), but must be interfaced via pure python (i.e., the external back-end must implement a python wrapper as a precondition to be integrated into TJPCov)
+
+- Boundaries of TJPCov:
+
+  - TJPCov does not support the calculation of covariance matrices that are not covariant with the main large-scale structure probes (i.e., supernovae luminosity distance)
+  - TJPCov performs theoretical computations of the covariance matrix given survey properties and assumed cosmology. It does not perform calculations on the data, e.g., various bootstrap techniques
+  - In general, TJPCov does not support covariance arising from systematic effects although exceptions could be made on an as-needed basis (i.e., we provide a covariance matrix as observed by a perfect instrument with no galactic foregrounds, etc.)