Merge pull request #161 from whole-tale/v1.1-fixes

V1.1 fixes

users_guide/customizing.rst

@@ -0,0 +1,139 @@
+
+.. _customizing: 
+
+Customizing the Environment
+===========================
+
+Whole Tale requires you to declare any software dependencies in a set
+of simple text files that are used to build a Docker image that is 
+used to run your code.
+
+The text file formats are based on formats supported by 
+`repo2docker <https://repo2docker.readthedocs.io/>`_ and, where possible, 
+follow package installation conventions of each language
+(e.g., requirements.txt for Python, DESCRIPTION for R). In other cases,
+simple formats are defined (e.g., apt.txt, toolboxes.txt).
+
+Seel also:
+
+* `Configuration Files <https://repo2docker.readthedocs.io/en/latest/config_files.html>`_ (repo2docker)
+
+
+Base environment
+----------------
+The base environment is always an Ubuntu "Long Term Support" (LTS) version, often
+the latest release. 
+
+The :code:`apt.txt` file contains a list of packages that can be installed via :code:`apt-get`.
+Entries may have an optional version (e.g., for use with  :code:`apt-get install <package name>=<version>`)
+
+For example:
+
+.. code:: bash
+   
+   libblas-dev=3.7.1-4ubuntu1
+   liblapack-dev=3.7.1-4ubuntu1
+
+
+See also:
+
+* `Install packages with apt-get <https://repo2docker.readthedocs.io/en/latest/config_files.html#apt-txt-install-packages-with-apt-get>`_ (repo2docker)
+
+
+MATLAB
+------
+MATLAB support is part of the Whole Tale repo2docker extension and not available in all
+repo2docker configurations. 
+
+MATLAB toolboxes must be declared in a :code:`toolboxes.txt` file. Each line contains a valid
+MATLAB product for the selected version.
+
+For a complete list of available packages for each supported version, see https://github.com/whole-tale/matlab-install/blob/main/products/.
+
+For example the following :code:`toolboxes.txt` wold install the Financial and
+Statistics and Machine Learning toolboxes.
+
+.. code:: bash
+
+   product.Financial_Toolbox
+   product.Statistics_and_Machine_Learning_Toolbox
+
+See also:
+
+* `MATLAB example <https://github.com/whole-tale/matlab-example>`_
+
+
+R/RStudio
+---------
+R packages may be specified in a :code:`DESCRIPTION` file or `install.R`. 
+
+For `install.R`, each line is an `install.packages()` statement for a given package:
+
+.. code:: bash
+
+   install.packages("ggplot2")
+   install.packages("reshape2")
+   install.packages("lmtest")
+
+
+To configure a specific version, we recommend configuring an MRAN date using the :code:`runtime.txt`
+file:
+
+.. code:: bash
+ 
+   r-2020-10-20
+
+
+This file contains the MRAN date containing the versions of packages specified in :code:`install.R`.
+
+Alternatively, you ca use the :code:`install_version` function in place of `install.packages` in 
+your :code:`install.R` file. 
+
+.. code:: bash
+
+   require(devtools)
+   install_version("ggplot2", version = "0.9.1")
+
+
+For more information see:
+
+* `RStudio example <https://github.com/whole-tale/rstudio-example>`_
+* `Install an R/RStudio environment <https://repo2docker.readthedocs.io/en/latest/config_files.html#install-r-install-an-r-rstudio-environment>`_ (repo2docker)
+* `Install an R package <https://repo2docker.readthedocs.io/en/latest/config_files.html#description-install-an-r-package>`_ (repo2docker)
+* `Specifying runtimes <https://repo2docker.readthedocs.io/en/latest/config_files.html#runtime-txt-specifying-runtimes>`_ (repo2docker)
+
+Python
+------
+
+Python packages can be specified using :code:`requirements.txt`, :code:`Pipfile`/:code:`Pipfile.lock`, or 
+Conda :code:`environment.yaml`.
+
+Example :code:`requirements.txt`: 
+
+.. code:: bash
+
+   bokeh==1.4.0
+   pandas==1.2.4
+   xlrda==2.0.1
+
+
+See also:
+
+* `Mapping Estimated Water Usage  <http://doi.org/10.5281/zenodo.4829933>`_ (Example tale)
+* `requirements.txt: Install a Python environment<https://repo2docker.readthedocs.io/en/latest/config_files.html#requirements-txt-install-a-python-environment>`_ (repo2docker)
+* `Pipfile and/or Pipfile.lock: Install a Python environment <https://repo2docker.readthedocs.io/en/latest/config_files.html#pipfile-and-or-pipfile-lock-install-a-python-environment>`_ (repo2docker)
+* `environment.yml: Install a Conda enviroment <https://repo2docker.readthedocs.io/en/latest/config_files.html#environment-yml-install-a-conda-environment>`_ (repo2docker)
+
+
+Other
+-----
+
+Non-standard packages can be installed (or arbitrary commands run) using a :code:`postBuild` script. 
+The :code:`start` script can be used to run arbitrary code before th user session starts.
+
+See also:
+
+* `postBuild: Run code after installing the environment  <https://repo2docker.readthedocs.io/en/latest/config_files.html#postbuild-run-code-after-installing-the-environment>`_ (repo2docker)
+* `start: Run code before the user session starts <https://repo2docker.readthedocs.io/en/latest/config_files.html#start-run-code-before-the-user-sessions-starts`_ (repo2docker)
+
+

users_guide/environments/index.rst

@@ -11,6 +11,8 @@ environments including Jupyter, RStudio, MATLAB, STATA and OpenRefine. You can c
 these environments by adding your own packages based on 
 `repo2docker <https://repo2docker.readthedocs.io/>`_ compatible configuration files.
 
+Read more about :ref:`Customizing the environment <customizing>`.
+
 .. toctree::
    :maxdepth: 1
    :caption: Contents:
@@ -21,6 +23,3 @@ these environments by adding your own packages based on
    matlab
    stata
    openrefine
-   customizing
-
-

users_guide/environments/matlab.rst

@@ -41,6 +41,12 @@ to the native MATLAB Linux desktop.
      :align: center
      :scale: 80%
 
+Customizing Your MATLAB Environment
+-----------------------------------
+In Whole Tale, users must declare their dependencies in a set of simple text
+files that are used to build a Docker image. For more information, see 
+:ref:`Customizing the Environment <customizing>`.
+
 Technical Details
 -----------------
 

users_guide/index.rst

@@ -17,6 +17,7 @@ This guide provides detailed information about the Whole Tale system for new use
    compose
    run
    environments/index
+   customizing
    manage
    sharing
    versioning

users_guide/manage.rst

@@ -1,51 +1,40 @@
 .. _manage:
 
-Managing Data
-==============
+File Management
+===============
 
-The **Manage** page allows you manage data in the Whole Tale system including:
+For each Tale, the **Files** tab allows you to manage any files or data
+in the Whole Tale system including:
 
 - Uploading data from your local machine
 - Registering data from remote resources
-- Sharing data with other users
 
-.. _home-folder:
-
-Home Folder
------------
 
-The **Home** folder is your personal workspace in the Whole Tale system. You
-can perform most common operations including uploading, creating, 
-moving, renaming, and deleting files and folders.  
+Tale Workspace
+--------------
 
-Your **Home** folder is mounted into every running tale instance with full
-read-write permissions. This means that you can access and manage files from
-both the Whole Tale dashboard and within tale environments. This is in contrast
-to the **Data** folder described below, which is limited to read-only access.
+The **Tale Workspace** folder is the primary storage area (or folder) for
+all files associated with a Tale. 
 
-Adding Data
-^^^^^^^^^^^
-Files on your computer can be uploaded to the **Home** folder by using the 
-file selector dialog. To open the file selector, 
+Common operations include:
+* Upload files from your local machine
+* Rename, move, remove, copy, download files
+* Create, rename, move, remove, copy, download folders
+* Copy or move files between Tale workspaces
 
-    1. Click on the **Home** directory
-    2. Mouse over the blue addition symbol
-    3. Select the **Upload** icon
+The **Tale Workspace** folder is also available in your running Tale environment 
+as :code:`workspace`.
 
-.. figure:: images/manage/home_upload.png
-     :align: center
-     :scale: 80%
-
-     An empty *Home* directory.
-
-.. _data-folder:
 
-Data Folder
------------
+External Data
+-------------
 
-The **Data** folder contains references to data you've registered in the system
+The **External Data** folder contains references to data you've registered in the system
 for use with tales. This data is meant to be read only, and can only be added
-from external sources.
+from external sources. With this folder you can:
+
+* Register data from external sources (e.g., via DOI or URL)
+* Select and add registered data to a Tale
 
 Supported Data Repositories
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^
@@ -64,31 +53,32 @@ The current supported repositories that you can register data from are
   A service geared towards researchers and computing managers that allows
   custom APIs, data publication, and file transfer.
 
+  4. Zenodo_:
+  A general purpose research data repository.
+
 
 Adding Data
 ^^^^^^^^^^^
-Files and folders cannot be uploaded to the **Data** folder directly. To
+Files and folders cannot be uploaded to the **External Data** folder directly. To
 encourage reproducibility, only data registered from external resources or
-associated with a tale will be available in the **Data** folder.
+associated with a tale will be available in the **External Data** folder.
 
 To register data from an external resource, use the data registration dialog,
 shown below.
 
 .. figure:: images/manage/data_registration_modal.png
      :align: center
-     :scale: 80%
 
      The data registration dialog allows you to search by DOI and ingest data
      into Whole Tale.
 
-To access this dialog, navigate to the **Data** folder by clicking the link icon
+To access this dialog, navigate to the **External Data** folder by clicking the link icon
 below the home directory folder.
 
 .. figure:: images/manage/data_upload.png
      :align: center
-     :scale: 80%
 
-     A user's *Data* folder, populated with data that was registered from
+     A user's *External Data* folder, populated with data that was registered from
      external sources.
 
 The blue addition icon will open the registration dialog where you can find 
@@ -111,7 +101,6 @@ Click "Register" to begin data registration.
 
 .. figure:: images/manage/dataone_doi.png
      :align: center
-     :scale: 80%
 
      A dataset that was found by searching for the DOI.
 
@@ -124,7 +113,6 @@ will give the same data package which can subsequently be registered.
 
 .. figure:: images/manage/dataone_url.png
      :align: center
-     :scale: 80%
 
      A dataset that was found by searching with the package's DataONE
      url.
@@ -194,22 +182,43 @@ Selecting a folder or file will present a menu with the following options:
 - Download: download a file or folder
 - Remove: remove a file or folder
 
-.. _environments:
-
-Environments
-------------
-
-The **Environments** panel lists all available environments and allows you 
-to view detailed information by clicking on the blue info icon next to its
-name. Each Environment comes bundles with a programming language and 
-pre-installed packages . To learn more, visit the
-page on `Environments <environments.html>`__.
-
-
 .. _DataONE: https://www.dataone.org/ 
 .. _information page: https://www.dataone.org/current-member-nodes
 .. _Dataverse: https://dataverse.org/
 .. _Globus: https://www.globus.org/
+.. _Zenodo: https://www.zenodo.org/
 .. _package: https://search.dataone.org/#view/doi:10.18739/A29G5GD0V
 .. _Materials Data Facility: https://www.materialsdatafacility.org/
 .. _Twin-mediated Crystal Growth an Enigma Resolved: https://publish.globus.org/jspui/handle/ITEM/
+
+
+.. _home-folder:
+
+Home Folder
+-----------
+
+The **Home** folder is your personal workspace in the Whole Tale system. You
+can perform most common operations including uploading, creating, 
+moving, renaming, and deleting files and folders.  
+
+Your **Home** folder is mounted into every running tale instance with full
+read-write permissions. This means that you can access and manage files from
+both the Whole Tale dashboard and within tale environments. This is in contrast
+to the **Data** folder described below, which is limited to read-only access.
+
+Adding Data
+^^^^^^^^^^^
+Files on your computer can be uploaded to the **Home** folder by using the 
+file selector dialog. To open the file selector, 
+
+    1. Click on the **Home** directory
+    2. Mouse over the blue addition symbol
+    3. Select the **Upload** icon
+
+.. figure:: images/manage/home_upload.png
+     :align: center
+
+     An empty *Home* directory.
+
+.. _data-folder:
+