Merge branch 'hotfix/1.15.2'

.gitignore

@@ -1,3 +1,4 @@
 docs/_build
+docs/dictionary.rst
 *.DS_Store
 *.pyc

docs/advanced-topics/metrics.rst

@@ -1,7 +1,11 @@
 Workflow Metrics
 ================
 
-What are workflow metrics?
+.. contents::
+   :local:
+   :depth: 1
+
+What Are Workflow Metrics?
 ------------------------------------
 Workflow metrics are metrics of workflow executions on a platform. Metrics include things like the resources used during the execution, and how often workflow executions succeeded or failed.
 
@@ -12,38 +16,62 @@ Users are able to execute workflows on various platforms using Dockstore's Launc
 
 Platforms are able to submit metrics of workflows executed on their platform to Dockstore and we aggregate the metrics and display them in the UI to the users in the :ref:`Metrics tab <How to view metrics>`.
 
-.. _How to view metrics:
+Why Would a Platform Owner Want to Submit Workflow Metrics?
+-----------------------------------------------------------
 
-How do I view workflow metrics?
--------------------------------
+As a platform owner, workflow metrics indicate to others that your platform is compatible with many workflows on Dockstore. Workflow metrics provide valuable information to users, including information about the resources and time needed to run the workflow. It helps the user determine if the workflow is high quality and likely to function correctly.
 
-To search for workflows with metrics, navigate to the `search <https://dockstore.org/search>`_ page and select a platform for the Execution Metrics and/or Validation Metrics facets.
+Platform owners can submit metrics for workflow executions, retrieve submitted executions, and update submitted executions. 
 
-.. figure:: /assets/images/docs/metrics/metrics-search-facets.png
-    :alt: Metrics search facets
+.. _Getting started with submitting metrics:
 
-Select a workflow and click on the Versions tab. Versions that have metrics have a check mark in the Metrics column.
+Getting Started With Submitting Metrics to Dockstore
+----------------------------------------------------
 
-.. figure:: /assets/images/docs/metrics/versions-metrics-column.png
-    :alt: Metrics column in Versions table
+The following sections go over how a platform can submit workflow metrics to Dockstore. 
 
-Select a version with metrics then click on the Metrics tab to view the metrics available.
+You will need to:
 
-.. figure:: /assets/images/docs/metrics/metrics-tab.png
-    :alt: Metrics tab
+.. contents::
+   :local:
+   :depth: 1
 
-Why would I, a platform owner, want to submit workflow metrics?
----------------------------------------------------------------
+Dockstore aggregates the workflow metrics you submit and displays them in the :ref:`Dockstore UI<How to view metrics>`.
 
-As a platform owner, workflow metrics indicate to others that your platform is compatible with many workflows on Dockstore. Workflow metrics provide valuable information to users, including information about the resources and time needed to run the workflow. It helps the user determine if the workflow is high quality and likely to function correctly.
+Record workflow execution metrics on your platform
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+The first step is to collect metrics for executions of public Dockstore workflows that occur on your platform. The mandatory metrics that Dockstore requires are the date of execution and the execution status. 
+If they are available, you are encouraged to collect additional metrics such as how long it took the workflow to run, CPU requirements, memory requirements, etc.
+
+Format the workflow metrics for Dockstore
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+The metrics that you collect may not be in the format that the Dockstore API expects. You have to format them so they follow Dockstore's schema, which you can view in the :ref:`Submit Workflow Metrics to Dockstore<Submit executions>` section.
+
+For example, execution dates are expected to be in ISO 8601 UTC date format, and there are a defined set of execution statuses that can be submitted.
 
-Platform owners are able to submit metrics for workflow executions, retrieve submitted executions, and update submitted executions.
+.. note::
+   If you are unable to format the workflow metrics to match Dockstore's schema or you would like to try submitting metrics without continuing with the following steps, 
+   contact us via our `GitHub <https://github.com/dockstore/dockstore/issues>`_ issues or open a :ref:`helpdesk ticket<faq-contact-admins>` on `Discourse <https://discuss.dockstore.org/>`_ and we can help get your metrics into Dockstore.
+
+   This may involve you providing us with a file of metrics, such as a CSV file, and Dockstore will process the file and ingest the metrics into our system.
 
-.. note:: Submitting metrics for workflow executions, and retrieving and updating submitted executions are only available for admins, curators, and platform partners. If you're a platform owner and you don't have a platform partner user, please contact us via our `GitHub <https://github.com/dockstore/dockstore/issues>`_ issues or open a helpdesk ticket on `Discourse <https://discuss.dockstore.org/>`_ and we will help you get set up.
+Register a platform partner user on Dockstore
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
+Only Dockstore admins, curators, and platform partners can :ref:`submit metrics<Submit executions>` for workflow executions, and :ref:`retrieve and update submitted executions<Viewing And Updating Submitted Workflow Metrics>`.
+If you're a platform owner and you don't have a platform partner user on Dockstore, please:
 
-How do I submit workflow metrics?
----------------------------------
+1. :doc:`Create a Dockstore account </getting-started/register-on-dockstore>`, if you haven't already. You will use the credentials of this user to submit Dockstore metrics.
+2. Contact us by opening a :ref:`helpdesk ticket<faq-contact-admins>` on `Discourse <https://discuss.dockstore.org/>`_ and we will give your Dockstore user a platform partner role.
+
+.. _Submit executions:
+
+Submit workflow metrics to Dockstore
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Once you have the metrics formatted and you registered a platform partner Dockstore user, you may use the Dockstore API to submit workflow metrics. You will need the Dockstore token of a user with a platform partner role.
 
 Go to https://dockstore.org/api/static/swagger-ui/index.html#/extendedGA4GH/executionMetricsPost and provide your Dockstore token using the lock icon at the top right of the endpoint. This is the endpoint used to submit metrics to Dockstore. Click the "Try it out” button.
 
@@ -184,12 +212,41 @@ The curl command looks something like:
       ]
    }'
 
-If it was submitted successfully, you should receive a ``204`` response code. 
+If it was submitted successfully, you should receive a ``204`` response code.
+
+.. _How to view metrics:
+
+Viewing Aggregated Workflow Metrics in Dockstore
+------------------------------------------------
+
+Dockstore aggregates the workflow execution metrics that platforms submit into aggregated metrics for users to view.
+
+To search for workflows with metrics, navigate to the `search <https://dockstore.org/search>`_ page and select a platform for the Execution Metrics and/or Validation Metrics facets.
+
+.. figure:: /assets/images/docs/metrics/metrics-search-facets.png
+    :alt: Metrics search facets
+
+Select a workflow and click on the Versions tab. Versions that have metrics have a check mark in the Metrics column.
+
+.. figure:: /assets/images/docs/metrics/versions-metrics-column.png
+    :alt: Metrics column in Versions table
+
+Select a version with metrics then click on the Metrics tab to view the metrics available.
+
+.. figure:: /assets/images/docs/metrics/metrics-tab.png
+    :alt: Metrics tab
+
+.. _Viewing And Updating Submitted Workflow Metrics:
+
+Viewing And Updating Submitted Workflow Metrics
+-----------------------------------------------
+
+As a platform owner who previously :ref:`submitted execution metrics to Dockstore<Getting started with submitting metrics>`, you may want to view or update submitted executions.
 
 .. _View submitted execution:
 
-How do I view a submitted execution?
-------------------------------------
+Viewing a submitted execution
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 To view an execution that you previously submitted, you can retrieve it by its execution ID.
 
@@ -248,8 +305,8 @@ If the request was successful, you should receive a ``200`` status code and the
 
 .. _Update submitted execution:
 
-How do I update workflow metrics?
----------------------------------
+Updating workflow metrics
+^^^^^^^^^^^^^^^^^^^^^^^^^
 
 You may want to update metrics that you have previously submitted because you received new metrics for the execution at a later time.
 

docs/faq.rst

@@ -91,6 +91,8 @@ How should I register my work in Dockstore?
 
 :ref:`(back to top) <topFAQ>`
 
+.. _faq-contact-admins:
+
 How do I send private messages to administrators or report security vulnerabilities?
 ------------------------------------------------------------------------------------