incomplete first draft of expanded data sharing docs (#590)

* incomplete first draft of expanded data sharing docs

* update sharing documentation

* improved data sharing docs

---------

Co-authored-by: Joseph Chatelain <jchatelain@lco.global>

docs/managing_data/index.rst

@@ -9,7 +9,8 @@ Managing Data
   ../api/tom_dataproducts/views
   plotting_data
   customizing_data_processing
-  customizing_data_sharing
+  tom_direct_sharing
+  stream_pub_sub
 
 
 :doc:`Creating Plots from TOM Data <plotting_data>` - Learn how to create plots using plot.ly and your TOM
@@ -18,4 +19,8 @@ data to display anywhere in your TOM.
 :doc:`Adding Custom Data Processing <customizing_data_processing>` - Learn how you can process data into your
 TOM from uploaded data products.
 
-:doc:`Adding Custom Data Sharing <customizing_data_sharing>` - Learn how you can share data from your TOM.
+:doc:`TOM-TOM Direct Sharing <tom_direct_sharing>` - Learn how you can send and receive data between your TOM and another TOM-Toolkit TOM via an API.
+
+:doc:`Publish and Subscribe to a Kafka Stream <stream_pub_sub>` - Learn how to publish and subscribe to a Kafka stream topic.
+
+

docs/managing_data/stream_pub_sub.rst

@@ -0,0 +1,78 @@
+Publish and Subscribe to a Kafka Stream
+---------------------------------------
+
+Publishing data to a stream and subscribing to a stream are handled independently and we describe each below.
+
+
+Publish Data to a Kafka Topic
+#############################
+
+TOM Toolkit supports publishing data to a Kafka stream such as `Hermes <https://hermes.lco.global>`_ (an interface to
+`HOPSKOTCH <https://hop.scimma.org>`_) and `GCNClassicOverKafka <https://gcn.nasa.gov>`_.
+
+When sharing photometry data via Hermes, the TOM publishes the data to be shared to a topic on the HOPSKOTCH
+Kafka stream. At this time, only photometry data is supported.
+
+
+Configuring your TOM to Publish Data to a stream:
+*************************************************
+
+You will need to add a ``DATA_SHARING`` configuration dictionary to your ``settings.py`` that gives the credentials
+for the various streams with which you wish to share data.
+
+.. code:: python
+
+   # Define the valid data sharing destinations for your TOM.
+   DATA_SHARING = {
+        'hermes': {
+           'DISPLAY_NAME': os.getenv('HERMES_DISPLAY_NAME', 'Hermes'),
+           'BASE_URL': os.getenv('HERMES_BASE_URL', 'https://hermes.lco.global/'),
+           'CREDENTIAL_USERNAME': os.getenv('SCIMMA_CREDENTIAL_USERNAME',
+                                             'set SCIMMA_CREDENTIAL_USERNAME value in environment'),
+           'CREDENTIAL_PASSWORD': os.getenv('SCIMMA_CREDENTIAL_PASSWORD',
+                                             'set SCIMMA_CREDENTIAL_PASSWORD value in environment'),
+           'USER_TOPICS': ['hermes.test', 'tomtoolkit.test']
+        },
+   }
+
+Subscribe to a Kafka Topic
+##########################
+
+TOM Toolkit allows a TOM to subscribe to a topic on a Kafka stream, ingesting messages from that topic and handling the data.
+This could involve simply logging the message or extracting the data from the message and saving it if it is properly formatted.
+
+Configuring your TOM to subscribe to a stream:
+**********************************************
+
+First you will need to add ``tom_alertstreams`` to your list of ``INSTALLED_APPS`` in your ``settings.py``.
+
+.. code:: python
+
+    INSTALLED_APPS = [
+        ...
+        'tom_alertstreams',
+    ]
+
+Then you will need to add an ``ALERT_STREAMS`` configuration dictionary to your ``settings.py``. This gives the credentials
+for the various streams to which you wish to subscribe. Additionally, the ``TOPIC_HANDLERS`` section of the stream ``OPTIONS``
+will include a list of handlers for each topic.
+
+Some alert handlers are included as examples. Below we demonstrate how to connect to a Hermes Topic. You'll want to check
+out the ``tom-alertstreams`` `README <https://github.com/TOMToolkit/tom-alertstreams>`_ for more details.
+
+.. code:: python
+
+    ALERT_STREAMS = [
+        {
+            'ACTIVE': True,
+            'NAME': 'tom_alertstreams.alertstreams.hopskotch.HopskotchAlertStream',
+            'OPTIONS': {
+                'URL': 'kafka://kafka.scimma.org/',
+                'USERNAME': os.getenv('SCIMMA_CREDENTIAL_USERNAME', 'set SCIMMA_CREDENTIAL_USERNAME value in environment'),
+                'PASSWORD': os.getenv('SCIMMA_CREDENTIAL_PASSWORD', 'set SCIMMA_CREDENTIAL_USERNAME value in environment'),
+                'TOPIC_HANDLERS': {
+                    'tomtoolkit.test': 'tom_dataproducts.alertstreams.hermes.hermes_alert_handler',
+                },
+            },
+        },
+    ]

docs/managing_data/tom_direct_sharing.rst

@@ -0,0 +1,32 @@
+Sharing Data with Other TOMs
+############################
+
+TOM Toolkit does not yet support direct sharing between TOMs, however we hope to add this functionality soon.
+
+
+..  Configuring your TOM to submit data to another TOM:
+..  ***************************************************
+..  
+..  You will need to add a ``DATA_SHARING`` configuration dictionary to your ``settings.py`` that gives the credentials
+..  for the various TOMs with which you wish to share data.
+..  
+..  .. code:: python
+..  
+..     # Define the valid data sharing destinations for your TOM.
+..     DATA_SHARING = {
+..          'tom-demo-dev': {
+..              'DISPLAY_NAME': os.getenv('TOM_DEMO_DISPLAY_NAME', 'TOM Demo Dev'),
+..              'BASE_URL': os.getenv('TOM_DEMO_BASE_URL', 'http://tom-demo-dev.lco.gtn/'),
+..              'USERNAME': os.getenv('TOM_DEMO_USERNAME', 'set TOM_DEMO_USERNAME value in environment'),
+..              'PASSWORD': os.getenv('TOM_DEMO_PASSWORD', 'set TOM_DEMO_PASSWORD value in environment'),
+..          },
+..          'localhost-tom': {
+..              # for testing; share with yourself
+..              'DISPLAY_NAME': os.getenv('LOCALHOST_TOM_DISPLAY_NAME', 'Local'),
+..              'BASE_URL': os.getenv('LOCALHOST_TOM_BASE_URL', 'http://127.0.0.1:8000/'),
+..              'USERNAME': os.getenv('LOCALHOST_TOM_USERNAME', 'set LOCALHOST_TOM_USERNAME value in environment'),
+..              'PASSWORD': os.getenv('LOCALHOST_TOM_PASSWORD', 'set LOCALHOST_TOM_PASSWORD value in environment'),
+..          }
+..  
+..     }
+..