Merge branch 'feature/tom2tom_data_sharing' into feature/tom-tom-targ…

…et-sharing

.github/dependabot.yml

@@ -9,5 +9,5 @@ updates:
   open-pull-requests-limit: 10
   target-branch: dev
   reviewers:
-  - dmcollom
+  - jchate6
   - phycodurus

docs/brokers/create_broker.rst

@@ -16,7 +16,7 @@ Be sure you've followed the :doc:`Getting Started </introduction/getting_started
 
     The following Python/Django concepts are used in this tutorial. While this tutorial does not assume familiarity with the concepts, you will likely find the tutorial easier to understand and build upon if you read these in advance.
 
-    - `Working with Django Forms <https://docs.djangoproject.com/en/2.1/topics/forms/>`_
+    - `Working with Django Forms <https://docs.djangoproject.com/en/stable/topics/forms/>`_
     - `Requests Official API Docs <http://docs.python-requests.org/en/master/>`_
 
 TOM Alerts module

docs/observing/observation_module.rst

@@ -11,8 +11,7 @@ You can use this example as the foundation to build an observing
 facility module to connect to a real observatory or track observations
 on non-API supported facilities.
 
-Be sure you’ve followed the `Getting
-Started </introduction/getting_started>`__ guide before continuing onto
+Be sure you’ve followed the :doc:`Getting Started </introduction/getting_started>` guide before continuing onto
 this tutorial.
 
 What is a observing facility module?
@@ -31,16 +30,21 @@ respective observatories through a TOM.
 Prerequisites
 ~~~~~~~~~~~~~
 
-You should have a working TOM already. You can start where the `Getting
-Started </introduction/getting_started>`__ guide leaves off. You should
-also be familiar with the observing facility’s API that you would like
-to work with.
+You should have a working TOM already. You can start where the :doc:`Getting Started </introduction/getting_started>`
+guide leaves off. You should also be familiar with the observing facility’s API that you would like to work with.
 
-Creating a custom robotic facility
-----------------------------------
+.. tip:: Read these first!
+
+    The following Python/Django concepts are used in this tutorial. While this tutorial does not assume familiarity with the concepts, you will likely find the tutorial easier to understand and build upon if you read these in advance.
+
+    - `Working with Django Forms <https://docs.djangoproject.com/en/stable/topics/forms/>`_
+    - `Requests Official API Docs <http://docs.python-requests.org/en/master/>`_
+
+Creating a Custom Facility
+~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 Defining the minimal implementation
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+-----------------------------------
 
 Within any existing module in your TOM you should create a new python
 module (file) named ``myfacility.py``. For example, if you have a fresh
@@ -101,7 +105,7 @@ This means our new observation facility module has been successfully
 loaded.
 
 BaseRoboticObservationFacility and BaseRoboticObservationForm
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+-------------------------------------------------------------
 
 You will have noticed our module consists of two classes that inherit
 from two other classes.
@@ -120,7 +124,7 @@ super class, contains logic and layout that all observation facility
 form classes should contain.
 
 Implementing observation submission
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+-----------------------------------
 
 Try to click on the button for ``MyFacility``. It should return an error
 that says everything it’s missing:
@@ -167,22 +171,26 @@ Reload the page and now it should look something like this:
 
 |image1|
 
-Some notes: 1. The form is empty, but we’ll fix that next. 2. The
-``name`` variable of ``MyObservationFacility`` determines what the top
-of the page says (``Submit an observation to MyFacility``). It also
-determines the name of the button under “Observe” on the target’s page.
-3. You should see a tab for ``Custom Observation`` as the only option on
-the page. This is read from the ``observation_forms`` variable in
-``MyObservationFacility``. That variable is a dict. The
-value of each dict item is the observation form class. The key of each
-dict item is what should be used to distinguish different observation types
-in your code, which will be displayed in Pascal Case in the observation form tabs.
-To see a demonstration of this, check out the `Las Cumbres Observatory <https://github.com/TOMToolkit/tom_base/blob/main/tom_observations/facilities/lco.py>`__
-facility’s ``observation_forms`` and ``get_form``.
+:Some notes:
+
+#. The form is empty, but we’ll fix that next.
+
+#. The ``name`` variable of ``MyObservationFacility`` determines what the top
+    of the page says (``Submit an observation to MyFacility``). It also
+    determines the name of the button under “Observe” on the target’s page.
+
+#. You should see a tab for ``Custom Observation`` as the only option on
+    the page. This is read from the ``observation_forms`` variable in
+    ``MyObservationFacility``. That variable is a dict. The
+    value of each dict item is the observation form class. The key of each
+    dict item is what should be used to distinguish different observation types
+    in your code, which will be displayed in Pascal Case in the observation form tabs.
+    To see a demonstration of this, check out the `Las Cumbres Observatory <https://github.com/TOMToolkit/tom_base/blob/main/tom_observations/facilities/lco.py>`__
+    facility’s ``observation_forms`` and ``get_form``.
 
 Now let’s populate the form. Let’s assume our observatory only requires
-us to send 2 parameters (besides the target data): exposure_time and
-exposure_count. Let’s start by adding them to our form class:
+us to send 2 parameters (besides the target data): ``exposure_time`` and
+``exposure_count``. Let’s start by adding them to our form class:
 
 .. code:: python
 
@@ -207,7 +215,7 @@ a crispy_forms class with ``from crispy_forms.layout import Layout``. Finally,
 we've defined a function ``layout(self)`` that is used to display the fields that 
 we've created.
 
-All fields must show be named in the ``layout`` function in order to be displayed, and 
+All fields must be named in the ``layout`` function in order to be displayed, and
 the ``layout`` function is also how we could make the layout more sophisticated. See the 
 `django-crispy-forms documentation <https://django-crispy-forms.readthedocs.io/en/latest/>`__ 
 and the `lco.py module <https://github.com/TOMToolkit/tom_base/blob/main/tom_observations/facilities/lco.py>`__ for examples.
@@ -283,27 +291,24 @@ its “Observations” tab you can see the parameters of the observation you
 just submitted in more detail.
 
 Filling in the rest of the functionality
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+----------------------------------------
 
 You’ll notice we added many more methods other than
 ``submit_observation`` to our Facility class. For now they return dummy
 data, but when you adapt it to work with a real observatory you should
 fill them in with the correct logic so that the whole module works
-correctly with the TOM. You can view explanations of each method `in the
-source
-code <https://github.com/TOMToolkit/tom_base/blob/main/tom_observations/facility.py#L142>`__
+correctly with the TOM. You can view explanations of each method in the :doc:`Facility Modules <../api/tom_observations/facilities>`
+section of the documentation or
+`in the source code <https://github.com/TOMToolkit/tom_base/blob/main/tom_observations/facility.py#L181>`__
 
-###Airmass plotting for new facilities The last step in adding a new
-facility is to get it to appear on airmass plots. If you input two dates
-into the “Plan” form under the “Observe” tab on a target’s page, you’ll
-see the target’s visibility. By default, the plot shows you the airmass
-at LCO and Gemini sites.
+Adding Sites to new Facilities
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 In our ``MyObservationFacility`` class, let’s define a new variable
 called ``SITES``. Modeling our ``SITES`` on the one defined for `Las
 Cumbres
 Observatory <https://github.com/TOMToolkit/tom_base/blob/main/tom_observations/facilities/lco.py>`__,
-we can easily put new sites into the airmass plots:
+we can easily put new sites to our facility that will then show up in the airmass plots:
 
 .. code:: python
 
@@ -329,15 +334,20 @@ we can easily put new sites into the airmass plots:
 (Koichi Itagaki is an “amateur” astronomer in Japan who has discovered
 many extremely interesting supernovae.)
 
-Now the new observatory site should show up when you generate airmass
-plots. Even if the facilities you observe at are not API-accessible, you
+Airmass plotting for new facilities
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Now that your facility has sites, they should appear on airmass plots. If you input two dates
+into the “Plan” form under the “Observe” tab on a target’s page, you’ll
+see the target’s visibility. By default, the plot shows you the airmass
+at LCO and Gemini sites. To remove these, you can delete these facilities from ``TOM_FACILITY_CLASSES`` in ``settings.py``.
+
+Even if the facilities you observe at are not API-accessible, you
 can still add them to your TOM’s airmass plots to judge what targets to
 observe when.
 
 Happy developing!
 
-Creating a custom manual facility
----------------------------------
 
 .. |image0| image:: /_static/observation_module/myfacility.png
 .. |image1| image:: /_static/observation_module/empty_form.png

setup.py

@@ -45,14 +45,15 @@
         'fits2image>=0.4.2',
         'Markdown~=3.4',  # django-rest-framework doc headers require this to support Markdown
         'numpy~=1.20',
-        'pillow~=9.2',
+        'pillow>=9.2,<11.0',
         'plotly~=5.0',
         'python-dateutil~=2.8',
         'requests~=2.25',
         'specutils~=1.8',
     ],
     extras_require={
-        'test': ['factory_boy~=3.2.1', 'responses~=0.23'],
+        'test': ['factory_boy>=3.2.1,<3.4.0',
+                 'responses~=0.23'],
         'docs': [
             'recommonmark~=0.7',
             'sphinx>=4,<8',

tom_alerts/brokers/alerce.py

@@ -132,8 +132,8 @@ def __init__(self, *args, **kwargs):
         self.helper.layout = Layout(
             HTML('''
                 <p>
-                Please see the <a href="http://alerce.science/">ALeRCE homepage</a> for information about the ALeRCE
-                filters.
+                Please see the <a href="http://alerce.science/" target="_blank">ALeRCE homepage</a> for information
+                about the ALeRCE filters.
             '''),
             self.common_layout,
             'oid',
@@ -308,18 +308,25 @@ def _clean_parameters(self, parameters):
     def _request_alerts(self, parameters):
         payload = self._clean_parameters(parameters)
         logger.log(msg=f'Fetching alerts from ALeRCE with payload {payload}', level=logging.INFO)
-        args = urlencode(self._clean_parameters(parameters))
+        args = urlencode(payload)
         response = requests.get(f'{ALERCE_SEARCH_URL}/objects/?count=false&{args}')
         response.raise_for_status()
         return response.json()
 
     def fetch_alerts(self, parameters):
+        """
+        Loop through pages of ALeRCE alerts until we reach the maximum pages requested.
+        This simply concatenates all alerts from n pages into a single iterable to be returned.
+        """
         response = self._request_alerts(parameters)
         alerts = response['items']
         broker_feedback = ''
-        if len(alerts) > 0 and response['page'] < parameters.get('max_pages', 1):
-            parameters['page'] = response.get('page') + 1
+        current_page = parameters.get('page', 1)
+        if len(alerts) > 0 and current_page < parameters.get('max_pages', 1):
+            # make new request for the next page. (by recursion)
+            parameters['page'] = current_page + 1
             alerts += self.fetch_alerts(parameters)[0]
+        # Bottom out of recursion and return accumulated alerts
         return iter(alerts), broker_feedback
 
     def fetch_alert(self, id):

tom_alerts/brokers/antares.py

@@ -11,7 +11,7 @@ def __init__(self, *args, **kwargs):
             HTML('''
                 <p>
                 This plugin is a stub for the ANTARES plugin. In order to install the full plugin, please see the
-                instructions <a href="https://github.com/TOMToolkit/tom_antares">here</a>.
+                instructions <a href="https://github.com/TOMToolkit/tom_antares" target="_blank">here</a>.
                 </p>
             '''),
             HTML('''<a class="btn btn-outline-primary" href={% url 'tom_alerts:list' %}>Back</a>''')

tom_alerts/brokers/gaia.py

@@ -30,8 +30,8 @@ def __init__(self, *args, **kwargs):
         self.helper.layout = Layout(
             HTML('''
                 <p>
-                Please see the <a href="http://gsaweb.ast.cam.ac.uk/alerts/tableinfo">Gaia homepage</a> for a detailed
-                description of this broker.
+                Please see the <a href="http://gsaweb.ast.cam.ac.uk/alerts/tableinfo" target="_blank">Gaia homepage</a>
+                for a detailed description of this broker.
             '''),
             self.common_layout,
             Fieldset(

tom_alerts/brokers/hermes.py

@@ -11,7 +11,7 @@ def __init__(self, *args, **kwargs):
             HTML('''
                 <p>
                 This plugin is a stub for the Hermes Broker plugin. In order to install the full plugin, please see the
-                instructions <a href="https://github.com/TOMToolkit/tom_hermes">here</a>.
+                instructions <a href="https://github.com/TOMToolkit/tom_hermes" target="_blank">here</a>.
                 </p>
             '''),
             HTML('''<a class="btn btn-outline-primary" href={% url 'tom_alerts:list' %}>Back</a>''')

tom_alerts/brokers/lasair.py

@@ -27,8 +27,8 @@ def __init__(self, *args, **kwargs):
         self.helper.layout = Layout(
             HTML('''
                     <p>
-                    Please see the <a href="https://lasair-ztf.lsst.ac.uk/api">Lasair website</a> for more detailed
-                    instructions on querying the broker.
+                    Please see the <a href="https://lasair-ztf.lsst.ac.uk/api" target="_blank">Lasair website</a> for
+                    more detailed instructions on querying the broker.
                 '''),
             self.common_layout,
             Fieldset(

tom_alerts/brokers/scout.py

@@ -18,8 +18,8 @@ def __init__(self, *args, **kwargs):
         self.helper.layout = Layout(
             HTML('''
                 <p>
-                Please see the <a href="https://ssd-api.jpl.nasa.gov/doc/scout.html">Scout API Reference</a>
-                for a detailed description of the service.
+                Please see the <a href="https://ssd-api.jpl.nasa.gov/doc/scout.html target="_blank"">Scout API
+                Reference</a> for a detailed description of the service.
                 </p>
             '''),
             self.common_layout,

tom_alerts/brokers/tns.py

@@ -32,9 +32,9 @@ class TNSForm(GenericQueryForm):
     days_ago = forms.FloatField(required=False, min_value=0.,
                                 label='Discovered in the Last __ Days',
                                 help_text='Leave blank to use the "Discovered After" field')
-    min_date = forms.DateTimeField(required=False,
-                                   label='Discovered After',
-                                   help_text='Most valid date formats are recognized')
+    min_date = forms.CharField(required=False,
+                               label='Discovered After',
+                               help_text='Most valid date formats are recognized')
     days_from_nondet = forms.FloatField(required=False, min_value=0.,
                                         label='Days From Nondetection',
                                         help_text='Maximum time between last nondetection and first detection')
@@ -44,8 +44,8 @@ def __init__(self, *args, **kwargs):
         self.helper.layout = Layout(
             HTML('''
                 <p>
-                Please see <a href="https://wis-tns.weizmann.ac.il/sites/default/files/api/TNS_APIs_manual.pdf">
-                the TNS API Manual</a> for a detailed description of available filters.
+                Please see <a href="https://wis-tns.weizmann.ac.il/sites/default/files/api/TNS_APIs_manual.pdf"
+                target="_blank">the TNS API Manual</a> for a detailed description of available filters.
                 </p>
             '''),
             self.common_layout,
@@ -106,7 +106,7 @@ def fetch_alerts(cls, parameters):
             public_timestamp = (datetime.utcnow() - timedelta(days=parameters['days_ago']))\
                 .strftime('%Y-%m-%d %H:%M:%S')
         elif parameters['min_date'] is not None:
-            public_timestamp = parameters['min_date'].strftime('%Y-%m-%d %H:%M:%S')
+            public_timestamp = parameters['min_date']
         else:
             public_timestamp = ''
         data = {

tom_common/templates/tom_common/index.html

@@ -9,7 +9,7 @@ <h2 class="text-center">Congratulations! Welcome to your new TOM.</h2>
     <h3>Next steps</h3>
     <ul>
       <li>
-        Check out the <a href="https://tom-toolkit.readthedocs.io/en/stable/" title="TOM Toolkit home page">TOM
+        Check out the <a href="https://tom-toolkit.readthedocs.io/en/stable/" title="TOM Toolkit home page" target="_blank">TOM
         Toolkit homepage</a> for the latest news, downloads and documentation.
       </li>
       <li>
@@ -21,13 +21,13 @@ <h3>Next steps</h3>
         project's <code>urls.py</code>.
       </li>
       <li>
-        Take a look at some <a href="https://tom-toolkit.readthedocs.io/en/stable/customization/index.html">common first customizations</a>.
+        Take a look at some <a href="https://tom-toolkit.readthedocs.io/en/stable/customization/index.html" target="_blank">common first customizations</a>.
       </li>
     </ul>
     <h3>Other Resources</h3>
     <ul>
-      <li>The official <a href="https://www.djangoproject.com/">Django documentation</a>.</li>
-      <li>The official <a href="http://www.astropy.org/">Astropy documentation</a>.</li>
+      <li>The official <a href="https://www.djangoproject.com/" target="_blank">Django documentation</a>.</li>
+      <li>The official <a href="http://www.astropy.org/" target="_blank">Astropy documentation</a>.</li>
     </ul>
   </div>
   <div class="col-md-4">

tom_dataproducts/views.py

@@ -183,17 +183,30 @@ def get_success_url(self):
         referer = urlparse(referer).path if referer else '/'
         return referer
 
-    def delete(self, request, *args, **kwargs):
+    def form_valid(self, form):
         """
-        Method that handles DELETE requests for this view. First deletes all ``ReducedDatum`` objects associated with
-        the ``DataProduct``, then deletes the ``DataProduct``.
-
-        :param request: Django POST request object
-        :type request: HttpRequest
+        Method that handles DELETE requests for this view. It performs the following actions in order:
+        1. Deletes all ``ReducedDatum`` objects associated with the ``DataProduct``.
+        2. Deletes the file referenced by the ``DataProduct``.
+        3. Deletes the ``DataProduct`` object from the database.
+
+        :param form: Django form instance containing the data for the DELETE request.
+        :type form: django.forms.Form
+        :return: HttpResponseRedirect to the success URL.
+        :rtype: HttpResponseRedirect
         """
-        ReducedDatum.objects.filter(data_product=self.get_object()).delete()
-        self.get_object().data.delete()
-        return super().delete(request, *args, **kwargs)
+        # Fetch the DataProduct object
+        data_product = self.get_object()
+
+        # Delete associated ReducedDatum objects
+        ReducedDatum.objects.filter(data_product=data_product).delete()
+
+        # Delete the file reference.
+        data_product.data.delete()
+        # Delete the `DataProduct` object from the database.
+        data_product.delete()
+
+        return HttpResponseRedirect(self.get_success_url())
 
     def get_context_data(self, *args, **kwargs):
         """