Add MAX_CUTOUT_IMAGES setting (#658)

* added MAX_CUTOUT_IMAGES setting
Truncates the number of displayed cutout images on the source detail page. Fixes #657.

* updated changelog

CHANGELOG.md

@@ -8,6 +8,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 
 #### Added
 
+- Added `MAX_CUTOUT_IMAGES` to the pipeline settings to limit the number of postage stamps displayed on the source detail page [#658](https://github.com/askap-vast/vast-pipeline/pull/658).
 - Added run config option to skip calculating measurement pair metrics [#655](https://github.com/askap-vast/vast-pipeline/pull/655).
 - Added support for Python 3.10 [#641](https://github.com/askap-vast/vast-pipeline/pull/641).
 - Added documentation versioning [#627](https://github.com/askap-vast/vast-pipeline/pull/627).
@@ -98,6 +99,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 
 #### List of PRs
 
+- [#658](https://github.com/askap-vast/vast-pipeline/pull/658): feat: Add MAX_CUTOUT_IMAGES setting.
 - [#655](https://github.com/askap-vast/vast-pipeline/pull/655): feat: Add run config option to disable measurement pairs.
 - [#648](https://github.com/askap-vast/vast-pipeline/pull/648): fix: make Image and Measurement creation atomic together.
 - [#653](https://github.com/askap-vast/vast-pipeline/pull/653): fix: Allow forced fitting on images with empty catalogues.

docs/exploringwebsite/sourcedetail.md

@@ -62,7 +62,7 @@ Users are able to read and post comments on a source using this form, in additio
 
 [JS9 website](https://js9.si.edu){ target=_blank }.
 
-More JS9 viewers are used to show the postage stamp FITS images of the measurements that are associated with the source, loaded from their respective image FITS files.
+More JS9 viewers are used to show the postage stamp FITS images of the measurements that are associated with the source, loaded from their respective image FITS files. The number of cutouts to display is configurable with the `MAX_CUTOUT_IMAGES` setting: only the first `MAX_CUTOUT_IMAGES` measurements will be displayed as cutout images. A warning will be displayed if the number of displayed cutouts has been truncated. Refer to the [pipeline configuration](../gettingstarted/configuration.md#pipeline) documentation for more information.
 
 !!! note
     If the image data is removed from its location when the pipeline run was processed the JS9 viewer will no longer work.

docs/gettingstarted/configuration.md

@@ -154,6 +154,7 @@ Shown below is the [`.env.template`](https://github.com/askap-vast/vast-pipeline
     # PIPELINE_MAINTAINANCE_MESSAGE=Uncomment and fill to show
     MAX_PIPELINE_RUNS=3
     MAX_PIPERUN_IMAGES=200
+    MAX_CUTOUT_IMAGES=30
 
     # Q_CLUSTER_TIMEOUT=86400
     # Q_CLUSTER_RETRY=86402
@@ -222,6 +223,7 @@ These settings apply to various aspects of the VAST pipeline itself. The table b
 | `PIPELINE_MAINTAINANCE_MESSAGE` | Disabled | The message to display at the top of the webserver. See image below this table for an example. Comment out the setting to disable. |
 | `MAX_PIPELINE_RUNS` | 3 | The allowed maximum number of concurrent pipeline runs. |
 | `MAX_PIPERUN_IMAGES` | 200 | The allowed maximum number of images in a single pipeline run (non-admins). |
+| `MAX_CUTOUT_IMAGES` | 30 | The maximum number of cutout images to display on the source detail pages. The first `MAX_CUTOUT_IMAGES` cutouts are displayed, in temporal order. |
 | `Q_CLUSTER_TIMEOUT` | 86400 | Number of seconds a Django-Q cluster worker may spend on a task before it is terminated. See the [Django-Q documentation](https://django-q.readthedocs.io/en/latest/configure.html#timeout). |
 | `Q_CLUSTER_RETRY` | 86402 | Number of seconds a Django-Q broker will wait for a cluster to finish a task before it's presented again. See the [Django-Q documentation](https://django-q.readthedocs.io/en/latest/configure.html#retry). |
 | `Q_CLUSTER_MAX_ATTEMPTS` | 1 | Number of times a failed task is retried. See the [Django-Q documentation](https://django-q.readthedocs.io/en/latest/configure.html#max-attempts). |

templates/source_detail.html

@@ -142,7 +142,7 @@ <h6 class="m-0 font-weight-bold text-primary">Details</h6>
           <h6 class="m-0 font-weight-bold text-primary">First Detection Postage Stamp</h6>
         </div>
         <div class="card-body">
-          <div class="JS9" id="JS9_{{ source.n_meas|add:"1" }}" data-width="450px" data-height="360px"></div>
+          <div class="JS9" id="JS9_{{ cutout_measurements|length|add:"1" }}" data-width="450px" data-height="360px"></div>
         </div>
       </div>
     </div>
@@ -282,6 +282,13 @@ <h6 class="card-header m-0 font-weight-bold text-primary">
     </h6>
     <div id="collapse-postage-stamps" class="collapse show" aria-labelledby="heading-example">
       <div class="card-body">
+        {% if cutout_measurements|length < datatables.0.dataQuery|length %}
+        <div class="alert alert-warning" role="alert">
+          Number of displayed postage stamp cutouts has been truncated. Displaying the first
+          {{ cutout_measurements|length }} of {{ datatables.0.dataQuery|length }}
+          measurement cutouts.
+        </div>
+        {% endif %}
         <div class="row">
           <button class="btn btn-info mr-1" onclick="focusOnSource(4.0);">Zoom to Source</button>
           <button class="btn btn-info mr-1" onclick="focusOnSource('toFit');">Zoom to Field</button>
@@ -290,7 +297,7 @@ <h6 class="card-header m-0 font-weight-bold text-primary">
           <div class="JS9Menubar" id="SUPERMENU_" data-displays="*"></div>
         </div>
         <div class="row">
-          {% for measurement in datatables.0.dataQuery %}
+          {% for measurement in cutout_measurements %}
           <div class="card">
             <div class="card-header">
               <span class="badge badge-pill badge-primary float-right">{{ forloop.counter0 }}</span>
@@ -458,7 +465,8 @@ <h6 class="m-0 font-weight-bold text-primary">Related Sources</h6>
     })
 </script>
 
-{{ datatables.0.dataQuery|json_script:"data-js9" }}
+{{ cutout_measurements|json_script:"data-js9" }}
+{{ first_det_meas|json_script:"data-js9-first-detection" }}
 <script type="text/javascript">
   let baseUrl = '{{ base_url }}' || null;
   // function to add base url to an existing url if valid
@@ -508,7 +516,8 @@ <h6 class="m-0 font-weight-bold text-primary">Related Sources</h6>
   // when JS9 is ready, load the cutout images into the correct display panels
   $(document).on("JS9:ready", async function () {
     var measurementData = JSON.parse(document.getElementById("data-js9").textContent);
-    measurementData.push(measurementData[{{ first_det_meas_index }}]);
+    var firstDetectionMeasurementData = JSON.parse(document.getElementById("data-js9-first-detection").textContent);
+    measurementData.push(firstDetectionMeasurementData);
     // load all images and wait until they're all done
     await Promise.all(measurementData.map((measurement, i) => {
       var displayId = "JS9_" + (i + 1);

vast_pipeline/views.py

@@ -1672,15 +1672,25 @@ def SourceDetail(request, pk):
         'forced',
         'image_id'
     ]
-    measurements = list(
-        Measurement.objects.filter(source__id=pk).annotate(
-            datetime=F('image__datetime'),
-            image_name=F('image__name'),
-            frequency=F('image__band__frequency'),
-        ).order_by('datetime').values(*tuple(cols))
+    measurements_qs = (
+        Measurement.objects.filter(source__id=pk)
+        .annotate(datetime=F("image__datetime"), image_name=F("image__name"))
+        .order_by("datetime")
     )
 
-    first_det_meas_index = [i['forced'] for i in measurements].index(False)
+    measurements = list(
+        measurements_qs.annotate(
+            frequency=F("image__band__frequency"),
+        ).values(*tuple(cols))
+    )
+    # subset of measurements used for the cutouts
+    measurements_cutouts = list(
+        measurements_qs[: settings.MAX_CUTOUT_IMAGES].values(
+            "id", "ra", "dec", "image_id", "image_name",
+        )
+    )
+    # get the measurement for the first detection - used for the first detection cutout
+    first_det_meas = measurements[[i['forced'] for i in measurements].index(False)]
 
     for one_m in measurements:
         one_m['datetime'] = one_m['datetime'].isoformat()
@@ -1793,8 +1803,9 @@ def SourceDetail(request, pk):
         'source': source,
         'source_next_id': source_next_id,
         'source_previous_id': source_previous_id,
-        'first_det_meas_index': first_det_meas_index,
+        'first_det_meas': first_det_meas,
         'datatables': [measurements, related_datatables],
+        'cutout_measurements': measurements_cutouts,
         # flag to deactivate starring and render yellow star
         'sourcefav': (
             SourceFav.objects.filter(

webinterface/.env.template

@@ -32,6 +32,7 @@ HOME_DATA_DIR=vast-pipeline-extra-data
 # PIPELINE_MAINTAINANCE_MESSAGE=Uncomment and fill to show
 MAX_PIPELINE_RUNS=3
 MAX_PIPERUN_IMAGES=200
+MAX_CUTOUT_IMAGES=30
 
 # Q_CLUSTER_TIMEOUT=86400
 # Q_CLUSTER_RETRY=86402

webinterface/settings.py

@@ -344,6 +344,9 @@
 # maximum number of images for non-admin runs
 MAX_PIPERUN_IMAGES = env('MAX_PIPERUN_IMAGES', cast=int, default=200)
 
+# maximum number of cutout images to render on the source detail page
+MAX_CUTOUT_IMAGES = env('MAX_CUTOUT_IMAGES', cast=int, default=30)
+
 # pipeline maintainance message/banner
 PIPELINE_MAINTAINANCE_MESSAGE = env('PIPELINE_MAINTAINANCE_MESSAGE', cast=str, default=None)