Lock operations around Process objects

The multiprocessing module is not thread-safe in general, but it even
has some multi-threading bugs when handling single objects due to
globals not being correctly synchronised. For an example of this see
python/cpython#85037.

This was found while working on YAN-975.

Signed-off-by: Rodrigo Tobar <rtobar@icrar.org>

daliuge-engine/dlg/drop.py

@@ -2615,6 +2615,8 @@ def async_execute(self):
             t.daemon = 1
             t.start()
 
+    _dlg_proc_lock = threading.Lock()
+
     @track_current_drop
     def execute(self, _send_notifications=True):
         """
@@ -2636,8 +2638,13 @@ def execute(self, _send_notifications=True):
             try:
                 if hasattr(self, "_tp"):
                     proc = DlgProcess(target=self.run, daemon=True)
-                    proc.start()
-                    proc.join()
+                    # see YAN-975 for why this is happening
+                    lock = InputFiredAppDROP._dlg_proc_lock
+                    with lock:
+                        proc.start()
+                    with lock:
+                        proc.join()
+                    proc.close()
                     if proc.exception:
                         raise proc.exception
                 else: