[DM-33438] Clean up run_demo.sh

[cbanek]

No description provided.

[cbanek]

This script works if I take out the multiprocessing stuff, but with the multiprocessing, I get the following error from the python tests:

___________________________________________________________________ PiplinesCheckTestCase.testExecutionButler (chain='demo_collection_exe') ___________________________________________________________________

self = <test_butler.PiplinesCheckTestCase testMethod=testExecutionButler>

def testExecutionButler(self):
    """Check outputs match in both runs."""

    for chain in (EXE_CHAIN, QBB_CHAIN):
        with self.subTest(chain=chain):
            # Check that we have identical datasets in both collections
            # except for the dataset.id
            main_datasets = self._get_datasets_from_chain(MAIN_CHAIN)
            datasets = self._get_datasets_from_chain(chain)
            self.assertGreater(len(datasets), 0)

          self.assertEqual(len(main_datasets), len(datasets))

E AssertionError: 21 != 13

tests/test_butler.py:101: AssertionError
___________________________________________________________________ PiplinesCheckTestCase.testExecutionButler (chain='demo_collection_qbb') ___________________________________________________________________

self = <test_butler.PiplinesCheckTestCase testMethod=testExecutionButler>

def testExecutionButler(self):
    """Check outputs match in both runs."""

    for chain in (EXE_CHAIN, QBB_CHAIN):
        with self.subTest(chain=chain):
            # Check that we have identical datasets in both collections
            # except for the dataset.id
            main_datasets = self._get_datasets_from_chain(MAIN_CHAIN)
            datasets = self._get_datasets_from_chain(chain)
            self.assertGreater(len(datasets), 0)

          self.assertEqual(len(main_datasets), len(datasets))

E AssertionError: 21 != 13

tests/test_butler.py:101: AssertionError

[timj]

That test is saying that the number of datasets created in the EXECUTION BUTLER bit of code (the first 3 pipetask calls using $NODE) is equal to the number of datasets created by the QUNTUM BACKED BUTLER bit of code (the second set of 3 pipetask calls using $NODE).

[timj]

This looks good. There is a small possibility that the two parallel processes hitting sqlite would fall over each other but given that the second run is using the graph and not the sqlite at all and we only have two processes, I think this will almost certainly be fine. cc/ @andy-slac