GDA should use exception status from Artemis to work out if to continue

[DominicOram]

Fixes #679

Note: this contains changes from #722, that PR should be reviewed/merged first

The associated GDA change is https://gerrit.diamond.ac.uk/c/gda/gda-mx/+/39268/3, please +2 this at the same time.

To test:

1. Confirm unit tests still all pass
2. Using the GDA change above set it up to talk to Artemis by adding the following properties:

gda.gridscan.artemis.enabled=true
gda.gridscan.artemis.grid.detection=true
gda.gridscan.artemis.flaskServerAddress=localhost:5005
gda.mx.callArtemisInDummyMode=true

3. Paste the following into the data collection table:

<?xml version="1.0" encoding="UTF-8"?>
<ExtendedCollectRequests>
    <usingDna>false</usingDna>
    <extendedCollectRequest>
        <collect_request>
            <fileinfo>
                <directory>/tmp/gda/i03/data/2023/cm33866-3/ssa</directory>
                <prefix>sdasda</prefix>
            </fileinfo>
            <oscillation_sequence>
                <start>0.0</start>
                <range>0.1</range>
                <number_of_images>100</number_of_images>
                <overlap>0.0</overlap>
                <exposure_time>0.01</exposure_time>
                <start_image_number>1</start_image_number>
                <number_of_passes>1</number_of_passes>
            </oscillation_sequence>
            <wavelength>0.97625</wavelength>
            <resolution>
                <upper>1.9780068845283907</upper>
            </resolution>
            <sample_reference>
                <code>NR</code>
                <container_reference>1</container_reference>
                <sample_location>1</sample_location>
            </sample_reference>
        </collect_request>
        <runNumber>0</runNumber>
        <sampleDetectorDistanceInMM>289.3</sampleDetectorDistanceInMM>
        <transmissionInPerCent>100.0</transmissionInPerCent>
        <sampleName>Hr: 1 Pos: 1</sampleName>
        <visitPath>/tmp/gda/i03/data/2023/cm33866-3</visitPath>
        <comment></comment>
        <centringMode>XRAY</centringMode>
        <experimentType>SAD</experimentType>
        <doseProtocol>target_exposure</doseProtocol>
        <totalNumberOfImages>100</totalNumberOfImages>
        <fileNameTemplate>/tmp/gda/i03/data/2023/cm33866-3/ssa/sdasda_0_%05d.%s</fileNameTemplate>
        <dnaFileNameTemplate>sdasda_0_#####.%s</dnaFileNameTemplate>
        <dnaFilePrefix>sdasda_0_</dnaFilePrefix>
        <dnaFileDir>/tmp/gda/i03/data/2023/cm33866-3/ssa</dnaFileDir>
        <hasTransmission>true</hasTransmission>
        <beamstopPosition>0</beamstopPosition>
        <chi>0.0</chi>
        <phi>0.0</phi>
        <omegaDelta>0.0</omegaDelta>
        <axisChoice>Omega</axisChoice>
        <beamProfile/>
    </extendedCollectRequest>
    <extendedCollectRequest>
        <collect_request>
            <fileinfo>
                <directory>/tmp/gda/i03/data/2023/cm33866-3/ssa</directory>
                <prefix>sdasda</prefix>
            </fileinfo>
            <oscillation_sequence>
                <start>0.0</start>
                <range>0.1</range>
                <number_of_images>100</number_of_images>
                <overlap>0.0</overlap>
                <exposure_time>0.01</exposure_time>
                <start_image_number>1</start_image_number>
                <number_of_passes>1</number_of_passes>
            </oscillation_sequence>
            <wavelength>0.97625</wavelength>
            <resolution>
                <upper>1.9780068845283907</upper>
            </resolution>
            <sample_reference>
                <code></code>
                <container_reference>35</container_reference>
                <sample_location>2</sample_location>
                <blSampleId>4288613</blSampleId>
            </sample_reference>
        </collect_request>
        <runNumber>0</runNumber>
        <sampleDetectorDistanceInMM>289.3</sampleDetectorDistanceInMM>
        <transmissionInPerCent>100.0</transmissionInPerCent>
        <sampleName>b2</sampleName>
        <visitPath>/tmp/gda/i03/data/2023/cm33866-3</visitPath>
        <comment></comment>
        <centringMode>XRAY</centringMode>
        <experimentType>SAD</experimentType>
        <doseProtocol>target_exposure</doseProtocol>
        <totalNumberOfImages>100</totalNumberOfImages>
        <fileNameTemplate>/tmp/gda/i03/data/2023/cm33866-3/ssa/sdasda_0_%05d.%s</fileNameTemplate>
        <dnaFileNameTemplate>sdasda_0_#####.%s</dnaFileNameTemplate>
        <dnaFilePrefix>sdasda_0_</dnaFilePrefix>
        <dnaFileDir>/tmp/gda/i03/data/2023/cm33866-3/ssa</dnaFileDir>
        <hasTransmission>true</hasTransmission>
        <beamstopPosition>0</beamstopPosition>
        <chi>0.0</chi>
        <phi>0.0</phi>
        <omegaDelta>0.0</omegaDelta>
        <axisChoice>Omega</axisChoice>
        <beamProfile/>
    </extendedCollectRequest>
</ExtendedCollectRequests>

4. With no s03 running start artemis with ./run_artemis.sh --skip-startup-connection
5. Queue both collections in GDA, confirm the first grid scan fails catastrophically and the second data collection does not happen
6. Modify full_grid_scan.py to have:

def create_devices():
    pass

def get_plan(
    parameters: GridScanWithEdgeDetectInternalParameters,
    subscriptions: FGSCallbackCollection,
    oav_param_files: dict = OAV_CONFIG_FILE_DEFAULTS,
) -> Callable:
    """
    A plan which combines the collection of snapshots from the OAV and the determination
    of the grid dimensions to use for the following grid scan.
    """
    from artemis.exceptions import WarningException

    raise WarningException("Don't stop!")

8. Restart Artemis
9. Queue the collections up again and confirm that you get the Don't Stop message in the GDA log but both collections happen

[codecov]

Codecov Report

Merging #723 (e9fc471) into main (08e6505) will increase coverage by 0.56%.
The diff coverage is 97.13%.

❗ Current head e9fc471 differs from pull request most recent head 50fb0ff. Consider uploading reports for the commit 50fb0ff to get more accurate results

@@            Coverage Diff             @@
##             main     #723      +/-   ##
==========================================
+ Coverage   93.95%   94.51%   +0.56%     
==========================================
  Files          35       37       +2     
  Lines        1721     1806      +85     
==========================================
+ Hits         1617     1707      +90     
+ Misses        104       99       -5     

Impacted Files	Coverage Δ
src/artemis/parameters/constants.py	100.00% <ø> (ø)
...plan_specific/grid_scan_with_edge_detect_params.py	95.12% <50.00%> (-4.88%)	⬇️
...emis/experiment_plans/optimise_attenuation_plan.py	94.44% <94.44%> (ø)
...is/parameters/plan_specific/fgs_internal_params.py	98.33% <96.96%> (-1.67%)	⬇️
src/artemis/__main__.py	91.86% <100.00%> (+3.35%)	⬆️
src/artemis/device_setup_plans/setup_zebra.py	100.00% <100.00%> (ø)
src/artemis/experiment_plans/full_grid_scan.py	100.00% <100.00%> (ø)
...rtemis/experiment_plans/oav_grid_detection_plan.py	100.00% <100.00%> (ø)
src/artemis/experiment_plans/rotation_scan_plan.py	100.00% <100.00%> (ø)
...ternal_interaction/callbacks/fgs/nexus_callback.py	100.00% <100.00%> (ø)
... and 7 more

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

[noemifrisina]

When running unit tests, currently getting the following error on the parameters:

ERROR  - pydantic.error_wrappers.ValidationError: 1 validation error for DetectorParams

I think it's because of the dodal version which is now set to an old commit. As the last merge main is from a couple of weeks ago and there's no dodal PR linked to this one, merging main again in the branch should fix it?

[noemifrisina]

Units tests are now okay, thanks.

However, it doesn't get to the Don't stop message now fails when running from GDA with the error

Artemis failed to start with message: <ValidationError: "'2.0.0' was expected"> <class 'artemis_gridscan.artemis_grid_scan.ArtemisFailedException'> Artemis failed to start with message: <ValidationError: "'2.0.0' was expected">

It looks like it doesn't find the correct parameter version

[noemifrisina]

Everything now works as expected, thank you.