Analyses Generation and Snapshot Testing

Setup

1. Follow the instructions in DEV_SETUP.md
2. cd analyses-snapshot-testing
3. use pyenv to install python 3.12 and set it as the local python version for this directory
4. make setup
5. Have docker installed and ready

Concepts

• If working locally the branch you have checked out is the test code/snapshots you are working with.
  • In CI this is the SNAPSHOT_REF. This is the branch or tag of the test code/snapshots that analyses generated will be compared to.
• The ANALYSIS_REF is the branch or tag that you want analyses generated from.

Running the tests locally

• Compare the current branch snapshots to analyses generated from the edge branch
  • make build-opentrons-analysis ANALYSIS_REF=edge this builds a docker image named and tagged opentrons-analysis:edge
    • this pulls the latest edge every time it builds!
  • make snapshot-test ANALYSIS_REF=edge
    • This runs the test. The test:
      • Spins up a container from the opentrons-analysis:edge image. ANALYSIS_REF=edge specifies the image to use.
      • Analyses as .json files are generated for all protocols defined in protocols.py and protocols_with_overrides.py
      • the test compares the generated analyses to the snapshots in the ./tests/snapshots/ directory

Updating the snapshots

• Assuming you have already built the opentrons-analysis:edge image
• make snapshot-test-update ANALYSIS_REF=edge
  • This will update the snapshots in the ./tests/snapshots/ directory with the analyses generated from the edge branch

Running the tests against specific protocols

We are omitting ANALYSIS_REF=edge because we can, it is the default in the Makefile

• make snapshot-test PROTOCOL_NAMES=Flex_S_v2_19_Illumina_DNA_PCR_Free OVERRIDE_PROTOCOL_NAMES=none
• make snapshot-test PROTOCOL_NAMES=none OVERRIDE_PROTOCOL_NAMES=Flex_X_v2_18_NO_PIPETTES_Overrides_BadTypesInRTP
• make snapshot-test PROTOCOL_NAMES="Flex_S_v2_19_Illumina_DNA_PCR_Free,OT2_S_v2_18_P300M_P20S_HS_TC_TM_SmokeTestV3" OVERRIDE_PROTOCOL_NAMES=none