Skip to content

Files

Latest commit

 

History

History
 
 

analyses-snapshot-testing

README.md

Analyses Generation and Snapshot Testing

Setup

  1. Follow the instructions in DEV_SETUP.md
  2. cd analyses-snapshot-testing
  3. use pyenv to install python 3.12 and set it as the local python version for this directory
  4. make setup
  5. Have docker installed and ready

Concepts

  • If working locally the branch you have checked out is the test code/snapshots you are working with.
    • In CI this is the SNAPSHOT_REF. This is the branch or tag of the test code/snapshots that analyses generated will be compared to.
  • The ANALYSIS_REF is the branch or tag that you want analyses generated from.

Build the opentrons-analysis image

This ALWAYS gets the remote code pushed to Opentrons/opentrons for the specified ANALYSIS_REF

  • build the base image
    • make build-base-image
  • build the opentrons-analysis image
    • make build-opentrons-analysis ANALYSIS_REF=release

Running the tests locally

  • Compare the current branch snapshots to analyses generated from the edge branch
    • make build-opentrons-analysis ANALYSIS_REF=edge this builds a docker image named and tagged opentrons-analysis:edge
      • this pulls the latest edge every time it builds!
    • make snapshot-test ANALYSIS_REF=edge
      • This runs the test. The test:
        • Spins up a container from the opentrons-analysis:edge image. ANALYSIS_REF=edge specifies the image to use.
        • Analyses as .json files are generated for all protocols defined in protocols.py and protocols_with_overrides.py
        • the test compares the generated analyses to the snapshots in the ./tests/snapshots/ directory

Updating the snapshots

  • Assuming you have already built the opentrons-analysis:edge image
  • make snapshot-test-update ANALYSIS_REF=edge
    • This will update the snapshots in the ./tests/snapshots/ directory with the analyses generated from the edge branch

Running the tests against specific protocols

We are omitting ANALYSIS_REF=edge because we can, it is the default in the Makefile

  • make snapshot-test PROTOCOL_NAMES=Flex_S_v2_19_Illumina_DNA_PCR_Free OVERRIDE_PROTOCOL_NAMES=none
  • make snapshot-test PROTOCOL_NAMES=none OVERRIDE_PROTOCOL_NAMES=Flex_X_v2_18_NO_PIPETTES_Overrides_BadTypesInRTP
  • make snapshot-test PROTOCOL_NAMES="Flex_S_v2_19_Illumina_DNA_PCR_Free,OT2_S_v2_18_P300M_P20S_HS_TC_TM_SmokeTestV3" OVERRIDE_PROTOCOL_NAMES=none

Running a Flex just like make -C robot-server dev-flex

This ALWAYS gets the remote code pushed to Opentrons/opentrons for the specified OPENTRONS_VERSION

cd analyses-snapshot-testing \
&& make build-base-image \
&& make build-rs OPENTRONS_VERSION=release \
&& make run-rs OPENTRONS_VERSION=release`

Default OPENTRONS_VERSION=edge in the Makefile so you can omit it if you want latest edge

cd analyses-snapshot-testing \
&& make build-base-image \
&& make build-rs \
&& make run-rs

Running the Analyses Battery against your local code

This copies in your local code to the container and runs the analyses battery against it.

  1. make build-base-image
  2. make build-local
  3. make local-snapshot-test

You have the option to specify one or many protocols to run the analyses on. This is also described above Running the tests against specific protocols

  • make local-snapshot-test PROTOCOL_NAMES=Flex_S_v2_19_Illumina_DNA_PCR_Free OVERRIDE_PROTOCOL_NAMES=none