Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(protocol-engine): implement well & labware's well accessor methods #8151

Merged
merged 17 commits into from
Jul 27, 2021

Conversation

sanni-t
Copy link
Member

@sanni-t sanni-t commented Jul 22, 2021

Overview

Adds well access methods to Labware class

Changelog

  • Implemented wells(), wells_by_name(), rows(), columns(), rows_by_name(), columns_by_name()
  • Labware now caches the labware definition as well as wells_by_name(), rows_by_name(), columns_by_name()
  • Implemented a few Well property methods
  • Added & updated tests

Review requests

  • The caching of labware definition & well properties during labware initialization makes stubbing Labware a bit more involved but still feels right to me since some of the well access methods are a bit expensive (potentially more for labware with lots of wells, esp when accessed many times in a protocol). But open to other thoughts.
    Update: After much discussion, decided caching is the right approach but caching in __init__ or using lru_cache isn't because the former makes test fixtures unnecessarily involved while lru_cache creates problems in testing if a Labware's id is not consistent (as the method refers to self). So, decided to make all cached methods use lazy properties which solves all testing issues.

  • Related to above, the protocol context load_labware tests now fail because we don't mock out Labware class. What's the best approach to refactoring these tests? Not an issue anymore because of lazy properties.

Risk assessment

Medium impact on PE. None on non-P.E code

@codecov
Copy link

codecov bot commented Jul 22, 2021

Codecov Report

Merging #8151 (c5ac262) into edge (73bd30c) will increase coverage by 0.04%.
The diff coverage is n/a.

Impacted file tree graph

@@            Coverage Diff             @@
##             edge    #8151      +/-   ##
==========================================
+ Coverage   87.20%   87.24%   +0.04%     
==========================================
  Files         445      427      -18     
  Lines       22697    22501     -196     
==========================================
- Hits        19793    19632     -161     
+ Misses       2904     2869      -35     
Impacted Files Coverage Δ
opentrons/protocol_engine/state/labware.py 100.00% <0.00%> (ø)
notify-server/notify_server/__init__.py
.../notify_server/models/hardware_event/door_state.py
notify-server/notify_server/models/payload_type.py
...er/notify_server/models/hardware_event/__init__.py
notify-server/notify_server/main.py
notify-server/notify_server/server/__init__.py
notify-server/notify_server/clients/serdes.py
notify-server/notify_server/network/connection.py
notify-server/notify_server/logging.py
... and 11 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c47f712...c5ac262. Read the comment docs.

}
self._well_grid = self._get_well_grid()
else:
raise Exception("Labware definition not found. Definition required for "
Copy link
Member Author

@sanni-t sanni-t Jul 22, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what kind of Exception do we want here? Make a custom LabwareDefinitionNotFound error?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

labware.get_labware_definition (which calls labware.get_labware_data_by_id and labware.get_definition_by_uri) will already raise if the labware ID or labware definition isn't found. For example:

def get_definition_by_uri(self, uri: LabwareUri) -> LabwareDefinition:
"""Get the labware definition matching loadName namespace and version."""
try:
return self._state.labware_definitions_by_uri[uri]
except KeyError:
raise errors.LabwareDefinitionDoesNotExistError(
f"Labware definition for matching {uri} not found."
)

@sanni-t sanni-t added this to the CPX Sprint 37 milestone Jul 22, 2021
@sanni-t sanni-t added robot-svcs Falls under the purview of the Robot Services squad (formerly CPX, Core Platform Experience). protocol-engine Ticket related to the Protocol Engine project and associated HTTP APIs labels Jul 22, 2021
Copy link
Contributor

@mcous mcous left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some early initial feedback

api/src/opentrons/protocol_api_experimental/labware.py Outdated Show resolved Hide resolved
}
self._well_grid = self._get_well_grid()
else:
raise Exception("Labware definition not found. Definition required for "
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

labware.get_labware_definition (which calls labware.get_labware_data_by_id and labware.get_definition_by_uri) will already raise if the labware ID or labware definition isn't found. For example:

def get_definition_by_uri(self, uri: LabwareUri) -> LabwareDefinition:
"""Get the labware definition matching loadName namespace and version."""
try:
return self._state.labware_definitions_by_uri[uri]
except KeyError:
raise errors.LabwareDefinitionDoesNotExistError(
f"Labware definition for matching {uri} not found."
)

api/src/opentrons/protocol_api_experimental/labware.py Outdated Show resolved Hide resolved
api/src/opentrons/protocol_api_experimental/labware.py Outdated Show resolved Hide resolved
api/src/opentrons/protocol_api_experimental/well.py Outdated Show resolved Hide resolved
for well_name in col:
match = pattern.match(well_name)
assert match, f"Well name did not match pattern {pattern}"
wells_by_rows[match.group(1)].append(well_name)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, I couldn't find a way to avoid doing pattern matching for rows because we refer to rows with row letters. So, in a situation where wells.ordering deviates from our expected naming pattern of letter+number, e.g: [['ab', 'cd'], ['wx', 'yz']], the columns would be parsed easily with a {'1': ['ab', 'cd'], '2': ['wx', 'yz']}, but for rows, a {'A': ['ab', 'wx'], 'B': ['cd', 'yz']} wouldn't make sense, and, doing something like {1: ['ab', 'wx'], 2: ['cd', 'yz']} would totally alter the behavior of rows_by_name()

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, to me, it seems like we need to stick to the r'^([A-Z]+)([0-9]+)$' pattern and expect the well definition to have wells named according to that pattern.
In which case, it feels a bit weird to have different ways of parsing rows and columns but I can get used to it, esp since column parsing like this (using well.ordering instead of pattern matching) is much easier.

Copy link
Contributor

@mcous mcous Jul 27, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we maybe add a TODO to evaluate if there are any differences we can find between ordering and well names in our existing definition?


Edit: I re-read this implementation more fully, and I think this is a good compromise for now

@sanni-t sanni-t marked this pull request as ready for review July 27, 2021 16:59
@sanni-t sanni-t requested a review from a team as a code owner July 27, 2021 16:59
@@ -31,6 +31,9 @@ def __init__(
self._engine_client = engine_client
self._labware = labware
self._well_name = well_name
self._well_definition = self._engine_client.state.labware.get_well_definition(
Copy link
Member Author

@sanni-t sanni-t Jul 27, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note from call:
Will be keeping this cached in __init__ as is for now but will probably need to be changed to a lazy property once we start adding more tests that access wells.

Copy link
Contributor

@mcous mcous left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🪂

for well_name in col:
match = pattern.match(well_name)
assert match, f"Well name did not match pattern {pattern}"
wells_by_rows[match.group(1)].append(well_name)
Copy link
Contributor

@mcous mcous Jul 27, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we maybe add a TODO to evaluate if there are any differences we can find between ordering and well names in our existing definition?


Edit: I re-read this implementation more fully, and I think this is a good compromise for now

"""Get well rows."""
definition = self.get_labware_definition(labware_id=labware_id)
wells_by_rows = defaultdict(list)
pattern = re.compile(WELL_NAME_PATTERN, re.X)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this compile be at the module level?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since the regex's used only in this function, I like it being contained in here.

for col in definition.ordering:
for well_name in col:
match = pattern.match(well_name)
assert match, f"Well name did not match pattern {pattern}"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok to leave around for now, but this assert makes me feel a little weird

Copy link
Contributor

@SyntaxColoring SyntaxColoring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed in a call with @mcous and @sanni-t.

api/tests/opentrons/protocol_engine/conftest.py Outdated Show resolved Hide resolved
Co-authored-by: Max Marrone <max@opentrons.com>
@sanni-t sanni-t merged commit bff281a into edge Jul 27, 2021
@sanni-t sanni-t deleted the pe-add_well_implementation branch July 27, 2021 19:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
protocol-engine Ticket related to the Protocol Engine project and associated HTTP APIs robot-svcs Falls under the purview of the Robot Services squad (formerly CPX, Core Platform Experience).
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants