Add support for Enum types to aiida.orm.utils.serialize
#5218
Conversation
Enum's are often used data types that currently cannot be used in process inputs, even if the ports are `non_db`, because even the `non_db` inputs are serialized as part of the entire `Process` instance that is serialized when a checkpoint is created. The custom YAML serializer and deserializer that are defined in the `aiida.orm.utils.serialize` module that are used for serializing process instances and their inputs, are updated with a dumper and a loader for `Enum` instances. The full class identifier is generated from the default object loader provided by `plumpy` and concatenated by the `|` sign and the enum's values, it is represented as a YAML scalar. The loader uses the same object loader to load the enum class from the identifier string and reconstruct the enum instance from the serialized value. The original tests were written in `tests/common/test_serialize.py` and are moved to `tests/orm/utils/test_serialize.py` to conform with the standard to mirror the package hierarchy. The tests are converted to `pytest` style and a new test is added by performing a round trip of an enum instance (de)serialization.
|
@giovannipizzi @bosonie this should provide what we need as discussed. @chrisjsewell @ramirezfranciscof feel free to review if you'd like. |
Codecov Report
@@ Coverage Diff @@
## develop #5218 +/- ##
===========================================
- Coverage 81.22% 81.21% -0.01%
===========================================
Files 532 532
Lines 37344 37357 +13
===========================================
+ Hits 30328 30334 +6
- Misses 7016 7023 +7
Flags with carried forward coverage won't be shown. Click here to find out more.
Continue to review full report at Codecov.
|
There was a problem hiding this comment.
Ok with me thanks! Just to double check two things:
- this means that one needs to be able to import the Enum python declaration, right? (I think it's OK
- This does not yet allow to store this in the DB as one of the inputs when
non_db=False? If so, would it be possible to store the string value in the DB in this case, or there are issues with this?
Yes, otherwise the deserialization will fail. I think there is no way around this and is an acceptable limitation.
It does indeed not, but in general we do not allow to store arbitrary inputs in the database. All inputs have to be nodes so they can be properly linked up. The only exception are the |
|
Wouldn't it be useful to also have a way (as an alternative) to store an input as a string (Str), but with automatic validation coming from the values of an Enum? This would be useful to store the inputs of the common workflows, for instance (e.g. the ElectronicType) |
|
Might be useful indeed. Would be multiple ways to go about it. Maybe most straightforward would be to implement the |
Fix #5217
Enum's are often used data types that currently cannot be used in
process inputs, even if the ports are
non_db, because even thenon_dbinputs are serialized as part of the entireProcessinstancethat is serialized when a checkpoint is created.
The custom YAML serializer and deserializer that are defined in the
aiida.orm.utils.serializemodule that are used for serializing processinstances and their inputs, are updated with a dumper and a loader for
Enuminstances. The full class identifier is generated from thedefault object loader provided by
plumpyand concatenated by the|sign and the enum's values, it is represented as a YAML scalar.
The loader uses the same object loader to load the enum class from the
identifier string and reconstruct the enum instance from the serialized
value.
The original tests were written in
tests/common/test_serialize.pyandare moved to
tests/orm/utils/test_serialize.pyto conform with thestandard to mirror the package hierarchy. The tests are converted to
pyteststyle and a new test is added by performing a round trip of anenum instance (de)serialization.