[nixio] Use Neo object names in NIX file #551
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This option triggers a check to determine if the Neo names satisfy the uniqueness requirement for NIX and if the check passes, the Neo objects are annotated with their own name (as nix_name) to make the writer use that name in NIX.
|
Thanks for the improvements. I tested it and it works. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Last year I changed the NixIO to use UUIDs as names for objects as a way to get around a couple of issues. Issue #311 is where the discussion happened and #331 is the relevant PR where this was implemented.
Some time later, there was a discussion on the mailing list regarding the write performance but, as a side topic, the issue of the naming also came up. In short, with the current naming scheme, the NIX/HDF5 file that is generated has names that make it difficult to work with and can be a hindrance (and an annoyance) for people who purposefully name their objects in order to be able to navigate a file.
This PR adds a new feature which somewhat backtracks from the design decision of automatically naming objects. The
write_all_blocks()andwrite_block()methods now accept an optional, boolean argument,use_obj_names(false, by default). If enabled, the objects in the NIX file will use the names of the corresponding Neo objects. Before writing, however, the whole Neo object tree is checked for conflicts, based on the requirements of NIX and HDF5. If a conflict is found, a ValueError is raised and no objects are written.I felt it was important to not check the names while writing and instead check names beforehand. This avoids potential issues with partially writing data with names that later turn out to have conflicts. It's also good for avoiding situations where a write will fail half way through the saving of a large file that could take minutes (or hours). I've clocked the name checking function and even with thousands of objects (the saving of which took over 20 mins), it only needs a few seconds to run.
Tests for the new functionality are also included.