Cleaning up core function and structures i/o

[vigji]

I started refactoring the core class as per #34. I realised that the AllenSDK stuff for the structure is quite useless for most of our cases, and it's very straightforward to implement ourselves what we need for handling the structures.

I added 2 classes for this - the first one I am not very sure about, the second is surely useful.

• The first one provide some minimal mesh API with caching of the mesh through meshio. It's basically a dictionary that stores the value of the mesh of a region after queried for it. Having this here can't do any harm, but would depend on whether we want to support mesh loading in this package or not.
• The second one is a neat way of doing all the region-related queries by either a string (acronym), an integer (id), or a list/tuple with any combination of the above:

structures = StructureDict(structures_list)
[structures[997], structures["grey"]] == structures[["root", 8]]

In this way all the Atlas.get_stuff_from_region(args) methods will just wrap the query to the Atlas.structures[args]["stuff"] and they will get for free the possibility of accepting any of these arguments without ifs.

I think that we most of the stuff that was in the AllenSDK classes won't be missed, and refactoring the useful part in this dictionary would be trivial.

[FedeClaudi]

Hey I'm in favor ditching the Allen SDK as I dislike how it's written, I find it very confusing.

The StructureDict class seems to handle the queries very nicely, I like it. I would keep some of the methods based on treelib that I had added to StructuresTree though, stuff like listing all the structures and printing the structure tree. treelib has a lot of useful utils to find parents/children of a given node, to get the subtree from a given node, leaf nodes etc that I suggest we use to handle similar methods.
Perhaps these methods can be implemented in a dedicated StructuersTree class of our own, maybe subclassing treelib.tree.

What do you think? I'm happy to add this in.

---

p.s.: I think that the tests are failing becaus meshio is not listed in setup.py

[vigji]

Yes, thanks, I will fix the test together with some more improvements!

I would definitively keep those functions that you added in the old class and i don't mind that light dependency. I would just wonder whether they make more sense to have as a function that takes the StructuresDict method or as a method of the class itself. What are the usage cases apart for printing the tree?

[FedeClaudi]

find parents/children of a given node, to get the subtree from a given node, leaf nodes etc

[vigji]

Ok - because parents / children are one-liners to find with the path entry of the dict right now (structure["structure_id_path"] is the parents, [s for s in structures_list if i in s["structure_id_path"]]), and maybe a couple more conditionals for check is something is a leaf..

Not that I mind the dependency, and for sure it's good for the printing, but if stuff like the above can be handled easily within the class it would be better to avoid redundancies of data representations

[FedeClaudi]

yeah but stuff like

• subtree: get all descendants of a region but keeping their hierarchical structure
• sibling
• root to node path

is implemented in treelib. Honestly I don;t know if any of this is necessary/important. I guess more generally I find treelib a very intuitive way to think about and manipulate the hierarchy, but if we don't need to use it we don't need to use it.

[vigji]

In principle I also thought that a truly tree-like structure was good to have, but the more I try stuff the more I realise that the crosses we will make are almost always good to do on a plain list (to parse stuff per regions). Let see how it progresses.

I changed stuff not to deal with the iteration in the StructuresDict. In real cases, we generally don't want to collect entire lists of dictionaries, but single elements (e.g. name) for a list of regions, in which case syntax would require in any case to parse out the key. So I just made a dedicated method in the class to parse one or multiple items there. The double query by id or acronym will remain in the StructuresDict.

[vigji]

Ok, if you guys can have a look, I like this current version, I would test it out in real applications to see how convenient it is.

Each structure is represented by a dictionary-like class that manages lazy loading of meshes; the structures tree is represented (in a plain version, not as tree) by a dictionary that can be queried either by structure id or acronym.

finally, the Atlas class has a structures attribute that can be used as a dictionary of dictionary (for queries like {k: s["rgb_triplet"] for k, s in self.structures.items() }); or through Atlas methods such as Atlas.mesh_from_structure(structure_id). In the latter case, there is a single method Atlas._get_from_structure(structure_id, key) that handles all the queries to the structures dictionary and support either single ids or lists of ids.

@FedeClaudi I moved the tree generation to a function in structure_tree_util. Do we want it as a method to define the print(StructuresDict) behaviour? The current implementation is printing the list of "{id} \t- {acronym}\n"

[FedeClaudi]

Perhaps the function that prints the whole tree is more useful for print(StructuresDict)?
Is that function also in the same script?

It all looks very good, will test tomorrow

[adamltyson]

Looks good to me. Don't have time to test properly at the moment though.

[vigji]

Ok, maybe we could merge and you can go back and potentially insult me later ? 😃

[adamltyson]

Ok, maybe we could merge and you can go back and potentially insult me later?

Sounds good to me!

[FedeClaudi]

it looks fine, let's merge