Mesh generation utils

[FedeClaudi]

Two main changes:

in volume_utils create_masked_array facilitates the creation of binary mask arrays for surface extraction

in mesh_tuils extract_mesh_from_mask automates the extraction of clean surface mesh with marching cubes, morphological transformations and vtkplotter magic.

[FedeClaudi]

With some parallel processing:
Creating meshes for 141 structures took: 187.711s

Main changes:

• mesh_utils.extract_mesh_from_mask is faster
• volume_utils.extract_volume_surface has been moved to mesh_utils.extract_mesh_from_mask_fast and improved
• added parallel processing for mesh extraction in humanatlas.py
• added 'mesh_utils.inspect_meshses_folder` to quickly inspect all .obj files in a folder

!!! mesh_utils.extract_mesh_from_mask limitation:
while mesh_utils.extract_mesh_from_mask very well to create better root meshes (#21) it sometimes fails with small structures made of two separate volumes (e.g. left and right amygdala). In this case the marching cubes algorithm sometimes only finds one of the two. However for this kind of structures, mesh_utils.extract_mesh_from_mask_fast is preferable anyways.

[FedeClaudi]

#TODO: add methods to compare annotate volumetric data with meshes created during atlas generation to asses the quality of the meshes' fit to the original data

[FedeClaudi]

merged extract_mesh_from_mask_fast and extract_mesh_from_mask into extract_mesh_from_mask. The default behaviour is that of extract_mesh_from_mask_fast which is more accurate and faster. Using the morphological transformations the resulting meshes are still good.

[vigji]

#TODO: add methods to compare annotate volumetric data with meshes created during atlas generation to asses the quality of the meshes' fit to the original data

This could be handled simply interpolating back stack from meshes and then compare if something has disappeared and if too many voxels are misassigned

[FedeClaudi]

yeah, that would be good.

Meanwhile I've added this viewer: #24 (comment)

[FedeClaudi]

@adamltyson does your code generate a mesh for every structure? With mine there are some for which I can't, I think it's mostly because the annotated volume is annotate at a fairly high hierarchical level so all the children of the regions in the atlas I can't generate a mesh for, but maybe I'm doing something else wrong.

I've merged your code on my branch as well btw.

[adamltyson]

does your code generate a mesh for every structure?

Yeah, it extracts every region, but I haven't fully tested it. Might be being dumb.

[FedeClaudi]

So one of the problems with the human brain atlas is that the regions that are represented in the annotated volume are fairly high in the structure tree ('True' below):

so any region that is children to these there's not much we can do about.

[vigji]

If the problem is to small meshes maybe we could default a sphere of small radius as mesh?

[FedeClaudi]

No the problem is that if, say, one of the regions that is in the annotated volume has two children, A and B, I don't now which of the region's voxels belong to A and which to B. So I can't create a mesh for the subregions.

I ended pruning the tree to remove structures for which we can't provide a mesh, though not sure what we prefer to do in these cases.

By the way, the tests fail because of this:

E       FileNotFoundError: [Errno 2] No such file or directory: '/home/travis/.brainglobe/test/metadata.json'

I'll try uploading a version of the test atlas with metadata.json instead of atlas_metadata.json.

[FedeClaudi]

@vigji and @adamltyson this is failing because of something wrong on the windows build, but it's passing all tests locally on a Window's machine.

I've made a bunch of changes so I'd suggest merging and then trying to figure out what's causing the failures.

[vigji]

Sorry for leaving this aside for a bit. Yes, it looks like a vtkplotter problem. Ideally though we should not get to ask for it at all when testing stuff in the core, yet it's invoked.

I think that the problem is having the description in the script_utils - as it totally makes sense to have them available in the whole package, just maybe move it into the brainatlas_api module would fix this. Then we can clean up tests in a separate branch

[vigji]

No the problem is that if, say, one of the regions that is in the annotated volume has two children, A and B, I don't now which of the region's voxels belong to A and which to B. So I can't create a mesh for the subregions.

So there are children that are not annotated in the annotation stack?

I ended pruning the tree to remove structures for which we can't provide a mesh, though not sure what we prefer to do in these cases.

If there's no voxels assigned to a region, I would prune it out from the tree. We can have a warning about that.

By the way, the tests fail because of this:

E       FileNotFoundError: [Errno 2] No such file or directory: '/home/travis/.brainglobe/test/metadata.json'

I'll try uploading a version of the test atlas with metadata.json instead of atlas_metadata.json.

Sorry, this was my mess for multiple versions.

This is why if you and @adamltyson agree I would have the version tag not specified in the scrip on a -per-brain fashion but in the descriptors together with the rest - this might still raise problems while we all work simultaneously but should give more stability later - what would you think?

[FedeClaudi]

If there's no voxels assigned to a region, I would prune it out from the tree. We can have a warning about that.

Yeah I pruned them.

This is why if you and @adamltyson agree I would have the version tag not specified in the scrip on a -per-brain fashion but in the descriptors together with the rest

To fix the errors I've removed version from name but I've kept it as another attribute in addition to atlas_name. Happy to find other solutions, but I'd suggest merging this first to make sure everything is updated. You might also want to use the methods in mesh_utils to make a better root for the fish atlas, it worked well for the human atlas.

[vigji]

Ok, great - I will merge it and fix it