Questions on obj file format conversion and nrrd file format support

[manoaman]

Hi Will,

More related to generating mesh in precomputed file format workflow Questions on understanding the workflow for generating mesh "precomputed" data #406, does zmesh support converting a collection of obj file format to one or more precomputed files? Basically looking to do the other way around of the following code. Also, is nrrd file format supported to generate mesh?

# Extremely common obj format
with open('iconic_doge.obj', 'wb') as f:
  f,write(mesh.to_obj())

# Common binary format
with open('iconic_doge.ply', 'wb') as f:
  f,write(mesh.to_ply())

# Neuroglancer Precomputed format
with open('10001001:0', 'wb') as f:
  f.write(mesh.to_precomputed())

Thank you,
-m

[william-silversmith]

Hi m,

Yes it does support converting obj. The CloudVolume Mesh object has the same properties too.

from zmesh import Mesh
obj = load_obj() # as string
mesh = Mesh.from_obj(obj)
binary = mesh.to_precomputed()

We also support the binary ply format as well.

We don't currently have support for the NRRD file format. I personally haven't used it though I've seen other people in the field using it. I googled around and saw that there's a Python library called pynrrd which might be useful for you. Once you have a zmesh Mesh object (or a CloudVolume Mesh object), you can access the underlying numpy arrays easily:

mesh.vertices
mesh.edges
mesh.normals

[manoaman]

Thank you Will, sorry it took me awhile to get to the datasets. Forgive me for my slow reply.

Two questions I have. If I understand correctly, once I save binary object to a file along with info file, would that be good enough to visualize in Neuroglancer? Or do I need to pass binary object to Igneous for mesh tasks? For example, further process with create_transfer_tasks and create_meshing_tasks and pass this binary as source?

Thank you,
-m

[william-silversmith]

Hi m,

Once you save the info file and the mesh in a file named $SEGID:0:$ARBITRARY_STRING and then run a meshmanifest task that simply generates a file $SEGID:0 that contains { "fragments": [ "$SEGID:0:$ARBITRARY_STRING" ] } you'll be able to visualize it in Neuroglancer.

Will

[manoaman]

Hi Will,

I'm trying to understand the precomputed file structure with mesh data and I'm having problems reading files into CloudVolume or zmesh, and convert them to wavefront obj files.

What I'm trying to accomplish is to convert precomputed format to wavefront obj. Precomputed file passed Igeneous tasks and verified mesh with two segmentations in Neuroglancer.

1. zmesh approach

from cloudvolume import CloudVolume
from zmesh import Mesh

mesh = Mesh.from_precomputed('/precomputed/mesh_mip_0_err_40/65534:0')

TypeError: a bytes-like object is required, not 'str'

I think I am confused here what to provide as a binary file which corresponds to a segmentation in mesh. Which binary file should I be providing here?

2. CloudVolume approach

info = CloudVolume.create_new_info(  # 'image' or 'segmentation'
                                     # can pick any popular uint
                                     # other options: 'jpeg', 'compressed_segmentation' (req. uint32 or uint64)
                                     # X,Y,Z values in nanometers
                                     # values X,Y,Z values in voxels
                                     # rechunk of image X,Y,Z in voxels
                                     # X,Y,Z size in voxels
    num_channels=1,
    layer_type='segmentation',
    data_type='uint16',
    encoding='raw',
    resolution=[201, 201, 998],
    voxel_offset=[0, 0, 0],
    chunk_size=[64, 64, 64],
    volume_size=[2048, 2048, 96],
    )
vol = CloudVolume('file:///precomputed/mesh_mip_0_err_40', info=info)
vol.mesh.save([32767, 65534], file_format='obj')

ValueError: Segment ID(s) 65534, 32767 are missing corresponding mesh manifests.
Aborted.

CloudVolume does not seem to find segmentations. Am I pointing to the wrong directory or am I missing something in info file?

% ls -l ./precomputed/ | awk '{print $9}'

1608_1608_998
201_201_998
402_402_998
804_804_998
info
mesh_mip_0_err_40
provenance

----

% ls -l ./precomputed/mesh_mip_0_err_40/ | awk '{print $9}'

0-512_0-512_0-96.spatial.gz
0-512_1024-1536_0-96.spatial.gz
0-512_1536-2048_0-96.spatial.gz
0-512_512-1024_0-96.spatial.gz
1024-1536_0-512_0-96.spatial.gz
1024-1536_1024-1536_0-96.spatial.gz
1024-1536_1536-2048_0-96.spatial.gz
1024-1536_512-1024_0-96.spatial.gz
1536-2048_0-512_0-96.spatial.gz
1536-2048_1024-1536_0-96.spatial.gz
1536-2048_1536-2048_0-96.spatial.gz
1536-2048_512-1024_0-96.spatial.gz
32767:0
32767:0:0-512_0-512_0-96.gz
32767:0:0-512_1024-1536_0-96.gz
32767:0:0-512_1536-2048_0-96.gz
32767:0:0-512_512-1024_0-96.gz
32767:0:1024-1536_0-512_0-96.gz
32767:0:1024-1536_1024-1536_0-96.gz
32767:0:1024-1536_1536-2048_0-96.gz
32767:0:1024-1536_512-1024_0-96.gz
32767:0:1536-2048_0-512_0-96.gz
32767:0:1536-2048_1024-1536_0-96.gz
32767:0:1536-2048_1536-2048_0-96.gz
32767:0:1536-2048_512-1024_0-96.gz
32767:0:512-1024_0-512_0-96.gz
32767:0:512-1024_1024-1536_0-96.gz
32767:0:512-1024_1536-2048_0-96.gz
32767:0:512-1024_512-1024_0-96.gz
512-1024_0-512_0-96.spatial.gz
512-1024_1024-1536_0-96.spatial.gz
512-1024_1536-2048_0-96.spatial.gz
512-1024_512-1024_0-96.spatial.gz
65534:0
65534:0:0-512_0-512_0-96.gz
65534:0:0-512_1024-1536_0-96.gz
65534:0:0-512_1536-2048_0-96.gz
65534:0:0-512_512-1024_0-96.gz
65534:0:1024-1536_0-512_0-96.gz
65534:0:1024-1536_1024-1536_0-96.gz
65534:0:1024-1536_1536-2048_0-96.gz
65534:0:1024-1536_512-1024_0-96.gz
65534:0:1536-2048_0-512_0-96.gz
65534:0:1536-2048_1024-1536_0-96.gz
65534:0:1536-2048_1536-2048_0-96.gz
65534:0:1536-2048_512-1024_0-96.gz
65534:0:512-1024_1024-1536_0-96.gz
65534:0:512-1024_1536-2048_0-96.gz
65534:0:512-1024_512-1024_0-96.gz
info

Thanks,
-m

[william-silversmith]

Hi m,

You're really close! Try something more like this and make sure there are at least two directories in the path.

cv = CloudVolume("file:///some_dir/precomputed") # CloudVolume doesn't support single directory paths.... 
vol.mesh.save([32767, 65534], file_format='obj')

CV path issue: seung-lab/cloud-volume#391

You can also do:

cv = CloudVolume("file:///some_dir/precomputed") # CloudVolume doesn't support single directory paths.... 
m = vol.mesh.get(32767)
with open(...) as f:
    f.write(m[32767].to_obj())

Let me know if you need more tips!

[manoaman]

Hi Will, Hmm... I think I have at least two directories but still seem to get the same errors. Any thoughts?

Traceback (most recent call last):
  File "test_obj_convert.py", line 39, in <module>
    vol.mesh.save([32767, 65534], file_format='obj')
  File "/usr/local/var/pyenv/versions/cloudvolume_test/lib/python3.6/site-packages/cloudvolume/datasource/precomputed/mesh/unsharded.py", line 196, in save
    mesh = self.get(segids, fuse=True, remove_duplicate_vertices=True)
  File "/usr/local/var/pyenv/versions/cloudvolume_test/lib/python3.6/site-packages/cloudvolume/datasource/precomputed/mesh/unsharded.py", line 131, in get
    .format(missing)
ValueError: Segment ID(s) 65534, 32767 are missing corresponding mesh manifests.
Aborted.

Thanks,
-m

[manoaman]

Hi Will, when I print manifest_paths from the code, CloudVolume seems to be looking for these paths.

['mesh/32767:0', 'mesh/65534:0']

Although, I don't think I have mesh folder generated from Igenous. What I see is mesh_mip_0_err_40. Do I need to rename this folder to mesh?

[william-silversmith]

Yes, you can either rename the "mesh" property in the info file or rename the directory.

…

On Mon, Nov 30, 2020, 8:35 PM manoaman ***@***.***> wrote: Hi Will, when I print manifest_paths from the code, CloudVolume seems to be looking for these paths. ['mesh/32767:0', 'mesh/65534:0'] Although, I don't think I have mesh folder generated from Igenous. What I see is mesh_mip_0_err_40. Do I need to rename this folder to mesh? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#16 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AATGQSLDWU4J4TIEGIEEK2DSSRB5VANCNFSM4SZBIYJQ> .

[manoaman]

Great, that seems to be working! Thanks Will, -m

[william-silversmith]

Glad that helped!

[manoaman]

Hi Will,

When I generate precomputed mesh files, the top level mesh (root, 997.obj) of the Allen CCF annotation appear differently on Neuroglancer. Is this expected? I tried Igneous mesh task to process tiff split files from the original nrrd.

http://download.alleninstitute.org/informatics-archive/current-release/mouse_ccf/annotation/ccf_2017/

[william-silversmith]

This is not expected lol. Can you show me your processing steps in more detail? I'm not familiar with the Allen model format.

[manoaman]

Sure. It's been awhile since I processed the files, and this is before I started using igneous cli. (Sorry, not sure which version of the igneous I used at the time.) I believe it is pretty straight forward steps I took. The other segmentation meshes appear good to me on Neuroglancer. I just happen to realize only the root (997) come out differently.

Steps:

nrrd ---> tiff splits ---> CloudVolume (xy chunking, info population) ---> Igneous (xyz chunking, mesh task)

nrrd_tiff_split.txt

cloud_volume.txt

igneous_mesh_task.txt

[kpbhat25]

nrrd to obj(ply,stl) is there any python code for the same??

[manoaman]

@kpbhat25 I haven't tried nrrd to obj conversion. obj/ply files are here as far as I know. https://download.alleninstitute.org/informatics-archive/current-release/mouse_ccf/annotation/ccf_2017/structure_meshes/

[william-silversmith]

Hi guys, sorry I just don't have the bandwidth to help with this right now. My sincere apologies.

[manoaman]

Hi @william-silversmith any clue so far? Perhaps missing parameter during a mesh task?

[william-silversmith]

Hi m,

I took a quick look and noticed two things about the dataset. One, during ingest, the tiffs are uint32s, but the ingest code is uint16 which causes warnings to appear about integer overflow. I fixed that and looked at the data itself. The resulting mesh is pretty ugly, but this gap does appear to be a legit representation of the underlying mesh for ID 997. Are you absolutely sure that pretty mesh you showed above was really derived from annotation_50.nrrd?

[william-silversmith]

@kpbhat25 sorry it took me a while to get back to you too. You can follow m's code as a guide. Once you have a Neuroglancer volume generated with meshes, you can use CloudVolume to convert them to OBJ using data = mesh.to_obj() and then manually save data into a file:

with open("mymesh.obj", "wt") as f:
    f.write(mesh.to_obj())

There are probably some shortcut ways to do this too if your data fits in memory, but I don't really know your problem that well.

[manoaman]

Hi @william-silversmith

Thanks for catching the data type incompatibility, I missed that part. To be honest with you, I don't know if underlying mesh for 997 is derived from a 50 nrrd file. There are 10, 25, 100(um) resolution nrrd files in the same folder which I haven't checked yet. It is kind of strange to think that only this mesh appear differently though. Perhaps 997 mesh is intentionally added separately?

[william-silversmith]

The thing is the 977 mesh is an enclosing mesh, so if there was a downsampling operation, it could have thinned out and disappeared crucial information about the envelope. Why don't you give some of the other volumes a try and let me know how it goes? I'll be on vacation for the next 3 weeks though.

…

On Tue, Jan 17, 2023 at 4:43 PM manoaman ***@***.***> wrote: Hi @william-silversmith <https://github.com/william-silversmith> Thanks for catching the data type incompatibility, I missed that part. To be honest with you, I don't know if underlying mesh for 997 is derived from a 50 nrrd file. There are 10, 25, 100(um) resolution nrrd files in the same folder <https://download.alleninstitute.org/informatics-archive/current-release/mouse_ccf/annotation/ccf_2017/> which I haven't checked yet. It is kind of strange to think that only this mesh appear differently though. Perhaps 997 mesh is intentionally added separately? — Reply to this email directly, view it on GitHub <#16 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AATGQSKXV5AJTQMNLGUILF3WS4HANANCNFSM4SZBIYJQ> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[manoaman]

Interesting point. Let me give it a try with other volumes too just in case. One thing to verify on the "data type", is this simply editing the "info" file or should I be running CloudVolume/Igneous again for downstream chunking? And I suppose always configure info file with tiff's data type before running? e.g.) 8-bit Grayscale ---> uint8

Have a great vacation @william-silversmith !!

[william-silversmith]

Hi m,

I think you need to re-run CV/Igneous as the buffer read that translated the tiff files to numpy arrays was potentially corrupted. You also have to change the info file. Yes, you have to make sure the data types match every time.

Thanks for the well wishes and good luck!
Will

[manoaman]

Okay, I reprocessed 10, 25, 100 um images with uint32, and ran Igneous for all three images. Unfortunately, 997 (root) mesh all appeared to take the same form as 50um mesh, not identical to the whole brain mesh from the .obj file. Not sure where to go from here.. thoughts?

10um

25um

100um

[manoaman]

Hi @william-silversmith

I was wondering if could consolidate all the structure mesh files (.obj), and then write to a precomputed format. And then run igneous commands (igneous image xfer,igneous mesh forge, igenous mesh merge) to see the generated mesh again on the viewer instead of processing from the original nrrd file.

Now, I remember CloudVolume handles each individual .obj to a precomputed format. What would be a better approach to consolidate all these .obj files so that I can process in igneous?

f = open("997.obj", "r")
obj = f.read()
mesh = Mesh.from_obj(obj)
binary = mesh.to_precomputed()

or maybe pass numpy to CloudVolume and start from there?

import pywavefront
import trimesh
import numpy as np

# Load the OBJ files
objs = [pywavefront.Wavefront("path/to/file_{}.obj".format(i)) for i in range(10)]

# Convert each OBJ file to a voxelized representation
voxel_representations = [trimesh.voxel.voxelize(obj.vertices, obj.faces) for obj in objs]

# Combine the voxelized representations into a single volume
combined_volume = np.sum(voxel_representations, axis=0)

Thanks,
-m

[william-silversmith]

Hi m,

I think you're on the right track.

with open("997.obj", "rt") as f:
    obj = f.read()
with open("998.obj", "rt") as f:
    obj2 = f.read()
mesh = Mesh.from_obj(obj)
mesh2 = Mesh.from_obj(obj2)
m = Mesh.concatenate(mesh, mesh2)
m.segid = 1

cv = CloudVolume(...)
cv.mesh.put(m)

[manoaman]

Hi Will (@william-silversmith),

It seems like these .obj files contain "e" notation in the vertices and mesh.py throws an error in converting mesh. How should I handle these "e" values?

https://download.alleninstitute.org/informatics-archive/current-release/mouse_ccf/annotation/ccf_2017/structure_meshes/884.obj
Line 920:

vn -4.73985e-05 -0.162096 0.986775

  File "/usr/local/var/pyenv/versions/cloudvolume/lib/python3.8/site-packages/zmesh/mesh.py", line 144, in from_obj
    (n1, n2, n3) = re.match(r'vn\s+([-\d\.]+)\s+([-\d\.]+)\s+([-\d\.]+)', line).groups()
AttributeError: 'NoneType' object has no attribute 'groups'

[william-silversmith]

I guess I should update the OBJ parser to handle that. Will do that if I get a chance today.

[manoaman]

Thank you @william-silversmith !!!

[william-silversmith]

Check out version 1.6.2

[manoaman]

Thank you Will for the latest updates on the zmesh library. Will there be updated cloud-volume (from_obj, mesh.py) release of this version as well?

[william-silversmith]

I'll try to also make that change there as well.

[manoaman]

Hi @william-silversmith , I think Neuroglancer does not like the way I create an info file. How should I configure info to view the aggregated mesh into a precomputed format? Another question, can I assign segid to each mesh when before concatenate? I was hoping to use a segment_property in the info file. Thank you. -m

def convert_objs_to_precomputed(path_to_obj_files, precomputed_folder_path):
    folder = Path(path_to_obj_files)
    meshes = []

    for file in tqdm(folder.glob("*.obj"), total=len(list(folder.glob("*.obj"))), desc="Loading OBJ Files"):
        with open(file, "rt") as f:
            obj = f.read()
        mesh = Mesh.from_obj(obj)        
        # mesh.segid = int(os.path.basename(file).split(".")[0])
        meshes.append(mesh)
    m = CV_Mesh.concatenate(*meshes)
    m.segid = 1


    info = CloudVolume.create_new_info(  # 'image' or 'segmentation'
                                     # can pick any popular uint
                                     # other options: 'jpeg', 'compressed_segmentation' (req. uint32 or uint64)
                                     # X,Y,Z values in nanometers
                                     # values X,Y,Z values in voxels
                                     # rechunk of image X,Y,Z in voxels
                                     # X,Y,Z size in voxels
        num_channels=1,
        layer_type='segmentation',
        data_type='uint32',
        encoding='raw',
        resolution=[50000, 50000, 50000],
        voxel_offset=[0, 0, 0],
        chunk_size=[64, 64, 64],
        volume_size=[228, 160, 264],
    )

    cv = CloudVolume(f'file://{precomputed_folder_path}', info=info)
    cv.provenance.description = 'Surfaces to a precomputed format'
    cv.commit_info()  # generates gs://bucket/dataset/layer/info json file
    cv.commit_provenance()  # generates gs://bucket/dataset/layer/provenance json file
    cv.mesh.put(m)

[william-silversmith]

Hi m,

I think you have to set the "mesh" directory in the info file first. Unfortunately, CV does not yet support annotations such as segment_properties at this time. PRs may be accepted for that functionality. I keep meaning to get to it eventually, but new projects keep coming up before I get to it.

Will

[manoaman]

Hi Will (@william-silversmith) ,

I think the Neuroglancer is expecting chunk files instead of a single file in a mesh folder from the way I configured. Here is my info file. A couple of questions here. 1) Is there a way to configure info file to only load 1:0:1.gz generated under the mesh folder? 2). Does CloudVolume (or Igneous) support chunking from a concatenated mesh which is saved as 1:0:1.gz in the mesh folder? Thank you!

(info file)

{
  "data_type": "uint32",
  "mesh": "mesh",
  "num_channels": 1,
  "scales": [
    {
      "chunk_sizes": [
        [
          64,
          64,
          64
        ]
      ],
      "encoding": "raw",
      "key": "50000_50000_50000",
      "resolution": [
        50000,
        50000,
        50000
      ],
      "size": [
        228,
        160,
        264
      ],
      "voxel_offset": [
        0,
        0,
        0
      ]
    }
  ],
  "type": "segmentation"
}

(Generated files)

precomputed/annotation_50_ccf_from_obj/

├── info
├── mesh
│   ├── 1:0
│   └── 1:0:1.gz
└── provenance

(Neuroglancer errors)

[manoaman]

CV does not yet support annotations such as segment_properties at this time. PRs may be accepted for that functionality.

What is "PRs"??

Okay! I've been using Google Colab to data wrangle this part of the file creation at the moment. It seems like I need to explicitly downgrade the numpy version to match with Python 3.8.x but managed to use CloudVolume with Colab. e.g.) "!pip install -U numpy==1.22.4 cloud-volume"

[william-silversmith]

Hi m,

Neuroglancer is just looking for the image files that aren't there. You can ignore those errors. If the mesh isn't appearing, make sure you're using igneous view DIRECTORY as regular static file servers don't know how to handle the .gz suffix.

A PR is a Github Pull Request. It means a contribution that is added to CloudVolume, that gets reviewed by me, and then merged into the codebase.

Will

[manoaman]

Hi Will,

It does seem to be loading looking at Neuroglancer chunk statistics. (Ignoring the last row because there is no image files.). One missing component was an info file under the mesh directory which I'm not sure if this template works with generated .gz mesh files. So far no luck. Is there a place I can fix?

(mesh/info)

{"@type":"neuroglancer_legacy_mesh","mip":0,"chunk_size":[128,128,128],"spatial_index":{"resolution":[50000,50000,50000],"chunk_size":[25600000,25600000,25600000]}}

[manoaman]

Hi Will (@william-silversmith ),

After some trials in the transformation matrix in the viewer, I managed to load up the concatenated mesh! So going back to the original question, precomputed mesh from .obj file seems to displayed the whole brain as expected. (997.obj). I suppose I should use .obj instead of slicing NRRD to TIFF stack, and then run CV/Igneous for generating mesh here. On the side note, do CV/Igneous support multi-resolution mesh from .obj files? Thank you for your help!

[william-silversmith]

Hi m,

Congrats on getting it working! Currently, creating multires from an obj is not supported as a workflow, but there's no reason it couldn't in principle be done. I don't currently have the bandwidth to modify the code myself, but if you would like, you can look at how unsharded multires files are created and adapt that code.

https://github.com/seung-lab/igneous/blob/master/igneous/tasks/mesh/multires.py#L45-L81