Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Export/Perf: use numpy in extract_primitives (2-8x speedup) #1120

Merged
merged 2 commits into from
Sep 5, 2020
Merged

Export/Perf: use numpy in extract_primitives (2-8x speedup) #1120

merged 2 commits into from
Sep 5, 2020

Conversation

scurest
Copy link
Contributor

@scurest scurest commented Jul 13, 2020

Work in progress. This is ready.

Rewrite extract_primitives to use numpy and foreach_get. This gives a pretty good perf boost; export times are now comparable to the other Python exporters (eg. FBX).

Some time comparisons.

Model Before After
CesiumMan, no anim 0.37s 0.09s
Ruby Rose, no anim (iss848) 6.0s 2.7s
01_vertex_count_32bit 2.6s 0.5s
Suzanne subdivided 4 times 9.6s 1.2s
AliciaSolid (iss967) 5.8s 1.7s
folded.blend (iss1112) 194s 35s

Dots

To replace the dict of verts, the idea is to use numpy.unique. We collect a "dot" for every loop in the mesh. Each dot stores the Blender vertex index (which fully determines all its per-vert data: position, vertex groups, etc), and all the per-loop data at that loop. To make a primitive, we slice out all the dots for the primitive's tris, use numpy.unique to dedupe them, and then read out the attribute arrays from the dots.

convert_swizzle_*

These aren't used by extract_primitives anymore, so I moved them to gltf2_blender_node where they're still called.

The transformations from eg. __convert_swizzle_normal are now part of the __get_normals function.

Changes

I think there are only two changes

  • the bitangent vector is never stored. Instead, only the bitangent sign is tracked. This should only flip if we apply a transform that reverses orientation (determinant < 0). Since the Zup2Yup doesn't change orientation, the only reason it would flip would be the apply_matrix transform from skinning.
  • there is no warning for 0-length tangents since convert_swizzle_tangent is gone. I can put one back in if you want though.

@fire
Copy link

fire commented Jul 23, 2020

Is there a good way to test on 2.83? Can you generate a zip for me?

@scurest
Copy link
Contributor Author

scurest commented Jul 23, 2020

@fire You'll need 2.90 for this. Actually the master branch just switched over to targeting 2.91, but this will still work on 2.90.

@scurest scurest changed the title [WIP] Export/Perf: use numpy in extract_primitives (2-8x speedup) Export/Perf: use numpy in extract_primitives (2-8x speedup) Jul 23, 2020
@scurest scurest marked this pull request as ready for review July 23, 2020 19:34
@scurest
Copy link
Contributor Author

scurest commented Jul 23, 2020

@julienduroure Think this is ready for review now, when you have time. I updated the OP.

@scurest
Copy link
Contributor Author

scurest commented Aug 5, 2020

Tested exporting original.glb from #1166 with default export options. The memory use also improves a lot.

  • master
    • time: 118s
    • peak mem: 7.4G
  • this branch
    • time: 14s
    • peak mem: 2.5G

@julienduroure julienduroure added this to the Blender 2.91 milestone Aug 8, 2020
@julienduroure julienduroure added enhancement New feature or request exporter This involves or affects the export process Mesh_&_Object labels Aug 10, 2020
@julienduroure julienduroure merged commit eace2a7 into KhronosGroup:master Sep 5, 2020
@julienduroure
Copy link
Collaborator

Thanks !

@scurest scurest deleted the extract-primitive-numpy branch September 5, 2020 18:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request exporter This involves or affects the export process Mesh_&_Object
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants