-
Notifications
You must be signed in to change notification settings - Fork 45
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Upgrade I3DM to GLB with EXT_mesh_gpu_instancing
#52
Conversation
- Moving functions from the PntsPointClouds class to the TileTableData classes, because they are applicable to both PNTS and I3DM feature tables - Setting up the tests and TilesetUpgrader functions for the conversion - Adding a first draft of the convertI3dmToGlb function in the TileFormatsMigration - highly preliminary!
The state here is HIGHLY preliminary, focussing on getting the right output for all test cases. It will (have to) be cleaned up in the next iterations.
Split PNTS/B3DM/I3DM migration into dedicated classes. Tried to get the rotation right in any many cases as possible...
Getting the conversion of I3DM to GLB right is far trickier than it might look at the first glance. There certainly are cases where it is "relatively easy". But there are many possible corner cases, and eventually, even cases where the conversion is not possible in practice. The cases where the conversion is not possible are related to the fact that in I3DM, the transforms are applied to the whole glTF. With The current state tries to compensate for that difference: It is computing the inverse of the node transform, and modifying the data for the instancing extensions so that these modifications basically "cancel out" the node transform. This seems to work for the cases for which tests exist in the CesiumJS specs, but these are still structurally simple. Creating further test cases (e.g. an I3DM where the glTF contains a mesh primitive is attached to a node that has a complex transform, involving rotations, translations and scales) could be worthwhile, although even the current spec cases involved a few headaches and caused me to doubt myself. The discussions in CesiumGS/cesium#10255 and related/linked issues/PRs convinced me that it is indeed tricky to get the order and coordinate system conversions right... In any case: As soon as glTF animations come into play, there will be a limit: The instancing extension can be attached to an animated node and create instances. But it cannot create animated instances. Points that are still TODO here:
|
The last commits addressed some of the points that still had been open:
Solved by passing in a function that receives the GLB URI, and returns the buffer with the GLB data.
This was addressed by grouping the nodes of the glTF by having epsilon-equal transforms. The current spec data does not contain such a case, so this has to be created. Other commits have been for testing the computation of the positions for the instancing extension more thoroughly. For that, I used an I3DM that I created dedicatedly for these tests: The nodes in this glTF contain non-trivial rotations, translations, and scales, and seem to be converted properly. There is a difference in the lighting that might be caused by differences in the normal computation in the shader, but this has to be investigated separately. This is added as Still open:
|
So far the intended approach was to compute the positions/translations/scales of the instancing extensions from the respective property of the I3DM. But there may be cases where this is not possible. For example: This is an I3DM, where the instance in the lower right is translated about (2,0,0), and rotated about 22.5 degrees around the x-axis. The It is translated. And it is rotated. But the translation that resulted from the rotation of the whole GLB is not taken into account here. In order to create proper glTF instancing extension transforms for each object (i.e. for the axes, the At some point, it could make sense to consider a completely different approach here - namely, to not ("analytically") compute the proper position/rotation/scale, but to stupidly apply all the transforms of the I3DM to the glTF, one after another, then just 'reverse engineer' the resulting positions/rotations/scale from the resulting (global) node matrices, and then throw away all glTF node transforms and replace them with the instancing extension transforms. This may be costly in some ways (also because it basically requires the extension to use rotation/translation/scale, even in cases where the actual instancing of the I3DM originally only used one of them). But it might be easier and more maintainable. I'll probably give it a try and see how it goes... |
From a first test, this seems to be orders of magnitude easier. All the manual computations and special cases collapse into a few simple steps:
Previously, I had played "whack-a-mole" for quite a while, consistently being ""successful"" in coming up with new test cases where the previous approach failed, but all these test cases seem to work with the new approach. (Except for animations, but these are not only difficult, but in many cases impossible to convert anyhow). I still have to do a cleanup pass, which will remove lots of code, but so far, this looks like a far more reasonable way to go here. |
The approach of computing the matrices from the I3DM data, splitting them into T/R/S, and applying the results to the "flattened" glTF seems to be far easier. The fact that this completely rules out animations is unfortunate. But in most cases, the GLBs in I3DMs will not be animated, and many forms of animations that could theoretically appear cannot be translated into the instancing extension anyhow. The last updates included an implementation of Judging from the |
This PR addresses the task of converting I3DM files to GLB with
EXT_mesh_gpu_instancing
during theupgrade
.Further details will be added here soon.