[Feature Request] TilesetCombiner fails for older tilesets with content adressed by url property instead of uri

[jo-chemla]

When using the combine method on tilesets produced by ContextCapture, then an error Content does not have a URI is raised. This is probably because the tile contents in the tileset root json file (plus subsequent tiles) are referred to via the url property rather than the uri property - which references either b3dm or json tile contents.

I know I can use the 3d-tiles-tools upgrade utility on local tilesets I do manage - although it will take a lot of time to upgrade loads of tilesets.
But for tilesets I do not manage, which are stored online on different servers and that I'd like to combine, then the easiest way would be for the combine utility to allow for tilesets which use either the url or the uri property to reference tile content.

[javagl]

We have been pondering this question internally.

The pragmatic view is: All inputs have to be valid. And for the case of the uri vs. url, the recommendation would be: Run upgrade on the input, and then use the upgraded tilesets for any further operations.

But... I can see that this is... invonvenient, particularly when the reason for something not working properly is literally that of a single letter (or "a pixel", as in "the difference between i and l...).

One reason of why we are (currently) not just ignoring this point is that the generalization of this question could be: Which forms of "invalid" inputs should be "ignored"? Or: Which parts of the upgrade should be done autmatically in other operations? At some point, we'd have to spread workarounds for legacy data, handling for special cases, and pseudo-upgrade functionality throughout the code. And it would really be better to summarize this in the upgrade command.

(One example: When you combine two tilesets that use url - should the combined tileset use uri or url then? It should probably use uri, but maybe you want to retain the original url...?)

That being said: The case of the content url was a special one for me as well: The original (pre-refactoring) state of the 3d-tiles-tools had been using the (somewhat unspecified) Tileset sample data extensively for tests. This one uses url. I created a utility function to handle this more transparently. But at some point, I just created a TilesetWithUris from that, to not have to cope with this issue for some tests 😕

So I agree that we should consider to make the tools more resilient for the particular, very special case of uri-vs-url...

[jo-chemla]

Hi again, and again thanks for all this context, very useful.
Understood for the fact that the upgrade utility would be the corner piece before any other workflow operation, which would therefore only have to deal with valid tilesets, simplifying the code a lot. Then indeed, the question of whether to perform such pseudo-upgrade within workflow operations could be important - and even generalized as a cli option for every workflow.

Good question regarding whether the output combined tileset should have a url or uri property. If the tile versioning is a property of each tile json, then since viewers can handle tilesets of different versions, I would say they can be mixed, but I don't know the intricacies of such a decision.

And it's true this uri-vs-url seem like a special corner-case. Being able to combine tilesets without the requirement for upgrading them would make it easier to source external tilesets (if served from a CORS enabled server) without the need to download/serve them and mix multiple sources for content.

[jo-chemla]

Edit final questions related to tiling:

• are utilities defined in Cesium3DTilesWriter meant to build a tool to tile arbitrary input meshes to 3D-tiles? Probably something that is used internally by the Cesium Ion pipeline but not meant to be open-sourced as a complete tool yet? All commercial photogrammetry suites have been offering 3D-tiles exporting capabilities for the past few years (RealityCapture, ContextCapture, Agisoft Metashape, etc), but no open-source suite does so at the moment (Colmap/Meshroom etc), probably because of that fact that there is no oss tiler for meshes existing yet, only for pointclouds.
• final question: when tiling a pointcloud into a Potree dataset, I can specify the bounding box of the input pointcloud data which will be used as the box of the root tile. Therefore if I reuse the same bbox for tiling multiple pointclouds (where coords are expressed in the same Coordinate Reference System), and PotreeConvert them sequentially with the overwrite option, then it is equivalent to appending pointclouds to the same tiled Potree container. Is there any standard representation as to how to do this for 3d-tiles? This way, we could probably combine tilesets to a master, earth-covering tileset, where tilesets would be appended at a given level in the hierarchy without affecting any other portions of the tileset, and this would be really useful - a way for us to combine a few hundreds indepdent tilesets to a single, earth-wide tileset ala google3dtiles.

[javagl]

Understood for the fact that the upgrade utility would be the corner piece before any other workflow operation,

Ideally, doing an upgrade should never be necessary. It should only be a last-resort option for the case that somebody has a legacy tileset, and wants to try and salvage it. One difficulty is that it is tremendously hard to say which "legacy" elements can be updated (and how), and what exactly the result will be. One example is that of tilesets that contain glTF 1.0 data. It is sometimes simply not possible to update this to glTF 2.0, and whether it is possible depends on many low-level technical factors, and it would be hard or impossible to establish a reliable contract for the behavior of the upgrade in this case.

The url-vs-uri, however, is relatively simple. It only twiddles in the JSON, and can be upgraded with a few lines of code. (So it doesn't really need the broader infrastructure that is offerd by the general upgrade command). We'll still have to see where/how to integrate that step in the most most sensible way. The two "extremes" would be:

• Handle this in the most fine-grained manner, on the fly. Instead of writing const uri = content.uri, we'd have to say const uri = Contents.getUri(content), where the latter is the utility function that returns the url from legacy content if necessary
• Handle it as a coarse-grained, blanket step. Whenever a tileset.json is read, there could be one pass to PseudoUpgrade.doThatUrlToUriUpgradeIfNecessary(tilesetJson), to have one place where this upgrade happens if necessary, and all the remaining code can use the content.uri directly, without having to worry about the case that it might be a url

Both have pros and cons. We'll have to sort that out..

---

Regarding the last questions...:

are utilities defined in Cesium3DTilesWriter meant to build a tool to tile arbitrary input meshes to 3D-tiles? Probably something that is used internally by the Cesium Ion pipeline but not meant to be open-sourced as a complete tool yet?

The Cesium3DTilesWriter in particular is just intended to have a mechanism for reading/writing the tileset JSON data. This code is auto-generated from the schema. Users could use this, for example, when they implement a tool that converts meshes to 3D Tiles. Considering the complexity of such a tool, the part that is responsible for writing the JSON would only be a minor, minor part, but ... at least, users wouldn't have to write that part from scratch.

final question: when tiling a pointcloud into a Potree dataset, ...

Sorry, I'm lacking a lot of context here. It sounds like this might be related to external tilesets in general, and maybe something that involves additive refinement. But I don't know Potree well enough to say more here. You might consider bringing this up in the forum at https://community.cesium.com/c/3d-tiles/16 , maybe someone with knowledge about Potree and a better understanding of your goals could chime in there.

[jo-chemla]

Thanks again for all the details regarding the intricacies of such a support url in tile contents feature - plus the two extreme ways to design the implementation!

Also thanks to highlight that this Cesium3DTilesWriter wrapper class is meant for handling metadata only - and indeed, I can imagine that tiling the mesh + texture is the heavier load/block of the tiling process.

---

Finally regarding my last comment, I used Potree simply to illustrate my point - and indeed additive vs replacement refinement + external tilesets is part of the solution. External tilesets would let me create a single master tileset covering the whole earth, and then insert within that root tileset, say a tileset covering Paris/Notre Dame (probably in the same coordinate system), and then again, insert a tileset covering Paris/Eiffel Tower.

That resulting tileset would be made mostly of pointers to external, focus (monument- or city-wide), tilesets, probably each inserted at a level in the hierarchy depending on where the inserted tileset tree representation differs from the hierarchy of the current tileset.

The thing I'm not entirely sure is whether this insertion step would be covered by the upgrade/merge utilities (not yet per my understanding), so I would have to design such a system. Given the mixed extent of the scans we produce (10-100m monument or 10km city), inserting at the same level within the tileset hierarchy would not make sense, hence we need to find a procedure for that insertion step to update the master tileset and references to external tilesets. Plus I do have to anticipate whether all these tilesets indeed have to be within the same coordinate system, or if I can mix them using the transform property of the tile - so master could be in EPSG:4326 but then other scans could be in national or UTM CRS.

[javagl]

Regarding the last points:

1. an issue that started as a 'bug' report about some handling of a legacy url property in JSON data is probably not the right place to elaborate that
2. I'm not the right person to answer all these questions profoundly.

The thing I'm not entirely sure is whether this insertion step would be covered by the upgrade/merge utilities (not yet per my understanding), so I would have to design such a system.

That's right. The 3d-tiles-tools only recently have been "revived", offering some of the "legacy" functionality, and some points on the roadmap for possible extensions that may be tackled in the near future. But what you mentioned does not sound like something that could be offered "directly" in the tools, because it is hard to describe this as a general operation (i.e. you cannot describe this as a function that receives a bunch of tileset.jsons and produces a new result).

The questions rather sound like very broad and general questions about design decisions and best practices for creating actual, real-world tilesets. And when the question is "How to define a 3D Tiles tileset of the world?", then there are (too) many variables to consider.

Given the mixed extent of the scans we produce (10-100m monument or 10km city), inserting at the same level within the tileset hierarchy would not make sense, hence we need to find a procedure for that insertion step to update the master tileset and references to external tilesets.

There probably are tilesets that represent the city of Paris with the Eiffel Tower just being a box with 8 vertices. But you might very well create a scan of the Eiffel Tower as a 10-billion-point photogrammetry model. And there is no blanket statement about where to "insert" such a model in your tile/tileset hierarchy. All I can say for now is that the mechanisms of 3D Tiles offer many degrees of freedom here. So you might have a tileset of Paris where the Eiffel Tower is a "box" with a high geometricError, or it might be a tileset consisting of many, large GLB files, with a low geometricError.

[jo-chemla]

Again thanks for all these details, very useful to understand the state of this toolset. And indeed probably not the right place to share these thoughts - the way I did it here is because from my point of view this could be what merge/combine are meant to do - combining tilesets, merging two or more hierarchical structures keeping only the lowest error/highest detail data.

You're right I'll iterate over these thoughts and ask along the way to the community forum instead if I need some feedback. Thanks again for all your help!

[javagl]

As I mentioned, there are some thoughts about possible extensions of the tools in the future. These thoughts differ in how likely it is for them to become actual points on a roadmap.

One example is that we have 'combine' (which creates one large tileset from one that had external tilesets), but we don't have an opposite of that - i.e. there is no "split" function. This could, in some way, be a low-hanging fruit: Just traverse the tileset, and whenever you reach depth x, start writing out a new one.

Now we could go and implement that. But...

1. nobody needed it until now (i.e. there was no feature request for that)
2. there is some overlap to concepts of implicit tilesets (the x would be the subtreeLevels, so to speak)
3. maybe most importantly: This could be generalized ... abritrarily

And before starting something like that, one should at least have a rough idea about how it could be generalized. For example: People might not want to split their tileset into "slices" with x levels each, but maybe based on some geometricError-threshold`, or maybe based on the file/data size of the results...

But if you have ideas (or even specific demand) for a certain functionality, just let us know... (or... open a PR, of course...)