What is CDB 2.0?

[cnreediii]

Taking from the recent 3D Models discussion (and thanks to @ccbrianf), I believe that the fundamental goals for CDB 2.0 is modernization with a number of objectives such as fixing long standing user defined issues, simplification (as appropriate) and relevance to other information domains/communities.

As @cc states: modernization is about supporting ever increasing content capacity and fidelity, modern/new data formats, and fixing or improving historical design choices, while still maintaining performance and expanding usage domains without excluding existing ones.

And taking @ccbrianf question: Can you enumerate how you would define CDB 2.0 that differs from that viewpoint? Let the discussion begin. Personally, I think that this is a great starting point but I agree that we need consensus - especially so we can write this into the executive summary of the ER! We will look kind of silly if we cannot provide a concise problem statement for the reader!

[ccbrianf]

It seems to me we need a revised problem statement for CDB 2.0, or at least a list of bullets like those in the CDB 1.X Problem Statement to guide our design and justify compromises.

While CDB 1.x is currently the same format for both off-line data store and run-time repository use cases, there are some specific discussions for each use case, so there is some precedent for possible profile differences to address these. However, part of what makes CDB CDB is it's client device (application) format independence. So, the more fractured the profile space becomes and the less deterministic run-time performance is preserved, the less CDB remains unique with respect to other GIS/Vis/Sim standards.

[christianmorrow]

@ccbrianf - the link in the comment above (CDB 1.X Problem Statement) is broken :/

This topic -- top level problem statement and high level design objectives -- should receive some attention, and maybe get a vote...really soon (@MichalaH?)

I've become skeptical about 1.X as an "off-line storage repository" since data like 3D models take a one-way trip into the CDB 1.X format. It's virtually impossible to extract/edit/re-insert 3D models in CDB, particularly GT models, and models with lights.

Perhaps the high level concept that needs a bit of steering is: a) a repository with data that is closer to it's original state - easy to collect, edit and maintain - at the cost of run-time performance without additional analysis and processing VS. b) having a repository that has been 'semi-compiled' and organized in a way that facilitates speedy real-time consumption - but requires the retention (and sharing?) of initial-state source data.

The third 'axis' (which I believe CDB 1.X addresses extremely well) in play is the whole-world design - the tiling scheme works great for rasters...the debate lives on for vectors.

Perhaps we should draft a CDB X Problem Statement, circulate for comments and move towards a consensus vote?

[cnreediii]

@christianmorrow - Hmm. Appears that you are getting at the characteristics of the content maintained in a CDB data store. Such discussion is fine when developing guidance on how to process and manage the content. However, at this point I think we should focus more on the architecture and conceptual and logical models. Content workflows should be independent of the architecture, storage models, etc. We simply do not have the bandwidth to address content processing or related issues. As long as we have the proper metadata and provenance information, your use case is easily accommodated - as is the ability to provide a well defined migration path from the CDB 1.x environment to the 2.0 environment.

[ccbrianf]

@christianmorrow, the CDB 1.X problem statement link works fine for me, but that is probably because I'm an OGC CDB SWG member that has opted into the CDB git repository. You can find the section referenced publicly here as section 6.2 Problem Definition, the wording of which has been in CDB for a long time.

I agree that CDB 1.X is problematic as an "off-line storage repository" especially for 3D models, but I disagree on the reasons and which type of models. GT models should be easy as they are a collection of standard OpenFlight files and textures linked by external references, which is pretty common usage for that format. GS models however do have their LoDs and textures separated and distributed into multiple zip files without an efficient way to find how they are logically related and linked together as a single model. Maybe your comment was a typo? That is one reason industry CDB 3.1 had the concept of a master OpenFlight file for GS models such that their parts would be logically linked together in a similar manner to how GT models work with the master file that calls out LoDs today which would aid in the non-runtime use cases as I've discussed before. I agree something like that is still needed, and I'm not quite sure why it disappeared in industry CDB 3.2 other than it wasn't totally fleshed out.
Also as discussed before, you are not required to strip light points off models when putting them into a CDB, so that complaint is only partially valid, presumably only for certain CDB creation tools.
Presagis Terra Vista does have moderate experience with using CDB as such a source repository and working around these pain points, but I agree improvement is certainly desired in multiple ways.

I keep hoping we can come to an option C that is the best of (or at least acceptable to) both worlds described, but it doesn't seem that too many others share the view that is possible without unacceptable compromises on one side or the other. I'm sure we could refine your option wording, but conceptually I think you have somewhat accurately captured the current debate. To me though, again as stated before, option a is not materially different from numerous other standards such as NPSI. And thus it is not what has historically made CDB stand out from the crowd. CDB certainly gets a C- (pun intended :-)) for how it currently satisfies option a though.

I agree with Christian that we need to settle direction here, and I'm having trouble understanding most of the @cnreediii interpretation/objection.

[PresagisHermann]

The 3D model group is facing an important decision that will impact everyone! Do we allow CDB X to mix terrain datasets in the 3D models? In short, do something like OWT by pushing all terrain, building and vectors into glTF encoding - a tiled mesh. This implies that imagery becomes textures and elevation is no longer a raster but a mesh .. among other thing.
I am of the opinion that this is going too far but we need to decide as a group.
This would align CDB X so much with OWT model to the point that they would be hard to distinguish. It also drift significantly from existing CDB 1.X in my view. OWT target a "close to runtime format" where CDB X is now directed by SOCOM to focus on source repository use case (which would tent to keep dataset separated rather than baked) . This also bring the question on the challenge to address two SOCOM requirements: increase interoperability with OWT and have CDB X address the repository usecase primarily.

Opinions?

[ryanfranz]

My opinion is that combining layers together like this gets away from the foundational repository use case. Some applications will want to have just the imagery, and some might only need the elevation data. How hard does it get to update imagery in a CDB X, without touching everything if most of the data layers are mixed together.

Also, from an M&S perspective, we would prefer to keep them separate. There are use cases where a CDB device might want a higher or lower resolution of imagery than the decision that was made when these data types were mixed. So if what was created/mixed together might look good on an HD system, a higher resolution of imagery might be really useful on a 4K projectors (or on emerging 8K systems that were being promoted at I/ITSEC last year). One strength of CDB has been mixing and matching data resolutions based on the needs of the system/device, and the standard not tying (or minimal tying) datasets/layers together.

There are systems that do virtual texture mapping of tiled imagery datasets, and don't need the one-to-one relationship between mesh and texture.

[cnreediii]

@ryanfranz - Agreed. Updates are also much easier if imagery layers are kept separate and not fused.

During OGC Testbed 15, the participants work on and developed prototype approaches for updates to features and to tiled imagery - including an API.

OGC Testbed-15: Images and ChangesSet API Engineering Report (http://docs.opengeospatial.org/per/19-070.html)

From the Abstract: The OGC API – Images: Enables managing (retrieving, creating and updating) sets of images that are georeferenced. The images does not follow any tile scheme, and can partiallyor totally overlap. The API enables a mosaicking use case (where the imagery is combined in a single bigger “picture”) but could also serve a use case in which a moving camera is taking pictures at locations along a route and then stores the images as a single collection.

OGC Testbed-15: Delta Updates Engineering Report (http://docs.opengeospatial.org/per/19-012r1.html)

From the Abstract: This ER documents the design of a service architecture that allows the delivery of prioritized updates of features to a client, possibly acting in a DDIL (Denied, Degraded, Intermitted or Limited Bandwidth) environment. Two different technical scenarios were investigated and tested:

[jerstlouis]

I would support having both options which can be selected by profiles, but when merging terrain and models in the same model file, it should at least be segmented and the models / terrain properly attributed to facilitate extraction and conversion between one approach to the other. I think this could be one way to support interoperability and bi-directional migration of data between OWT / CDB X / CDB 1.

I do see having a single 3D model (e.g. glTF) representing all of the geospecific models within a tile (while preserving proper attribution of the content) as a step in facilitating both direct visualization in simulators and streaming to the edge. We had related discussions about approaches which can keep the models elevation separate to support clamping to the terrain.

[PresagisHermann]

I do not believe we can easily support both solution via profiles. Merging features and terrain into a 3D mesh has implication on LODs, vector linkage to features, independence of datasets (clamping models on terrain elevation) etc... In my view, this is no longer a profile but a different terrain model altogether. Note that imagery would no longer be ortho at that point but texture polygons with UVs mapped on side of buildings the same way as on terrain ... very different. As @ryanfranz point out, this is going away from a source repository.

You can create a textured mesh from a CDB 1.X by merging all dataset. This is what render engines do in a way. The reverse will be challenging. We will have to demonstrate this with OWT import/convert to CDB.

That said, if CDB X need to be close to source, what are the sources? OWT seem to focus in 3D reconstruction as the source, then adds attribution. Traditional CDBs use to do the revers and start from attribution and multi-source (satellite imagery, radar based elevation model, vector based features etc...) to create 3D. So, maybe there is a desire to store those new 3D meshes from reconstruction in CDB if we consider them sources... But I feel that this would break alot of the CDB dataset concepts...

Just to be clear, the question is about having a complete mesh of terrain and features. @jerstlouis point about putting many 3D features into one glTF is a different issue that can be addressed once we decide if CDB accepts complete terrain tile mesh (terrain and features in on mesh).

The voiced opinion up to now are in favor of keeping the datasets separate and not support mesh terrain. At some point we will call for a vote on this but let's allow people to comment on this some more.

[jerstlouis]

@PresagisHermann

what are the sources?

That's exactly the question, and why based on whether the source is a 3D point cloud from photogrammetry and/or LiDAR scan, or originally from separate layers, the closer to source approach for a particular repository might be the integrated mesh, or the separate layers. This is why I suggest the different profiles.

The reverse will be challenging.

A segmented (differentiated terrain and features) and attributed mesh is one step closer towards the traditional CDB model with terrain elevation and ortho photo. From that it should not be too difficult to generate the traditional separate layers (just move the features to their own geospeciffic 3D models layer, center them on elevation point to handle clamping to terrain, flatten the terrain under them, re-project the elevation and imagery to be orthorectified layers).

Most render engines would probably not need to do any of that, since as you point out they would normally do the opposite and create this integrated mesh at runtime anyways. But CDB X profile conversion tools could easily do this, and transition from OWT to CDB X integrated mesh profile would require the more involved processing of properly segmenting and attributing the terrain and features (unless OWT already does all of that).

Of course there are different challenges associated with the integrated mesh approach, but I believe the features and clamping could be addressed with proper attribution of that mesh, and the integrated mesh could follow the same LOD & Tiling model as everything else, with potentially less of the more complicated models-specific LOD approach of CDB 1, and more like the simple vector/raster/elevation tiles where every level is redefined at a different LOD.

All this being said, we ourselves prefer the separate layers approach, but understand that OWT is heavily based on the integrated mesh approach, and I feel the two profiles & segmentation/attribution may help establish a bridge between OWT and CDB X.