3D Tiles Roadmap #3241

Open
pjcozzi opened this Issue Nov 24, 2015 · 20 comments

Comments

Projects
None yet
4 participants
@pjcozzi
Member

pjcozzi commented Nov 24, 2015

New to 3D Tiles? See Introducing 3D Tiles.

This work is in the 3d-tiles branch.

In pseudo-priority order within each section:

Draft 1.0

  • Cesium 3D Tiles implementation
    • Inspector, #4846
    • Is the load order correct? The unit tests look OK; however, for example, the buildings on the right should load sooner: http://cesiumjs.org/WashingtonDC/
    • Memory issues?
      • Unloading issue, #4673
      • Empty tile memory usage, #3453
      • Crash with high memory usage, #4549
    • glTF 2.0, #4009
    • Merge request scheduler into master, #3476
      • IE performance, #4166
      • Unable to have multiple CesiumWidets on the same page in 3d-tiles branch #5226
    • Loose OBBs #4387
    • Translucency and artifacts
      • Combine translucent back/front commands into a single translucent command, however this will look worse when OIT is not supported.
      • Pick translucent features, see TODO in Cesium3DTileBatchTableResources.getVertexShaderCallback
      • Finish derived commands, especially Cesium3DTileBatchTableResources.getAddCommand, #3806
      • Frustum artifacts #4381
    • Finish Styling GLSL-backend AnalyticalGraphicsInc/3d-tiles#140
    • Styling - vector support for built-in functions #4865
  • Doc and Examples
    • cesium.com hosted examples in the Sandcastle example
    • Tutorials
      • Using the Cesium API to load 3D Tiles and interact with individual models.
      • Declarative styling.
    • Copy 3D Tiles Sandcastle tests from the Specs/Data folder to the Apps/SampleData folder.
      • Or store test data in cesium.com and download on npm install or something similar?
  • Any outstanding TODOs in the code
  • Spec draft 1.0 issues
  • Tools - 3d-tiles-tools
  • drillPick point cloud - #4895

Post Draft 1.0

  • Vector tile - instance geometry volumes, #4665
  • Vector tile - z-order, #4665
  • Vector tile and geometry tile - clipping planes
  • If requested - classify with i3dm, #6033
  • Technique like #5128 for fuse tilesets/terrain at runtime
  • Draco compression (then quantize at runtime), #5120
  • Vector tile format. AnalyticalGraphicsInc/3d-tiles#25, #2172
    • Increase bounding volume by pixel size like BillboardCollection and PointCollection. See the Cesium3DTile constructor.
  • Point cloud attenuation example using declarative styling (and vertex shader culling?)
  • scene.pickPosition returns incorrect position #4368
  • Better error metric than SSE? #4043
  • Reuse Primitive Batch Table, #4328
  • Add convenience functions to the Cesium API for querying features based on the batch table hierarchy - like getFeaturesByProperty(propertyName, value) and getFeaturesByClass(className)
  • Cache Management, @lasalvavida
    • Refactor TileReplacementQueue to use DoublyLinkedList?
    • Example code for app that uses tileUnload and tileVisible events to re-style a tile that was unloaded, e.g., by tracking features that changed or higher-level context like how 3D Tiles styling works.

Later + Ongoing Performance

  • 3D Tiles spec and Cesium implementation
  • Cesium 3D Tiles implementation
    • AO for 3D buildings (integrate gltf-pipeline)
    • tileNewlyVisible and tileNewlyNotVisible events
    • Performance: Best SSE in center
    • Performance: Add extra frustum planes to account for how wide the FOV is and that the viewer might not notice objects on the edge and in the distance (horizon views for sure, maybe not aerial), e.g, two planes with normals perpendicular to the view direction
      • Also, tell users to use a more narrow FOV
    • Performance: cull/LOD select with union of children bounding volumes instead of the tile's BV. Does the extra CPU overhead result in fewer tiles on average? Is it a performance win?, #2861
    • Performance: for fast moving views (like flying from home view to Canary Wharf), increase the geometric error so few tiles are requested and then only shown briefly or are already out of view. Could track the viewer's velocity and perhaps heuristics for if a requested tile was immediately culled
    • Performance: optimize replacement refinement - instead of requesting all children when a tile needs to refine, request only the visible ones, and then allow refining a tile if all the required visible tiles are loaded (instead of all - including invisible - tiles). When the camera moves, and children that are not yet loaded are needed, either render the parent or render the available children and the parent with clipping, #4287
      • Still need to clip the parent to the loaded children
    • Performance: skip levels when refining with replacement refinement
      • For additive replacement, load tiles deep in the tree first, then ancestors
    • Performance: optimize batch table memory usage, see PERFORMANCE_IDEA in Batched3DModel3DTileContent.js and Cesium3DTileBatchTableResources.js.
    • Evaluate LZ4m: A Fast Compression Algorithm for In-Memory Data
    • Performance: Turn off opaque textures for picking pass, e.g., in Miami for rollover picking.
    • Performance: exploit temporal coherence in sortChildrenByDistanceToCamera
      • Sort by pixel size instead of distance? What about large bounding volumes with sparse contents?
      • Is selectTiles better implemented with a priority queue instead of explicit sort?
    • Performance: nest spatial data structure to optimize additive refinement, AnalyticalGraphicsInc/3d-tiles#11
    • Performance idea: prefetch children or siblings? For example, see Figure 3. Track stats over time to change algorithm or disable.
    • Performance: Reduce per-DrawCommand overhead
    • Performance: Fuse nodes at runtime to reduce the number of DrawCommands.
    • Performance: "to avoid jarring popping artifacts, it could be useful for a runtime to scale the viewerRequestVolume to be slightly larger, and then check the geometricError for prefetching (but not rendering until the viewer enters the unscaled volume).", AnalyticalGraphicsInc/3d-tiles#101
    • Performance: devicePixelRatio on mobile, #3288
    • Performance: Omni-directional Relief Impostors (ORI), #4908 and https://groups.google.com/forum/#!topic/cesium-dev/tCCooBxpZFU%5B101-125%5D
    • Point clouds: hidden point removal, anisotropic filling, and shape depiction, and compute normals for lighting. Also edge-preserving blur in Game Engine Gems 3 and CIS 565.
    • Labels on buildings
    • Occlusion culling. Drive refinement with VMSSE?
    • Morph between LODs
    • Support 2D and Columbus view. #4884
    • Make PointGeometry and PointAppearance public. #3203
    • Job scheduler, #2655
    • Memory-efficient (bitfield) per-building cast/receive shadows if needed
    • More Dynamic SSE #4307
      • Increase SSE as the view gets more horizon
      • Load tiles up to a max distance first, then load tiles in the distant if the camera doesn't move
      • Fog to help visual quality?
    • Cache Management
      • When skip LOD is used, remove non-leaf tiles from the cache if they are touched by not rendered
      • Research strategies for proactive cache trimming, e.g., number of seconds/frame a tile was not selected, how far out of view a tile is, etc.
        • Potentially track statistics to see if the cache is trashing, and then adjust the replacement algorithm.
      • Is it possible to come up with a better default for maximumNumberOfLoadedTiles since the use cases are so diverse?
      • Unload sub-trees from tiles with tileset.json content when all their content is unloaded
      • Vertex/texture cache across tiles
      • Provide more precise cache size than number of tiles? Count composite tiles as more? Include geometry/texture size with each tile? Perhaps number of tiles is the easiest thing to understand.
    • POP buffer for progressive replacement refinement
  • Review "Interactive inspection of complex multi-object industrial assemblies", #4908

Older

@e-andersson

This comment has been minimized.

Show comment
Hide comment
@e-andersson

e-andersson Nov 26, 2015

Contributor

Does "Cache management and replacement" include handling of which Cesium3DTilesets will be loaded or is it aimed at the lower-level tile handling? We're quite interested in tileset loading/unloading since we'll potentially have a large number of tilesets available to the viewer, causing a lot of initial tiles.json loading and repeating iteration over primitives.

Contributor

e-andersson commented Nov 26, 2015

Does "Cache management and replacement" include handling of which Cesium3DTilesets will be loaded or is it aimed at the lower-level tile handling? We're quite interested in tileset loading/unloading since we'll potentially have a large number of tilesets available to the viewer, causing a lot of initial tiles.json loading and repeating iteration over primitives.

@pjcozzi

This comment has been minimized.

Show comment
Hide comment
@pjcozzi

pjcozzi Nov 26, 2015

Member

Does "Cache management and replacement" include handling of which Cesium3DTilesets will be loaded or is it aimed at the lower-level tile handling?

It means lower-level tile handling, but thanks for reminding me about which Cesium3DTileset to load. I added that to the roadmap. :)

If you are interested in working on either, I'm happy to discuss in more detail.

Member

pjcozzi commented Nov 26, 2015

Does "Cache management and replacement" include handling of which Cesium3DTilesets will be loaded or is it aimed at the lower-level tile handling?

It means lower-level tile handling, but thanks for reminding me about which Cesium3DTileset to load. I added that to the roadmap. :)

If you are interested in working on either, I'm happy to discuss in more detail.

@pjcozzi

This comment has been minimized.

Show comment
Hide comment
@pjcozzi

pjcozzi Jan 22, 2016

Member

Stick with the Cesium3DTile prefix for names?

After doing a bunch of reference doc, I don't have a strong preference, but I am find with this, e.g.,

new Cesium.Cesium3DTileset

Compared to

new Cesium.Tileset3D
new Cesium.Tiles3D
new Cesium.ThreeDTileset

Cesium.Tileset3D would also be OK. @lilleyse what do you think?

Member

pjcozzi commented Jan 22, 2016

Stick with the Cesium3DTile prefix for names?

After doing a bunch of reference doc, I don't have a strong preference, but I am find with this, e.g.,

new Cesium.Cesium3DTileset

Compared to

new Cesium.Tileset3D
new Cesium.Tiles3D
new Cesium.ThreeDTileset

Cesium.Tileset3D would also be OK. @lilleyse what do you think?

@pjcozzi

This comment has been minimized.

Show comment
Hide comment
@pjcozzi

pjcozzi Jan 25, 2016

Member

@TomPed I added a section above for you for when you start on 3D Tiles.

Member

pjcozzi commented Jan 25, 2016

@TomPed I added a section above for you for when you start on 3D Tiles.

@TomPed

This comment has been minimized.

Show comment
Hide comment
@TomPed

TomPed Feb 2, 2016

Contributor

Rename Cesium3DTile#readyPromise to Cesium3DTile#contentReadyPromise

@pjcozzi does this mean the field readyPromise in the file Cesium3DTile.js?

Contributor

TomPed commented Feb 2, 2016

Rename Cesium3DTile#readyPromise to Cesium3DTile#contentReadyPromise

@pjcozzi does this mean the field readyPromise in the file Cesium3DTile.js?

@pjcozzi

This comment has been minimized.

Show comment
Hide comment
Member

pjcozzi commented Feb 2, 2016

@TomPed yes.

@TomPed

This comment has been minimized.

Show comment
Hide comment
@TomPed

TomPed Feb 3, 2016

Contributor

Likewise, replace isReplacementRefinable and isContentUnloaded with readonly properties. Complete doc.

@pjcozzi I can't seem to find isReplacementRefinable in the project.

Contributor

TomPed commented Feb 3, 2016

Likewise, replace isReplacementRefinable and isContentUnloaded with readonly properties. Complete doc.

@pjcozzi I can't seem to find isReplacementRefinable in the project.

@lilleyse

This comment has been minimized.

Show comment
Hide comment
@lilleyse

lilleyse Feb 3, 2016

Contributor

Ah, don't worry about that we removed that. I'll update the task.

Contributor

lilleyse commented Feb 3, 2016

Ah, don't worry about that we removed that. I'll update the task.

@TomPed

This comment has been minimized.

Show comment
Hide comment
@TomPed

TomPed Feb 3, 2016

Contributor

3DTileContentProvider

Should this be Cesium3DTileContentProvider?

Contributor

TomPed commented Feb 3, 2016

3DTileContentProvider

Should this be Cesium3DTileContentProvider?

@lilleyse

This comment has been minimized.

Show comment
Hide comment
@lilleyse

lilleyse Feb 3, 2016

Contributor

Yeah. The asterisks are just saying all things named like that.

Contributor

lilleyse commented Feb 3, 2016

Yeah. The asterisks are just saying all things named like that.

@TomPed

This comment has been minimized.

Show comment
Hide comment
@TomPed

TomPed Feb 3, 2016

Contributor

Oh gotcha, so I'll change anything with 3DTileContentProvider to 3DTileContent.

Contributor

TomPed commented Feb 3, 2016

Oh gotcha, so I'll change anything with 3DTileContentProvider to 3DTileContent.

@lilleyse

This comment has been minimized.

Show comment
Hide comment
@lilleyse

lilleyse Feb 3, 2016

Contributor

Yup

Contributor

lilleyse commented Feb 3, 2016

Yup

@lilleyse

This comment has been minimized.

Show comment
Hide comment
@lilleyse

lilleyse Feb 4, 2016

Contributor

Other than the tests, is Cesium3DTilesetState needed? Can we remove it and just replace it with a _rootRequested boolean in Cesium3DTileset?

It can be removed. I added it in order to support attempting to load tileset.json across multiple frames if the RequestScheduler was full, but that was before I added the RequestScheduler.request function. @pjcozzi I'm more familiar with how this should work, should I handle this one?

Contributor

lilleyse commented Feb 4, 2016

Other than the tests, is Cesium3DTilesetState needed? Can we remove it and just replace it with a _rootRequested boolean in Cesium3DTileset?

It can be removed. I added it in order to support attempting to load tileset.json across multiple frames if the RequestScheduler was full, but that was before I added the RequestScheduler.request function. @pjcozzi I'm more familiar with how this should work, should I handle this one?

@pjcozzi

This comment has been minimized.

Show comment
Hide comment
@pjcozzi

pjcozzi Feb 4, 2016

Member

@lilleyse sure, whatever you think.

Member

pjcozzi commented Feb 4, 2016

@lilleyse sure, whatever you think.

@TomPed

This comment has been minimized.

Show comment
Hide comment
@TomPed

TomPed Feb 4, 2016

Contributor

UI elements for loadProgress (commented out) in Cities.html

Are these commented lines to be deleted? Or is the code to be uncommented and fixed?

Contributor

TomPed commented Feb 4, 2016

UI elements for loadProgress (commented out) in Cities.html

Are these commented lines to be deleted? Or is the code to be uncommented and fixed?

@lilleyse

This comment has been minimized.

Show comment
Hide comment
@lilleyse

lilleyse Feb 4, 2016

Contributor

We want to bring it back, and add the messages to the UI rather than just the console. Maybe it's best to skip this one for now because @pjcozzi is simplifying 3D Tiles.html on the style branch,

But eventually I think the idea is to have a better UI for visualizing 3D Tiles, similar to the Cesium Inspector demo. This may require looking around at other Sandcastle files for ideas.

Contributor

lilleyse commented Feb 4, 2016

We want to bring it back, and add the messages to the UI rather than just the console. Maybe it's best to skip this one for now because @pjcozzi is simplifying 3D Tiles.html on the style branch,

But eventually I think the idea is to have a better UI for visualizing 3D Tiles, similar to the Cesium Inspector demo. This may require looking around at other Sandcastle files for ideas.

@pjcozzi pjcozzi referenced this issue Nov 23, 2016

Open

BIM roadmap #4678

30 of 53 tasks complete
@pjcozzi

This comment has been minimized.

Show comment
Hide comment

@pjcozzi pjcozzi referenced this issue Apr 11, 2017

Merged

Skip LODs #5128

6 of 7 tasks complete

@pjcozzi pjcozzi referenced this issue Nov 30, 2017

Merged

Vector Tiles #4665

0 of 2 tasks complete
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment