maximumScreenSpaceError a bad metric for 3D Tiles

[mramato]

I spoke with both @lilleyse and @bagnell about this offline and Sean asked me to submit an issue.

maximumScreenSpaceError does not work very well as a metric for tile selection because it doesn't take into account the size of the Cesium canvas being drawn too, i.e. the same camera view will create wildly different images depending on the size of the window. Instead of absolute pixel size, perhaps this can be better represented as a percentage of pixels of the overall canvas?

I took a quick stab at illustrating this with the below Sandcastle example. If you resize the width of the window, you'll notice the view stays much more consistent than the current behavior.

var viewer = new Cesium.Viewer('cesiumContainer', {
    scene3DOnly : true,
    shadows : true
});

var scene = viewer.scene;
scene.fog.enabled = false;
scene.debugShowFramesPerSecond = true;

var tileset = scene.primitives.add(new Cesium.Cesium3DTileset({
    url : 'https://cesiumjs.org/NewYork/3DTilesGml/tileset.json'
}));

Cesium.when(tileset.readyPromise)
  .then(function(tileset) {
    var boundingSphere = tileset.boundingSphere;
    viewer.camera.viewBoundingSphere(boundingSphere, new Cesium.HeadingPitchRange(0, -2.0, 0));
    viewer.camera.lookAtTransform(Cesium.Matrix4.IDENTITY);
  });


var lastWidth;
scene.preRender.addEventListener(function(){
    var maximumScreenSpacePercentage = 0.01;

    var width = scene.canvas.clientWidth;
    if(lastWidth !== width){
        var maximumScreenSpaceError = maximumScreenSpacePercentage * width;
        lastWidth = width;
        console.log(maximumScreenSpaceError);
        console.log(width);    
        tileset.maximumScreenSpaceError = maximumScreenSpaceError;
    }
});

CC @pjcozzi

[mramato]

For the record, I'm not sure my approach is the best choice either, just one possible alternative.

[pjcozzi]

Added this to #3241. Whatever we do here we would also want to do for terrain/imagery.

[kring]

@mramato can you explain what problem you're trying to solve? IMO a larger screen should use a different level of detail than a small screen, given the same camera position. That's a feature, not a bug.

[mramato]

@kring I think the problem I'm seeing may be specific to tilesets with additive refinement instead of replacement refinement (additive has been used in most of the cities we have worked with so far use). The problem is, unlike terrain where you get a lower-resolution but visually consistent view at lower resolutions (thanks to geometric error), additive refinement means that at lower resolutions, nothing gets rendered at all, so you can zoom in to "Chicago" for instance and see no buildings because no single building is taking up enough pixels to be rendered,. If you decrease the error value, you have the opposite problem where almost all buildings show up even at a distance view because they all meet the criteria.

I'm just reporting what I'm seeing, but perhaps there's already an item somewhere in the spec to improve the mechanism here (or maybe it's all on the implementation side of things). I honestly don't know.

[kring]

@mramato I don't think that's caused by additive refinement, exactly. It sounds like the problem is that the geometric error of the tiles doesn't really reflect their actual content.

I don't know all the details of the Cesium 3D Tiles building rendering, but I can make some educated guesses. Each tile has a geometric error value, of course, and presumably you choose which buildings to include in each tile with a simple heuristic like "if this building's height is less than the geometric error value for this tile, I don't need to include it." But that's not actually right. When you're looking straight down, for example, the height of the building is pretty much irrelevant for deciding whether you need to include it.

Or maybe it's more clever and is considering the horizontal dimensions of the building as well. But now you risk either rendering heaps of buildings, or you tune it differently and buildings disappear while they still take up significant space on the screen.

This isn't easy to solve, but I don't think abandoning screen space error will help you. It will just create new problems. You could maybe add some sort of lower bound. E.g. use screen space error, but use a screen size of at least X by Y in your computations, even if the real screen in smaller.

You guys have thought about this more than me, so no guarantees, but here's an idea that I think would work really well.

When rendering buildings, include not just a gltf model for the tile but also an (optional?) simple rasterized imagery tile. Use only the building height to decide which tiles to include in the gltf portion. The horizontal dimensions don't matter. For all buildings that don't meet the threshold according to their height, rasterize them to the imagery tile. That way the buildings are always visible, but if their heights won't be two pixels (or whatever max SSE you choose), you can just render them as flat images. For bonus points, you can include a normal map for use with lighting computations so the buildings still look like they have a bit of texture to them.