Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

scene.pickPosition returns incorrect position #4368

Open
hpinkos opened this issue Sep 26, 2016 · 27 comments
Open

scene.pickPosition returns incorrect position #4368

hpinkos opened this issue Sep 26, 2016 · 27 comments

Comments

@hpinkos
Copy link
Contributor

@hpinkos hpinkos commented Sep 26, 2016

Reported on the forum: https://groups.google.com/forum/?hl=en#!topic/cesium-dev/0E-aKBTLESk

The user created this sandcastle example to demonstrate the problem: http://hosting.virtualcitysystems.de/demos/temp/pickProblem/Apps/Sandcastle/?src=3D%20Tiles.html&label=undefined

The demo is using the latest 3d-tiles branch

pickPosition works fine if you run the example inside sandcastle, but if you click 'Open in New Window' sometimes the returned position sits in front of the building you clicked instead of where the click intersects with it. Maybe it's related to the canvas ratio? See the forum post for more details.

I couldn't reproduce this with any of our sample models, but I asked the user if he could share a tile for us to test with.

@hpinkos hpinkos added the type - bug label Sep 26, 2016
@lucasvw

This comment has been minimized.

Copy link

@lucasvw lucasvw commented Sep 27, 2016

Here is the tileset which is used in the sandcastle demo: http://hosting.virtualcitysystems.de/demos/temp/b3dm

Here is a video which shows the behavior: https://youtu.be/9twwwMHbjKU
And here is another that shows that it has something to do with the size of the screen: https://youtu.be/IbDNwqKVOik

And zooming : https://youtu.be/nJpHGOdTssY

@lucasvw

This comment has been minimized.

@hpinkos

This comment has been minimized.

Copy link
Contributor Author

@hpinkos hpinkos commented Sep 27, 2016

Thanks @lucasvw!

@lilleyse

This comment has been minimized.

Copy link
Contributor

@lilleyse lilleyse commented Sep 27, 2016

I'll take a look at this soon.

@pjcozzi pjcozzi mentioned this issue Sep 27, 2016
58 of 115 tasks complete
@lilleyse

This comment has been minimized.

Copy link
Contributor

@lilleyse lilleyse commented Oct 4, 2016

Just to update this issue, it looks like this is related to inconsistencies with depth picking with multifrustum rendering, so it's a bug in the core engine. I still need to investigate more.

@lucasvw

This comment has been minimized.

Copy link

@lucasvw lucasvw commented Oct 14, 2016

Is this going to be part of the bug-bash next week? +1 for yes ;)

@mramato mramato added the bug bash label Oct 14, 2016
@mramato

This comment has been minimized.

Copy link
Contributor

@mramato mramato commented Oct 14, 2016

@lucasvw I marked us to at least look at it as part of the bash, but depending on how involved the fix is, I can't promise it will be resolved. We'll try our best!

@lucasvw

This comment has been minimized.

Copy link

@lucasvw lucasvw commented Nov 3, 2016

@mramato @lilleyse do you perhaps have an idea when you will be able to fix this ?

@lilleyse

This comment has been minimized.

Copy link
Contributor

@lilleyse lilleyse commented Nov 3, 2016

Sorry about the delay, I will spend some more time looking into this.

@lucasvw

This comment has been minimized.

Copy link

@lucasvw lucasvw commented Nov 8, 2016

That would be great! Thanks a lot @lilleyse

@lucasvw

This comment has been minimized.

Copy link

@lucasvw lucasvw commented Nov 22, 2016

@lilleyse I barely dare to ask... but eh.. any updates on this ?

;)

@lilleyse

This comment has been minimized.

Copy link
Contributor

@lilleyse lilleyse commented Nov 22, 2016

Sorry but not yet. I did find a slight bug with the calculated pick position (#4615) but it didn't solve the problem unfortunately.

If I remember correctly the core issue here is that Cesium uses multi-frustum rendering and when the pick position is calculated it looks to see if a depth exists starting from the first frustum. The problem is that each frustum may have a different depth at that pixel and it doesn't know which frustum to use. That's just a theory though, but it could be tough to solve. I promise I'll get to this eventually.

@lucasvw

This comment has been minimized.

Copy link

@lucasvw lucasvw commented Nov 24, 2016

Thanks for the update and the explanation!

@kring

This comment has been minimized.

Copy link
Member

@kring kring commented Jan 19, 2017

In looking at this while debugging #4855, I saw a case where I had only a single frustum, and yet the position returned by pickPosition was wildly wrong. For example, values in pickGlobe in ScreenSpaceCameraController.js:

image

That's without anything in the scene except for the globe (and sun and moon and such I suppose, but not explicitly-added primitives).

So I don't think it's necessarily multi-frustum related.

@emackey

This comment has been minimized.

Copy link
Contributor

@emackey emackey commented Feb 1, 2017

Our "pick position" Sandcastle demo limits your picking to just a 3D model of the milk truck. Check out what happens if you open it up to allow picking the entire globe. It works at close range to the truck, but the more you zoom out, the worse it gets.

badpickposition_v3

var viewer = new Cesium.Viewer('cesiumContainer', {
    selectionIndicator : false,
    infoBox : false
});

var scene = viewer.scene;
var handler;

var modelEntity = viewer.entities.add({
    name : 'milktruck',
    position : Cesium.Cartesian3.fromDegrees(-123.0744619, 44.0503706),
    model : {
        uri : '../../SampleData/models/CesiumMilkTruck/CesiumMilkTruck-kmc.gltf'
    }
});
viewer.zoomTo(modelEntity);

var labelEntity = viewer.entities.add({
    label : {
        show : false,
        showBackground : true,
        font : '14px monospace',
        horizontalOrigin : Cesium.HorizontalOrigin.LEFT,
        verticalOrigin : Cesium.VerticalOrigin.TOP,
        pixelOffset : new Cesium.Cartesian2(15, 0)
    }
});

var sceneModeWarningPosted = false;

// Mouse over the globe to see the cartographic position
handler = new Cesium.ScreenSpaceEventHandler(scene.canvas);
handler.setInputAction(function(movement) {
    var foundPosition = false;

    var scene = viewer.scene;
    var pickedObject = scene.pick(movement.endPosition);
    if (scene.pickPositionSupported) {
        if (scene.mode === Cesium.SceneMode.SCENE3D) {
            var cartesian = viewer.scene.pickPosition(movement.endPosition);

            if (Cesium.defined(cartesian)) {
                var cartographic = Cesium.Cartographic.fromCartesian(cartesian);
                var longitudeString = Cesium.Math.toDegrees(cartographic.longitude).toFixed(2);
                var latitudeString = Cesium.Math.toDegrees(cartographic.latitude).toFixed(2);
                var heightString = cartographic.height.toFixed(2);

                labelEntity.position = cartesian;
                labelEntity.label.show = true;
                labelEntity.label.text =
                    'Lon: ' + ('   ' + longitudeString).slice(-7) + '\u00B0' +
                    '\nLat: ' + ('   ' + latitudeString).slice(-7) + '\u00B0' +
                    '\nAlt: ' + ('   ' + heightString).slice(-7) + 'm';

                var camera = scene.camera;
                labelEntity.label.eyeOffset = new Cesium.Cartesian3(0.0, 0.0, camera.frustum.near * 1.5 - Cesium.Cartesian3.distance(cartesian, camera.position));

                foundPosition = true;
            }
        } else if (!sceneModeWarningPosted) {
            sceneModeWarningPosted = true;
            console.log("pickPosition is currently only supported in 3D mode.");
        }
    }

    if (!foundPosition) {
        labelEntity.label.show = false;
    }
}, Cesium.ScreenSpaceEventType.MOUSE_MOVE);
@duvifn

This comment has been minimized.

Copy link
Contributor

@duvifn duvifn commented Feb 8, 2017

@emackey

  1. I'm not sure that the 'jittering' in your animation is due to the pickPosition results . On my machine, If I set eyeOffset.z to a fixed value this doesn't happen.

  2. I think it's worth adding scene.globe.depthTestAgainstTerrain = true; to the code in your comment, because this is not the default in the sandcastle.
    pickPosition returns different values on terrain if depthTestAgainstTerrain == false , usually below the ellipsoid surface (maybe of the DepthPlane).

  3. The Alt string in this example code is truncated so it shows an wrong height.
    for example: -143976.41485355987 becomes 3976.41.

@denverpierce

This comment has been minimized.

Copy link
Contributor

@denverpierce denverpierce commented Feb 13, 2017

What GPUs/Browsers does everyone run into this on? I get this pretty consistently on a discrete AMD GPU, which makes me think that @kring idea (caught up with the zoom issues which were exacerbated by this issue) that it's depth buffer related with AMD cards may have something to do with it.

@jbo023

This comment has been minimized.

Copy link
Contributor

@jbo023 jbo023 commented Jul 13, 2017

@lilleyse
After some hours of debugging i found a possible solution for this.

The problem is that scene.pickPosition sometimes returns a negative height value if used in combination with scene.pick.

I noticed that the number of frustums in the pickPosition rendering pass differs from the number of frustums in the normal rendering update, when pickPosition delivers a corrupt result.

After some digging I figured that the scene.pick function changes the framestate cullingVolume which is then also used from the pickPosition function.

So when I use the pickPosition function without calling any scene.pick() in the same frame, pickPosition will use the cullingVolume from the last rendering pass and delivers the correct position result.

var viewer = new Cesium.CesiumWidget('cesiumContainer', {
        scene3DOnly : true,
        shadows : true
    });

    //var viewer = cesium.getViewer();

    var scene = viewer.scene;

    viewer.camera.up = new Cesium.Cartesian3(0.602242291265668, 0.47745810596428245, 0.6397952638618689);
    viewer.camera.direction = new Cesium.Cartesian3(-0.09199309329813982, 0.8376012488044867, -0.5384806577646075);
    viewer.camera.position = new Cesium.Cartesian3(3785056.8752774675, 901611.4457953185, 5037174.477028298);


    var tileset = scene.primitives.add(new Cesium.Cesium3DTileset({
        url : 'https://d35ei6ur3bjvr1.cloudfront.net/berlin/3c2a32d3-633c-462d-b79c-7215dcfbc44f',
    }));

    var dataSourceCollection = new Cesium.DataSourceCollection();
    var dataSourceDisplay = new Cesium.DataSourceDisplay({
          scene : scene,
          dataSourceCollection : dataSourceCollection
        });

        var clock = viewer.clock;

        var eventHelper = new Cesium.EventHelper();
        clock.onTick.addEventListener(function(clock){
          var time = clock.currentTime;
          dataSourceDisplay.update(time);
        }.bind(this));

    // Mouse over the globe to see the cartographic position
    var handler = new Cesium.ScreenSpaceEventHandler(viewer.scene.canvas);
    handler.setInputAction(function(click) {
        var foundPosition = false;

        var scene = viewer.scene;
        var pickedOject = scene.pick(click.position);        
        if (true) {
            var position = viewer.scene.pickPosition(click.position);            
            var carto = Cesium.Cartographic.fromCartesian(position);
            console.log('pos' + carto);
            
            
            if (Cesium.defined(position)) {
                dataSourceDisplay.defaultDataSource.entities.add({
                    position : position,
                    point : {
                        pixelSize : 10,
                        color : Cesium.Color.YELLOW
                    }
                });
            }
        }
    }, Cesium.ScreenSpaceEventType.LEFT_CLICK);


With this Sandcastle code i could reproduce the corrupt height value if i click on the building in the center of the screen. If you just remove the line var pickedOject = scene.pick(click.position); pickPosition works correct.

@lilleyse

This comment has been minimized.

Copy link
Contributor

@lilleyse lilleyse commented Jul 13, 2017

This helps a lot @jbo023.

So why does it matter that the pick pass renders lesser frustums if it is still writing out depth correctly? Is the pick depth-copy broken? Just questions I'm thinking about right now...

@likangning93 likangning93 mentioned this issue Apr 19, 2018
10 of 10 tasks complete
@thw0rted

This comment has been minimized.

Copy link

@thw0rted thw0rted commented Jul 20, 2018

Hey all, it's been a year and I'm pretty sure that (at least the way I'm using it?) scene.pickPosition is still giving me hilariously-wrong results. I'm passing the endPosition from a MoveEvent and getting back a point that, when converted to Cartographic, has a height in the negative millions. Is there another way I'm supposed to get lat / lon point under the cursor on the rendered globe along with a relatively accurate height-above-ellipsoid? Nothing is working...

@hpinkos

This comment has been minimized.

Copy link
Contributor Author

@hpinkos hpinkos commented Jul 20, 2018

@thw0rted this is for using pickPosition to pick the globe? I would recommend using either scene.camera.pickEllipsoid if you don't have terrain or scene.globe.pick if you do have terrain. pickPosition works great for getting the position on a 3D tileset or a glTF model, but we haven't had a chance to fix the globe picking issue.

@thw0rted

This comment has been minimized.

Copy link

@thw0rted thw0rted commented Jul 23, 2018

I don't think I'm getting sane altitudes from any method. For comparison, I made a quick SandCastle testbed. Zoom pretty far out (at least so you can see the whole globe) then just kind of noodle around -- when I have North America in the center of the globe, I can mouse over the Amazon, near the edge of the visible globe, and get altitude values like -18000 m (!) for Globe.pick. Most values are negative and the magnitude of the value tends to increase the further your view is zoomed out -- once you zoom in far enough, you start to see some positive values show up even outside of major mountain ranges.

The interesting bit is that Globe.pick gives massive negative altitudes even when the Ellipsoid terrain provider is used. In theory, the values for Globe.pick and Camera.pickEllipsoid should line up every time, right? I mean, if the globe is using the ellipsoid to model the mesh, and Globe.pick is based on a mesh/pick-ray intersection. I guess what we're seeing is some "fudge factor" where the actual mesh is contained by the ellipsoid but doesn't actually reach it, since zooming in reduces this discrepancy and eventually removes it -- zoom in to maybe a state or city-sized region and the Globe.pick altitude shows up as (negative) 0m. Using the Ion terrain provider changes the numbers somewhat, but the trend (more negative when zoomed out further) remains.

All that to say: am I making a mistake somewhere, or is it just not possible to get a realistic terrain-height-under-cursor right now?

@hpinkos

This comment has been minimized.

Copy link
Contributor Author

@hpinkos hpinkos commented Jul 23, 2018

@thw0rted that is expected behavior. Globe.pick is retrieving the height of whatever terrain LOD is currently visible on the globe. When you're zoomed out far, the terrain tiles are very low resolution, hence the negative altitude values.

TerriaJS (a library that uses both Cesium and leaflet) has a mouse over coordinate tool that does a fast approximation of height using globe.pickTriangle, then gets the actual terrain height using sampleTerrainMostDetailed. It displays the approximate height until the result from sampleTerrainMostDetailed is returned. I would recommend doing something similar. See https://github.com/TerriaJS/terriajs/blob/master/lib/ReactViewModels/MouseCoords.js

@thw0rted

This comment has been minimized.

Copy link

@thw0rted thw0rted commented Jul 23, 2018

I was looking at something similar on my own, actually, but it turns out the terrain provider I'm using at the moment (an older in-house Google Earth server) doesn't support availability and that's required for sampleTerrainMostDetailed. I could use sampleTerrain but I didn't know a good way to guesstimate the appropriate level to pass, so I put it on the back burner. I just want to know how tall stuff is :-/

ETA: I had a bit of a poke around, it looks like globe.pickTriangle no longer exists -- would that have been any more acurate than globe.pick, in terms of getting a sane height when zoomed out significantly?

@hpinkos

This comment has been minimized.

Copy link
Contributor Author

@hpinkos hpinkos commented Aug 31, 2018

See #6990 for a great code example. pickPosition works decently well on the ellipsoid when globe.depthTestAgainstTerrain = true but doesn't work at all otherwise.

@drmoose

This comment has been minimized.

Copy link

@drmoose drmoose commented Oct 24, 2018

For others who find this bug because pickPosition is wildly inaccurate and seems to change based on the camera position, on my system (Linux / Firefox / nvidia) the entire problem turned out to be CanvasBlocker intentionally tampering with the WebGL APIs that Cesium uses to query the depth buffer to prevent that from being used for fingerprinting.

@hpinkos

This comment has been minimized.

Copy link
Contributor Author

@hpinkos hpinkos commented Jan 24, 2019

Also reported by @hieeyh in #7506

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
You can’t perform that action at this time.