New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use ray picking to query extrusions in "queryRenderedFeatures" #3122

Closed
1ec5 opened this Issue Sep 1, 2016 · 16 comments

Comments

Projects
None yet
@1ec5
Copy link
Member

1ec5 commented Sep 1, 2016

With 3D polygon extrusions (mapbox/mapbox-gl-style-spec#456) on a tilted map, feature eventing will continue to return the same results regardless of any extrusion. That is, even if the query point visually falls upon a building wall, the results will reflect whatever would happen to be rendered at ground level had the building wall been absent (which may be the footprint of a different building). Ideally, querying a point on screen would return the building (due to any of its walls), the building across the street, and so on until reaching the ground.

Typically, interactive 3D environments implement ray picking. Ray picking would probably be more straightforward to implement if GL JS would position each component of the map in a 3D coordinate space. For better or worse, GL JS instead treats the entire map as a flat surface, modulo any transforms. So I don’t know that we’ll be able to take advantage of usual approaches to ray picking.

/cc @lbud @peterqliu @ansis

@1ec5 1ec5 added the feature 🍏 label Sep 1, 2016

@lucaswoj lucaswoj changed the title Ray picking for feature eventing of extrusion layers Use ray picking to query extrusions in "queryRenderedFeatures" Sep 2, 2016

@lbud lbud self-assigned this Sep 6, 2016

@samanpwbb

This comment has been minimized.

Copy link
Member

samanpwbb commented Oct 21, 2016

Just ran into this on a demo map:

screen shot 2016-10-21 at 6 42 28 pm
screen shot 2016-10-21 at 6 42 31 pm

@1ec5

This comment has been minimized.

Copy link
Member Author

1ec5 commented Oct 27, 2016

Ray picking would need to be optional, because there are also use cases that call for the current behavior, for example finding the height of a building at a particular geographic coordinate (which would be mapped to a screen coordinate before querying).

@holmesal

This comment has been minimized.

Copy link

holmesal commented Oct 28, 2016

+1 - I'm running into this issue while trying to create hoverable extrusions.

Users generally hover over the extruded body, but queryRenderedFeatures only returns features when the cursor is inside the "footprint" polygon.

This doesn't return the desired feature (pink arrow == mouse position):
image

This does return the desired feature:
image

@andrewharvey

This comment has been minimized.

Copy link
Collaborator

andrewharvey commented Feb 22, 2017

Typically, interactive 3D environments implement ray picking. Ray picking would probably be more straightforward to implement if GL JS would position each component of the map in a 3D coordinate space. For better or worse, GL JS instead treats the entire map as a flat surface, modulo any transforms. So I don’t know that we’ll be able to take advantage of usual approaches to ray picking.

Is this where you render another pass with each object a unique colour, then simply lookup the x/y pixel in that buffer and map back from the color to the object id?

UPDATE: As pointed out by @pbabik this would only return the "topmost hit", Some applications might want the features behind at that point too (eg. if you have transparent buildings, clicking might let you cycle through which one you want to select).

@pbabik

This comment has been minimized.

Copy link

pbabik commented Feb 22, 2017

That's how feature selection works in OpenLayers when Canvas renderer is in use. But as I understand this will allow for "topmost hit" selection only (still better than current functionality)

@cmarfil

This comment has been minimized.

Copy link

cmarfil commented May 2, 2017

From what I've been able to investigate, mapbox dosn't work if you need to select buildings/extrusions or add hover effects on them... It's a shame..

@billyc

This comment has been minimized.

Copy link

billyc commented Jun 11, 2017

It's very counterintuitive to click an extrusion and have the building behind it get selected. Or three our four behind it, depending on the pitch! It breaks any semblance of "realness" to the 3D objects displayed on the map.

@bravecow

This comment has been minimized.

Copy link

bravecow commented Aug 16, 2017

Hi guys. Please tell, will this be planned for the nearest time? Or this feature is not a priority? Thanks.

@nicooprat

This comment has been minimized.

Copy link

nicooprat commented Sep 1, 2017

@bravecow Looks like it's coming in the next few months: https://www.mapbox.com/mapbox-gl-js/roadmap/

@mushon

This comment has been minimized.

Copy link

mushon commented Apr 9, 2018

It's been a few months now of this issue pending in the roadmap, any chance we can get an updated ETA? Will this bug be fixed together with the #6022 enhancement?

@andrewharvey

This comment has been minimized.

Copy link
Collaborator

andrewharvey commented May 6, 2018

OSMBuildings appears to implement this with the render to a framebuffer with each building in a unique color then just lookup the pixel color and map back to the feature.

https://github.com/OSMBuildings/OSMBuildings/blob/ea70e1f1708c678fdb8301e55660d1b9d0f26736/src/render/Picking.js

@chloekraw

This comment has been minimized.

Copy link
Contributor

chloekraw commented Oct 8, 2018

Hi all - wanted to reassure folks that this is a priority for us and we have been looking into it; however, the solution is not straightforward and may require several more months of effort. Thank you for your patience.

@wangxiaoyu

This comment has been minimized.

Copy link

wangxiaoyu commented Dec 27, 2018

Does the ray picking change come with an on/off switch?

The proposed change was the feature I expect however after read related issues now I have a second thought. Basically the change is to opt for selection on screen visuals instead of selection on a (tilted) map.

By the function name queryRenderedFeatures(0 the change is pretty straightforward. However this change the behavior and could cause unexpected results. Besides how would it interact with fill-extrusion-opacity?

It seems the query result would be an array with the front object as the first element. However if the front feature is visually larger and blocks other features behind it then none of them could be selected, unless the function also return something like "angle-of-view" so that the user could customize selection based on small AOV values. Otherwise in order to select those blocked features the user has to constantly rotate the map to find the right pitch and bearing which would be quite a nuisance, especially in dense areas where small features are surrounded by large features.

kepler.gl has the wanted behavior in its example "New york city population by census tract" and playing with it you can see blocked objects are visible but not selectable. OSM has similar issues for OSM 3D buildings in NYC.

After playing with both I feel the current mapbox implementation is actually ok or even preferred for many use cases such as building selections. In the end the proposed change is a nice enhancement, but not a fix so I wish to have the option to keep the current implementation.

@ansis

This comment has been minimized.

Copy link
Member

ansis commented Jan 8, 2019

@wangxiaoyu thanks for your questions

queryRenderedFeatures will return all features under the mouse pointer, not just the front one. They are sorted in order of closest to furthest but the developer could implement a way of selecting further ones (first click selects the closest, second goes to the next, click again to cycle through all the features).

Or if you want the old behavior (based on the footprints) you can create a regular fill layer under the fill-extrusions and query that instead of the 3d buildings.

While I understand the challenges this API change adds I don't think we want to add an "angle-of-view" option because of the complexity that would introduce.

Would either of the two workarounds work for you?

@ansis ansis closed this in #7499 Jan 9, 2019

@stevage

This comment has been minimized.

Copy link
Contributor

stevage commented Jan 11, 2019

Awesome! Really excited to see this in action!

@KravMaguy

This comment has been minimized.

Copy link

KravMaguy commented Jan 20, 2019

I am just seeing a post I made was referenced here. I am now seeing its an issue thats been discussed for a while, I followed peters instructions on that thread to create this :

fill extrusion hover

As you can see in the gif the tooltip is following the cursor and displaying the extrusion properties correctly, but only on the ground floor. So this is the issue apparently everyone is having.

queryRenderedFeatures will return all features under the mouse pointer, not just the front one. They are sorted in order of closest to furthest but the developer could implement a way of selecting further ones (first click selects the closest, second goes to the next, click again to cycle through all the features).

Yes this is an acceptable solution. I am looking forward to using it, and looking forward to seeing this fixed in ten days. As I was unable to use master branch.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment