Expose click event with 2D and 3D information

[oscarlorentzon]

There will be need for exposing a click event for example for placing things on a map based on the position clicked in the image.

The event can contain both 2D information about where in the image the click occured (basic coordinates [x, y]) and where in the 3D world (geographic coordinates [lat, lon, alt]).

The nature of the image planes makes the 3D positioning inaccurate.

To discuss:

Do we need to provide 3D coordinates for clicks everywhere in the image?
Can we limit to intersections with the ground plane in some way?
Should we use clicks in multiple images to improve positioning?

[gyllen]

@paulinus @jesolem Any ideas what to expose in this click?

[jesolem]

Do we need to provide 3D coordinates for clicks everywhere in the image?

I don't think we need to do that. But then you need some visual feedback that clicking where you just clicked will not work.

Can we limit to intersections with the ground plane in some way?

I'm OK with this for map markers. BUT for things like measuring, which we will support at some point, you will need to be able to do intersections anywhere off the ground plane. Think about @paulinus mark and measure demo.

Should we use clicks in multiple images to improve positioning?

Yes. Click in one image will give a ray in space and [lat,lon] that could be used to show a marker on the map. Clicking in a second image with the same "object identifier" (= you're still working on the same object) gives a second ray in space and an improved [lat,lon].

Re "rays in space", these is the blue lines in @paulinus demo and I think it would be great if we could show them in the image, just like in that demo. Possibly also expose them as a line in [lat,lon] for showing 2D projection on map. Cyclomedia/Blom does rays on the map and I think people are used to that.

[oscarlorentzon]

In the first iteration of a mouse event API the idea is the following:

Expose the following mouse events:

click
contextmenu
dblclick
mousedown
mousemove
mouseout
mouseover
mouseup

The data structure emitted when those events are fired will look like this:

interface IViewerMouseEvent {
    basicPoint: number[]; // Basic image coordinate
    latLon: ILatLon; // Latitude longitude
    pixelPoint: number[]; // Pixel position on viewer canvas
    originalEvent: MouseEvent; // Original DOM event
    target: Viewer; // The viewer instance
    type: string; // Type of the event. One of the above 
}

Notes regarding the data structure.

1. For the current iteration we will only give a latLon based on an intersection of the unprojected mouse with an approximated ground plane 2 meters below the camera. This will work well in flat areas where the reconstruction is good. It will not work well in hilly terrains and where the reconstruction is not good / tilted. We will improve on this at a later time.

2. We will not provide an altitude value in the event data currently.

3. If there is no intersection with the ground plane or the intersectionis to far away from the camera (e.g. more than 100 meters) we will not provide a latLon in the event data. The latLon property will then be set to null. Library users will need to handle this.

4. We will provide the basic coordinates in the image that the mouse position corresponds to. If the mouse position is outside of the image the basicPoint property will be set to null. The library user will need to handle this.

5. If the click is outside of the image both the latLon and the basicPoint will always be set to null.

This version of the mouse will be targeted for placing markers on the ground, not for measuring. That will come at a later point in time.

Based on the behavior of the data structure described above it is possible for the library user to give visual indications about when a click -> latLon will work or not.

Apart from the mouse events the idea is to also add an unproject method on the viewer for converting a pixel position to a latLon. Then, e.g. when a moveend event is emitted, it is possible to get a latLon from a canvas position at any time without getting an actual mouse event.

Q: Should the unproject method return the latLon only or also the basicPoint? Calculating the basic point is needed for ensuring that the pixel position is within the image. I'm leaning towards including the basicPoint in the unproject result although it is technically a projection.

[gyllen]

@oscarlorentzon Looks good and I think we answered all questions on this mornings meeting.

[oscarlorentzon]

Implemented according to comment above for 2.5.0. Will create a new issues in the future for improve positioning by clicking in multiple images, better estimation of ground plane as well as for providing altitude when we have improved estimates.