Support geo_hex aggregations over geo_shape fields #90026
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
craigtaverner
added
>feature
:Analytics/Geo
|
Hi @craigtaverner, I've created a changelog YAML for you. |
craigtaverner
force-pushed
the
geo_shape_hex_grids
branch
from
cbb57c4
to
d37df87
Compare
13 tasks
|
Hi @craigtaverner, I've created a changelog YAML for you. |
craigtaverner
force-pushed
the
geo_shape_hex_grids
branch
2 times, most recently
from
92e88df
to
d195f11
Compare
craigtaverner
commented
Oct 3, 2022
| * 2.0. | ||
| */ | ||
|
|
||
| package org.apache.lucene.spatial3d.geom; |
There was a problem hiding this comment.
We cannot have lucene packages in Elastic. We only did this to get around the fact that the class we extend is not public, but instead we should make a copy of the relevant non-public lucene code inside ES, and make an issue in Lucene to make that code public. Later once Lucene is available with the changes, we can remove the temporary copy.
craigtaverner
force-pushed
the
geo_shape_hex_grids
branch
2 times, most recently
from
0a324d8
to
0cd45bd
Compare
Tests still fail, so a lot is still needed here.
The GeoS2Shape class was almost completely what was needed for H3 hexagons, and while it was possible to create a GeoH3Shape class, this more general form could be ported back into Lucene and used for both S2 and H3.
Previously the generation of innner/outer points from the H3 cells was failing for high latitudes due to using mercator space for calculating interpolations, which was inaccurate.
Most h3 cells are convex, and we can use optimized code, but cells that cross icosohedron edges have additional points and sides and are concave, and optimizations fail, so we revert to the full triangulated polygon solution.
We cannot use lucene package names due to java modules. Instead we need to get around the issue with extending a lucene non-public class by copying it into the ES code-base. This should be temporary as we wish to either move the GeoRegalarConvexPolygon into Lucene, or make the Lucene GeoBaseAreaShape public.
This is to ensure that aggregations that fill MVT tiles for Kibana, do not exceed the 64k bucket limit.
We have now both planar and spherical coordinate versions of H3LatLonGeometry. The Spherical one is used for geo-point for backwards compatibility, while the planar version is used for geo-shape to improve the visual alignment on mercator maps like Kibana.
craigtaverner
force-pushed
the
geo_shape_hex_grids
branch
from
0cd45bd
to
9dc2606
Compare
When we moved from Spherical to mercator we discussed the variation in scalefactor required to cover all child cells was much higher for some cells. As a result we've implemented a mechanism to use some hard-coded scale factors for the extreme outliers
Since not all tiles need to inflate the same, and some stand out, in order to cover children, we need a mechanism to inflate the edge cases more. This is particularly true when using planar coordinates due to increased distortion.
|
This prototype was replaced by a new version at #91956 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Support geo_hex aggregations over geo_shape fields