Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Geo: Polygon based filter #294

Closed
kimchy opened this issue Aug 3, 2010 · 8 comments
Closed

Geo: Polygon based filter #294

kimchy opened this issue Aug 3, 2010 · 8 comments

Comments

@kimchy
Copy link
Member

kimchy commented Aug 3, 2010

A filter allowing to include hits that only fall within a polygon of points. Here is an example:

{
    "filtered" : {
        "query" : {
            "match_all" : {}
        },
        "filter" : {
            "geo_polygon" : {
                "person.location" : {
                    "points" : [
                        {"lat" : 40, "lon" : -70},
                        {"lat" : 30, "lon" : -80},
                        {"lat" : 20, "lon" : -90}
                    ]
                }
            }
        }
    }
}

Other formats supported:

{
    "filtered" : {
        "query" : {
            "match_all" : {}
        },
        "filter" : {
            "geo_polygon" : {
                "person.location" : {
                    "points" : [
                        [40, -70],
                        [30, -80],
                        [20, -90]
                    ]
                }
            }
        }
    }
}

and,

{
    "filtered" : {
        "query" : {
            "match_all" : {}
        },
        "filter" : {
            "geo_polygon" : {
                "person.location" : {
                    "points" : [
                        "40, -70",
                        "30, -80",
                        "20, -90"
                    ]
                }
            }
        }
    }
}

and

{
    "filtered" : {
        "query" : {
            "match_all" : {}
        },
        "filter" : {
            "geo_polygon" : {
                "person.location" : {
                    "points" : [
                        "drn5x1g8cu2y",
                        "30, -80",
                        "20, -90"
                    ]
                }
            }
        }
    }
}
@kimchy
Copy link
Member Author

kimchy commented Aug 3, 2010

Geo: Polygon based filter, closed by 959eb0e.

@otisg
Copy link

otisg commented Aug 9, 2010

What is that "drn5x1g8cu2y" in the last example? Thanks.

@kimchy
Copy link
Member Author

kimchy commented Aug 9, 2010

Thats a geohash.

@richardsyeo
Copy link

I am having a problem getting the geo location stuff to work with v0.15 on windows 7 x64. Sun JDK 1.6.0.20.

I have done many things successfully with elasticsearch but just struggling with this. Also why is pin.location required, i.e. why is pin outer required? I was following examples.

Remember on windows need to use " instead of '. The " need to be repeated 3 times to escape.

The following command returns success...

\gnu\bin\curl\curl -XPUT "http://localhost:9200/scratch/test/12345" -d "{"""pin""":{"""MyId""":12345,"""location""":{"""type""":"""geo_point""","""lat""":40.12,"""lon""":-71.34}}}"

The following errors

\gnu\bin\curl\curl -XGET "http://localhost:9200/scratch/test/_search?pretty=true" -d "{"""filtered""":{"""query""":{"""match_all""":{}},"""filter""":{"""geo_polygon""":{"""pin.location""":{"""points""":[{"""lat""":40,"""lon""":-70},{"""lat""":30,"""lon""":-80},{"""lat""":20,"""lon""":-90}]}}}}}"

{
"error" : "SearchPhaseExecutionException[Failed to execute phase [query], total failure; shardFailures {[AlmTJezeRUir7yIqVQ5cWA][s
cratch][0]: SearchParseException[[scratch][0]: from[-1],size[-1]: Parse Failure [Failed to parse source [{"filtered":{"query":{
"match_all":{}},"filter":{"geo_polygon":{"pin.location":{"points":[{"lat":40,"lon":-70},{"lat":30,"lon":-80},{"lat
":20,"lon":-90}]}}}}}]]]; nested: SearchParseException[[scratch][0]: from[-1],size[-1]: Parse Failure [No parser for element [fil
tered]]]; }{[AlmTJezeRUir7yIqVQ5cWA][scratch][1]: SearchParseException[[scratch][1]: from[-1],size[-1]: Parse Failure [Failed to par
se source [{"filtered":{"query":{"match_all":{}},"filter":{"geo_polygon":{"pin.location":{"points":[{"lat":40,"lon
":-70},{"lat":30,"lon":-80},{"lat":20,"lon":-90}]}}}}}]]]; nested: SearchParseException[[scratch][1]: from[-1],size[-1]: Par
se Failure [No parser for element [filtered]]]; }{[AlmTJezeRUir7yIqVQ5cWA][scratch][2]: SearchParseException[[scratch][2]: from[-1],
size[-1]: Parse Failure [Failed to parse source [{"filtered":{"query":{"match_all":{}},"filter":{"geo_polygon":{"pin.loca
tion":{"points":[{"lat":40,"lon":-70},{"lat":30,"lon":-80},{"lat":20,"lon":-90}]}}}}}]]]; nested: SearchParseExceptio
n[[scratch][2]: from[-1],size[-1]: Parse Failure [No parser for element [filtered]]]; }{[AlmTJezeRUir7yIqVQ5cWA][scratch][3]: Search
ParseException[[scratch][3]: from[-1],size[-1]: Parse Failure [Failed to parse source [{"filtered":{"query":{"match_all":{}},
"filter":{"geo_polygon":{"pin.location":{"points":[{"lat":40,"lon":-70},{"lat":30,"lon":-80},{"lat":20,"lon":-90}
]}}}}}]]]; nested: SearchParseException[[scratch][3]: from[-1],size[-1]: Parse Failure [No parser for element [filtered]]]; }{[AlmTJ
ezeRUir7yIqVQ5cWA][scratch][4]: SearchParseException[[scratch][4]: from[-1],size[-1]: Parse Failure [Failed to parse source [{"filt
ered":{"query":{"match_all":{}},"filter":{"geo_polygon":{"pin.location":{"points":[{"lat":40,"lon":-70},{"lat":30
,"lon":-80},{"lat":20,"lon":-90}]}}}}}]]]; nested: SearchParseException[[scratch][4]: from[-1],size[-1]: Parse Failure [No par
ser for element [filtered]]]; }]",
"status" : 500
}

[2011-03-05 13:04:26,294][DEBUG][action.search.type ] [Stonecutter] [scratch][4], node[AlmTJezeRUir7yIqVQ5cWA], [P], s[STARTED
]: Failed to execute [org.elasticsearch.action.search.SearchRequest@9c4dfd]
org.elasticsearch.search.SearchParseException: [scratch][4]: from[-1],size[-1]: Parse Failure [Failed to parse source [{"filtered":{
"query":{"match_all":{}},"filter":{"geo_polygon":{"pin.location":{"points":[{"lat":40,"lon":-70},{"lat":30,"lon":-80},{"lat":20,"lon
":-90}]}}}}}]]
at org.elasticsearch.search.SearchService.parseSource(SearchService.java:416)
at org.elasticsearch.search.SearchService.createContext(SearchService.java:331)
at org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:165)
at org.elasticsearch.search.action.SearchServiceTransportAction.sendExecuteQuery(SearchServiceTransportAction.java:132)
at org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction$AsyncAction.sendExecuteFirstPhase(TransportSearc
hQueryThenFetchAction.java:76)
at org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeActio
n.java:191)
at org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.access$000(TransportSearchTypeAction.java:
75)
at org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction$1.run(TransportSearchTypeAction.java:150)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
Caused by: org.elasticsearch.search.SearchParseException: [scratch][4]: from[-1],size[-1]: Parse Failure [No parser for element [fil
tered]]
at org.elasticsearch.search.SearchService.parseSource(SearchService.java:401)
... 10 more

@kimchy
Copy link
Member Author

kimchy commented Mar 8, 2011

Can you create a recreation and gist it, and then post it on the mailing list? (check http://www.elasticsearch.org/help).

@richardsyeo
Copy link

Got it working eventually. Unfortunately the examples were not clear / misleading / being new to elasticsearch.

We found a subsequent comment somewhere which mentioned that geo_point no longer automatically detected.

I manually defined the index and specified type as geo_point. Extra " because via windows command line...

{"""location""":{"type":"""geo_point"""}}

I didn't wrap with outer "pin".

I then imported data.

We then had to wrap the "filtered" with a "query":{ } which is not shown in examples.

Here is example of query for windows command line which returns matches for UK and Ireland...

\curl -XGET "http://localhost:9200/scratch/test/_search?pretty=true" -d "{"""query""":{"""filtered""":{"""query""":{"""match_all""":{}},"""filter""":{"""geo_bounding_box""":{"""location""":{"""top_left""":{"""lat""":60,"""lon""":-10},"""bottom_right""":{"""lat""":50,"""lon""":-0}}}}}}}"

Useful website for working out long and lats.
http://www.getlatlon.com/

@kimchy
Copy link
Member Author

kimchy commented Mar 9, 2011

Great. In the geo polygon filter it states that the geo_type is required: http://www.elasticsearch.org/guide/reference/query-dsl/geo-polygon-filter.html.

@richardsyeo
Copy link

Thanks. As a newbie to elasticsearch I wasn't clear what this meant. I tried passing as part of query explicitly. If examples were a little more complete and included actual curl command would greatly assist for novices. ElasticSearch is looking very promising!

williamrandolph pushed a commit to williamrandolph/elasticsearch that referenced this issue Jun 4, 2020
Since in many cases Pig internal assumptions force the use of 1 reducer,
no matter the job properties or the number of splits, document the
settings that disable this behavior (and which unfortunately cannot be
set directly by es-hadoop).

fix elastic#294
mindw pushed a commit to mindw/elasticsearch that referenced this issue Sep 5, 2022
…ull request elastic#294)

Initial support for ARM based EMS EC2 instances

Add support for c6g type instances so we could start validating AWS claims for
better price performance (allegedly 40%):

* Added "--arch" build flag.
* EMS AMI names include architecture: packer-ems-<amd64|arm64>-<version>-<date>-<time>
* Terraform infers the architecture based on the instance family - it it ends
  with "g" it will look for arm64 AMIs.
* Updated provisioner scripts that assumed amd64 architecture. In some cases,
  took a shortcut for now: dequeue does not support ARM because of lack of
  chrome builds, temporarily skipping cloud watch agent.
* Also: delete_ami.py can delete multiple AMI IDs


Approved-by: fabien
costin added a commit that referenced this issue Dec 6, 2022
Rearrange some classes into the process
Introduce small additional infra classes such as FilterExec
costin added a commit that referenced this issue Dec 6, 2022
#294 tries to improve field extraction however it doesn't seem to be
working. This PR logically reverts the PR and temporary brings back the
old behavior.

See #352
This issue was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants