Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

source exclusion mapping prevents geo shape coordinates to be returned in query result source field #2944

Closed
fxh opened this issue Apr 26, 2013 · 4 comments
Closed

source exclusion mapping prevents geo shape coordinates to be returned in query result source field #2944

fxh opened this issue Apr 26, 2013 · 4 comments
Assignees
@spinscale
Labels
>bug v0.90.2 v1.0.0.Beta1

Comments

@fxh
Copy link

fxh commented Apr 26, 2013

ES Version: 0.90.0.RC2

Steps to reproduce:

ok case, without source exclusion:

curl -XDELETE 'http://localhost:9200/geo_test'
curl -XPUT 'http://localhost:9200/geo_test'
curl -XPUT 'http://localhost:9200/geo_test/item/_mapping' -d '{
    "item" : {
        "properties" : {
            "location" : {
                "type" : "object",
                "properties": {
                     "point": {"type": "geo_point"},
                     "area": {"type": "geo_shape"}
                }
            }
        }
    }
}'
curl -XPUT 'http://localhost:9200/geo_test/item/1' -d '{
    "location": {"point": [45.0, 45.0]}
}'
curl -XPUT 'http://localhost:9200/geo_test/item/2' -d '{
    "location": {
        "area": {
            "type" : "envelope",
            "coordinates" : [[44.0, 46.0], [45.0, 45.0]]
        }
    }
}'
curl -XPOST 'http://localhost:9200/geo_test/item/_search?pretty' -d '{
    "query": {"match_all": {}}
}'

returns the coordinates for geo_point and geo_shape items:

{
    "took": 1,
    "timed_out": false,
    "_shards": {
        "total": 1,
        "successful": 1,
        "failed": 0
    },
    "hits": {
        "total": 2,
        "max_score": 1.0,
        "hits": [{
            "_index": "geo_test",
            "_type": "item",
            "_id": "1",
            "_score": 1.0,
            "_source": {
                "location": {
                    "point": [45.0, 45.0]
                }
            }
        }, {
            "_index": "geo_test",
            "_type": "item",
            "_id": "2",
            "_score": 1.0,
            "_source": {
                "location": {
                    "area": {
                        "type": "envelope",
                        "coordinates": [
                            [45.0, 45.0],
                            [44.0, 46.0]
                        ]
                    }
                }
            }
        }]
    }
}

however the same scenario but with a source exclusion mapping of an arbitrary field does not return the geo_shape coordinates any longer:

curl -XDELETE 'http://localhost:9200/geo_test'
curl -XPUT 'http://localhost:9200/geo_test'
curl -XPUT 'http://localhost:9200/geo_test/item/_mapping' -d '{
    "item": {
        "_source": {
            "excludes": ["body"]
        },
        "properties" : {
            "location" : {
                "type" : "object",
                "properties": {
                     "point": {"type": "geo_point"},
                     "area": {"type": "geo_shape"}
                }
            }
        }
    }
}'
curl -XPUT 'http://localhost:9200/geo_test/item/1' -d '{
    "location": {"point": [45.0, 45.0]}
}'
curl -XPUT 'http://localhost:9200/geo_test/item/2' -d '{
    "location": {
        "area": {
            "type" : "envelope",
            "coordinates" : [[44.0, 46.0], [45.0, 45.0]]
        }
    }
}'
curl -XPOST 'http://localhost:9200/geo_test/item/_search?pretty' -d '{
    "query": {"match_all": {}}
}'

returns only the geo_point coordinates but not the geo_shape coordinates:

{
    "took": 1,
    "timed_out": false,
    "_shards": {
        "total": 1,
        "successful": 1,
        "failed": 0
    },
    "hits": {
        "total": 2,
        "max_score": 1.0,
        "hits": [{
            "_index": "geo_test",
            "_type": "item",
            "_id": "1",
            "_score": 1.0,
            "_source": {
                "location": {
                    "point": [45.0, 45.0]
                }
            }
        }, {
            "_index": "geo_test",
            "_type": "item",
            "_id": "2",
            "_score": 1.0,
            "_source": {
                "location": {
                    "area": {
                        "type": "envelope",
                        "coordinates": []
                    }
                }
            }
        }]
    }
}
@spinscale
Copy link
Contributor

Hey,

this looks really strange.. I managed to reproduce it with a smaller example and will try to take a look at it

curl -XDELETE 'http://localhost:9200/geo_test'
curl -XPUT 'http://localhost:9200/geo_test'
curl -XPUT 'http://localhost:9200/geo_test/item/_mapping' -d '{
    "item": {
        "_source": { "excludes": ["body"] },
        "properties" : {
            "area": {"type": "geo_shape"}
        }
    }
}'

curl -XPUT 'http://localhost:9200/geo_test/item/1?refresh=true' -d '{
    "area": {
        "type" : "envelope",
        "coordinates" : [[-45.0, 45.0], [45.0, -45.0]]
    }
}'

curl -XPOST 'http://localhost:9200/geo_test/item/_search?pretty' -d '{
    "query": {"match_all": {}}
}'

curl http://localhost:9200/geo_test/item/1

As soon as the source exclude is omitted in the mapping, everything is working again. The GET on the id also works as expected.

@ghost ghost assigned spinscale Apr 30, 2013
spinscale added a commit to spinscale/elasticsearch that referenced this issue Apr 30, 2013
@spinscale
XContentMapValues.filter now works with nested arrays
1cc8b48 
The filter method of XContentMapValues actually filtered out nested
arrays/lists completely due to a bug in the filter method, which threw
away all data inside of such an array.

Closes elastic#2944
This bug was a follow up problem, because of the filtering of nested arrays
in case source exclusion was configured.
@spinscale spinscale mentioned this issue Apr 30, 2013
Closed
@fxh
Copy link
Author

fxh commented Jun 3, 2013

Hi, I was just trying this in ES 0.90.1 and I still get the same error as reported above. Could you please reopen this issues and doublecheck again? Thanks a lot.

@spinscale
Copy link
Contributor

@fxh hey, will reopen it and recheck as soon as possible

@spinscale spinscale reopened this Jun 3, 2013
spinscale added a commit that referenced this issue Jun 5, 2013
@spinscale
XContentMapValues.filter now works with nested arrays
906ff27 
The filter method of XContentMapValues actually filtered out nested
arrays/lists completely due to a bug in the filter method, which threw
away all data inside of such an array.

Closes #2944
This bug was a follow up problem, because of the filtering of nested arrays
in case source exclusion was configured.
@spinscale
Copy link
Contributor

@fxh we did not include the fix in the 0.90 branch, even though I thought so. I have just pushed it into the 0.90 release branch so it will be included in the next release. Sorry for the inconvenience and thanks for bringing it up again!

mute pushed a commit to mute/elasticsearch that referenced this issue Jul 29, 2015
@spinscale
XContentMapValues.filter now works with nested arrays
5594920 
The filter method of XContentMapValues actually filtered out nested
arrays/lists completely due to a bug in the filter method, which threw
away all data inside of such an array.

Closes elastic#2944
This bug was a follow up problem, because of the filtering of nested arrays
in case source exclusion was configured.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@spinscale spinscale

Labels
>bug v0.90.2 v1.0.0.Beta1
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

XContentMapValues.filter now works with nested arrays spinscale/elasticsearch
2 participants
@spinscale @fxh