Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

source exclusion mapping prevents geo shape coordinates to be returned in query result source field #2944

Closed
fxh opened this Issue Apr 26, 2013 · 4 comments

Comments

Projects
None yet
2 participants
@fxh
Copy link

fxh commented Apr 26, 2013

ES Version: 0.90.0.RC2

Steps to reproduce:

ok case, without source exclusion:

curl -XDELETE 'http://localhost:9200/geo_test'
curl -XPUT 'http://localhost:9200/geo_test'
curl -XPUT 'http://localhost:9200/geo_test/item/_mapping' -d '{
    "item" : {
        "properties" : {
            "location" : {
                "type" : "object",
                "properties": {
                     "point": {"type": "geo_point"},
                     "area": {"type": "geo_shape"}
                }
            }
        }
    }
}'
curl -XPUT 'http://localhost:9200/geo_test/item/1' -d '{
    "location": {"point": [45.0, 45.0]}
}'
curl -XPUT 'http://localhost:9200/geo_test/item/2' -d '{
    "location": {
        "area": {
            "type" : "envelope",
            "coordinates" : [[44.0, 46.0], [45.0, 45.0]]
        }
    }
}'
curl -XPOST 'http://localhost:9200/geo_test/item/_search?pretty' -d '{
    "query": {"match_all": {}}
}'

returns the coordinates for geo_point and geo_shape items:

{
    "took": 1,
    "timed_out": false,
    "_shards": {
        "total": 1,
        "successful": 1,
        "failed": 0
    },
    "hits": {
        "total": 2,
        "max_score": 1.0,
        "hits": [{
            "_index": "geo_test",
            "_type": "item",
            "_id": "1",
            "_score": 1.0,
            "_source": {
                "location": {
                    "point": [45.0, 45.0]
                }
            }
        }, {
            "_index": "geo_test",
            "_type": "item",
            "_id": "2",
            "_score": 1.0,
            "_source": {
                "location": {
                    "area": {
                        "type": "envelope",
                        "coordinates": [
                            [45.0, 45.0],
                            [44.0, 46.0]
                        ]
                    }
                }
            }
        }]
    }
}

however the same scenario but with a source exclusion mapping of an arbitrary field does not return the geo_shape coordinates any longer:

curl -XDELETE 'http://localhost:9200/geo_test'
curl -XPUT 'http://localhost:9200/geo_test'
curl -XPUT 'http://localhost:9200/geo_test/item/_mapping' -d '{
    "item": {
        "_source": {
            "excludes": ["body"]
        },
        "properties" : {
            "location" : {
                "type" : "object",
                "properties": {
                     "point": {"type": "geo_point"},
                     "area": {"type": "geo_shape"}
                }
            }
        }
    }
}'
curl -XPUT 'http://localhost:9200/geo_test/item/1' -d '{
    "location": {"point": [45.0, 45.0]}
}'
curl -XPUT 'http://localhost:9200/geo_test/item/2' -d '{
    "location": {
        "area": {
            "type" : "envelope",
            "coordinates" : [[44.0, 46.0], [45.0, 45.0]]
        }
    }
}'
curl -XPOST 'http://localhost:9200/geo_test/item/_search?pretty' -d '{
    "query": {"match_all": {}}
}'

returns only the geo_point coordinates but not the geo_shape coordinates:

{
    "took": 1,
    "timed_out": false,
    "_shards": {
        "total": 1,
        "successful": 1,
        "failed": 0
    },
    "hits": {
        "total": 2,
        "max_score": 1.0,
        "hits": [{
            "_index": "geo_test",
            "_type": "item",
            "_id": "1",
            "_score": 1.0,
            "_source": {
                "location": {
                    "point": [45.0, 45.0]
                }
            }
        }, {
            "_index": "geo_test",
            "_type": "item",
            "_id": "2",
            "_score": 1.0,
            "_source": {
                "location": {
                    "area": {
                        "type": "envelope",
                        "coordinates": []
                    }
                }
            }
        }]
    }
}
@spinscale

This comment has been minimized.

Copy link
Member

spinscale commented Apr 30, 2013

Hey,

this looks really strange.. I managed to reproduce it with a smaller example and will try to take a look at it

curl -XDELETE 'http://localhost:9200/geo_test'
curl -XPUT 'http://localhost:9200/geo_test'
curl -XPUT 'http://localhost:9200/geo_test/item/_mapping' -d '{
    "item": {
        "_source": { "excludes": ["body"] },
        "properties" : {
            "area": {"type": "geo_shape"}
        }
    }
}'

curl -XPUT 'http://localhost:9200/geo_test/item/1?refresh=true' -d '{
    "area": {
        "type" : "envelope",
        "coordinates" : [[-45.0, 45.0], [45.0, -45.0]]
    }
}'

curl -XPOST 'http://localhost:9200/geo_test/item/_search?pretty' -d '{
    "query": {"match_all": {}}
}'

curl http://localhost:9200/geo_test/item/1

As soon as the source exclude is omitted in the mapping, everything is working again. The GET on the id also works as expected.

@ghost ghost assigned spinscale Apr 30, 2013

spinscale added a commit to spinscale/elasticsearch that referenced this issue Apr 30, 2013

XContentMapValues.filter now works with nested arrays
The filter method of XContentMapValues actually filtered out nested
arrays/lists completely due to a bug in the filter method, which threw
away all data inside of such an array.

Closes elastic#2944
This bug was a follow up problem, because of the filtering of nested arrays
in case source exclusion was configured.

@spinscale spinscale closed this in d5f4c82 Apr 30, 2013

@fxh

This comment has been minimized.

Copy link
Author

fxh commented Jun 3, 2013

Hi, I was just trying this in ES 0.90.1 and I still get the same error as reported above. Could you please reopen this issues and doublecheck again? Thanks a lot.

@spinscale

This comment has been minimized.

Copy link
Member

spinscale commented Jun 3, 2013

@fxh hey, will reopen it and recheck as soon as possible

@spinscale spinscale reopened this Jun 3, 2013

spinscale added a commit that referenced this issue Jun 5, 2013

XContentMapValues.filter now works with nested arrays
The filter method of XContentMapValues actually filtered out nested
arrays/lists completely due to a bug in the filter method, which threw
away all data inside of such an array.

Closes #2944
This bug was a follow up problem, because of the filtering of nested arrays
in case source exclusion was configured.
@spinscale

This comment has been minimized.

Copy link
Member

spinscale commented Jun 5, 2013

@fxh we did not include the fix in the 0.90 branch, even though I thought so. I have just pushed it into the 0.90 release branch so it will be included in the next release. Sorry for the inconvenience and thanks for bringing it up again!

@spinscale spinscale closed this Jun 5, 2013

mute pushed a commit to mute/elasticsearch that referenced this issue Jul 29, 2015

XContentMapValues.filter now works with nested arrays
The filter method of XContentMapValues actually filtered out nested
arrays/lists completely due to a bug in the filter method, which threw
away all data inside of such an array.

Closes elastic#2944
This bug was a follow up problem, because of the filtering of nested arrays
in case source exclusion was configured.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.