New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Java heap space Error on document indexing with spatial field #16444

Open
matthesrieke opened this Issue Feb 4, 2016 · 4 comments

Comments

Projects
None yet
4 participants
@matthesrieke

matthesrieke commented Feb 4, 2016

Hi Elasticsearch team,

I have recently migrated from ES 1.7.x to ES 2.x. I came across a bug that seem to be related to the storage of spatial mappings. I have seen this behaviour on ES 2.0.x and the latest 2.2.0. In 1.7.x this worked without any issues. I am running Ubuntu 15.10 64bit with ES 2.2.0 installed from the elastic PPA. This also occured on a Ubuntu 14.04 64bit installation.

I have attached the json files to recreate the issue. Here are the curl requests using the json files as the payload content.

curl -XDELETE localhost:9200/test

curl -vX PUT localhost:9200/test -d @create-index_broken.json --header "Content-Type: application/json"

curl -vX PUT localhost:9200/test/entry/124 -d @put-record_working.json --header "Content-Type: application/json"
(this takes about 6 seconds on my machine)

curl -vX PUT localhost:9200/test/entry/123 -d @put-record_broken.json --header "Content-Type: application/json"

The spatial extent is bigger than in the previous request. This takes some time, and eventually I get the following error:

{
    "error": {
        "root_cause": [
            {
                "type": "index_failed_engine_exception",
                "reason": "Index failed for [entry#123]",
                "shard": "0",
                "index": "test"
            }
        ],
        "type": "index_failed_engine_exception",
        "reason": "Index failed for [entry#123]",
        "shard": "0",
        "index": "test",
        "caused_by": {
            "type": "out_of_memory_error",
            "reason": "Java heap space"
        }
    },
    "status": 500
}

When I create the index with a larger precision (1000m), both records are inserted in acceptable time:

curl -vX PUT localhost:9200/test -d @create-index_working.json --header "Content-Type: application/json"

The json files are attached as a zip:
spatial-index-bug.zip

@matthesrieke

This comment has been minimized.

Show comment
Hide comment
@matthesrieke

matthesrieke Feb 4, 2016

/var/log/elasticsearch/elasticsearch.log does not provide too much information, still here are the entries starting with index creation:

spatial-index-bug_log.txt

matthesrieke commented Feb 4, 2016

/var/log/elasticsearch/elasticsearch.log does not provide too much information, still here are the entries starting with index creation:

spatial-index-bug_log.txt

@dakrone

This comment has been minimized.

Show comment
Hide comment
@dakrone

dakrone Feb 4, 2016

Member

@nknize maybe you could take a look at this?

Member

dakrone commented Feb 4, 2016

@nknize maybe you could take a look at this?

@nknize

This comment has been minimized.

Show comment
Hide comment
@nknize

nknize Feb 5, 2016

Member

Hi @matthesrieke

I'll try to reproduce the issue myself in a bit. In the meantime, if you have a chance try explicitly setting distance_error_pct : 0.025.

Member

nknize commented Feb 5, 2016

Hi @matthesrieke

I'll try to reproduce the issue myself in a bit. In the meantime, if you have a chance try explicitly setting distance_error_pct : 0.025.

@nknize nknize self-assigned this Feb 5, 2016

@nknize nknize added the :Search/Geo label Feb 5, 2016

@matthesrieke

This comment has been minimized.

Show comment
Hide comment
@matthesrieke

matthesrieke Feb 9, 2016

hi @nknize
thanks for investigating. indeed, setting distance_error_pct (even to a very small double, e.g. 0.0001) seems to resolve the memory error and improve performance!

matthesrieke commented Feb 9, 2016

hi @nknize
thanks for investigating. indeed, setting distance_error_pct (even to a very small double, e.g. 0.0001) seems to resolve the memory error and improve performance!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment