Planet build import all fails #217
Comments
|
What was the output when you ran |
|
I don't have it recorded now but it didn't show any errors and was successful. seems fine. Also looks OK except that it didn't write anything into it. |
|
Exactly! I've been struggling with this one as well! Docker invocation for I tried Thoughts: elasticsearch URL or network is off or pelias.json is not properly mapped into |
This comment was marked as spam.
This comment was marked as spam.
|
I'm getting a similar error as @xiaofengilove. |
|
@xiaofengilove Placing an empty file in the blacklist folder named osm.txt resolved the parser errors for me. |
|
I had got the error from the first post running Pelias in Docker together with Elasticsearch instance on the host running at port 9200. My docker-compose.yml have the following part: But pelias command and |
|
Hi, the same error, only when importing whosonfirst data: |
|
I just comment this string |
|
I'm hitting an identical looking error with a planet build, and similarly, everything seems to work fine with the smaller portland build. Given the long time it takes before the command errors (+2minutes), maybe (just a guess) the root problem is hitting the 120s timeout and it's being surfaced in a confusing way. output# Sanity check that I'd created the index:
$ time pelias elastic create
--------------
create index
--------------
[resource_already_exists_exception] index [pelias/FuIGtngnT5SiGwgBUl5Nyw] already exists, with { index_uuid="FuIGtngnT5SiGwgBUl5Nyw" & index="pelias" }
real 0m13.242s
user 0m0.102s
sys 0m0.018s
$ time pelias import all
ERROR: Elasticsearch index pelias does not exist
You must use the pelias-schema tool (https://github.com/pelias/schema/) to create the index first
For full instructions on setting up Pelias, see http://pelias.io/install.html
/code/pelias/whosonfirst/node_modules/pelias-dbclient/src/configValidation.js:39
throw new Error(`elasticsearch index ${config.schema.indexName} does not exist`);
^
Error: elasticsearch index pelias does not exist
at existsCallback (/code/pelias/whosonfirst/node_modules/pelias-dbclient/src/configValidation.js:39:15)
at respond (/code/pelias/whosonfirst/node_modules/elasticsearch/src/lib/transport.js:368:9)
at /code/pelias/whosonfirst/node_modules/elasticsearch/src/lib/transport.js:396:7
at Timeout. (/code/pelias/whosonfirst/node_modules/elasticsearch/src/lib/transport.js:429:7)
at listOnTimeout (internal/timers.js:554:17)
at processTimers (internal/timers.js:497:7)
real 2m16.389s
user 0m0.084s
sys 0m0.029s
$ time pelias import all
ERROR: Elasticsearch index pelias does not exist
You must use the pelias-schema tool (https://github.com/pelias/schema/) to create the index first
For full instructions on setting up Pelias, see http://pelias.io/install.html
/code/pelias/whosonfirst/node_modules/pelias-dbclient/src/configValidation.js:39
throw new Error(`elasticsearch index ${config.schema.indexName} does not exist`);
^
Error: elasticsearch index pelias does not exist
at existsCallback (/code/pelias/whosonfirst/node_modules/pelias-dbclient/src/configValidation.js:39:15)
at respond (/code/pelias/whosonfirst/node_modules/elasticsearch/src/lib/transport.js:368:9)
at /code/pelias/whosonfirst/node_modules/elasticsearch/src/lib/transport.js:396:7
at Timeout. (/code/pelias/whosonfirst/node_modules/elasticsearch/src/lib/transport.js:429:7)
at listOnTimeout (internal/timers.js:554:17)
at processTimers (internal/timers.js:497:7)
real 2m49.125s
user 0m0.071s
sys 0m0.044s
This is running from within a checkout of https://github.com/pelias/docker, [^slightly modified] to use a different planet build and disable interpolation, but all the software versions are as specified as of 6c50a65 While trying to debug this issue a couple weeks ago, I was running the command repeatedly, and it eventually just worked. But it's generally quite reproducible for me (unfortunately 😆), so let me know if there's any debugging information I can get to you. [slightly modified]
|
A corroborating data point: After having the planet sized import fail a couple dozen times with the default 2 minute timeout, I specified a timeout of 10 minutes and was able to complete the import on the first try. pelias config: |
|
Really strange. I am able to confirm the issue as well on a more-or-less fresh clone (removed interpolation but that's about it). As noted by others above, curl instantly shows that the index is indeed there. For some reason, it just takes a LONG time to do something at the start of the wof import. I can confirm that bumping the timeout to 10mins as @michaelkirk does above appears to resolve the issue. We aren't running on quite as beefy a server as the core team seems to be, but we have 12 cores and 64GB RAM, and it's essentially idle except for one node thread until getting over the initial hump. I'm commenting here since this is the more active thread, but I suspect the issue is with the wof importer rather than anything specific to the docker repo. |
|
Even with the latest pelias/docker version (in July 2023) I still run into this issue (that the wof import for a planet setup causing 'ERROR: Elasticsearch index pelias does not exist')... After I have added to my pelias.json the 10min timeout (thanks @michaelkirk!) the importer started after a short while - there is still a error reported (but after that output it looks like the wof import is running fine): |
This comment was marked as spam.
This comment was marked as spam.
|
I think the issue is that here https://github.com/pelias/dbclient/blob/master/src/configValidation.js#L34 |
This comment was marked as spam.
This comment was marked as spam.
Hi,
We are trying to setup a full planet build. All steps succeed until we get to run "pelias import all" which fails with:
The smaller portland-metro build works find.
The text was updated successfully, but these errors were encountered: