Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How to run Pelias geocoder in docker with my own .pbf file data without external data #352

Open
bmyashaswi opened this issue Jul 23, 2024 · 5 comments
Labels
bug Something isn't working

Comments

@bmyashaswi
Copy link

bmyashaswi commented Jul 23, 2024

Describe the bug
I am trying to run Pelias geocoder in Docker using my own .pbf file data. I want to do this without using any external data sources. However, I am encountering issues with the setup.

Steps to Reproduce

Prepare a Docker environment with Pelias.
Use a local .pbf file for data import.
Configure Pelias to use this .pbf file.
Start Pelias using Docker and attempt to import the .pbf file.
Expected behavior
Pelias should successfully import and index the data from my .pbf file and be able to handle geocoding requests using this data.

References

@bmyashaswi bmyashaswi added the bug Something isn't working label Jul 23, 2024
@bmyashaswi bmyashaswi reopened this Jul 23, 2024
@bmyashaswi bmyashaswi closed this as not planned Won't fix, can't repro, duplicate, stale Jul 23, 2024
@bmyashaswi bmyashaswi reopened this Jul 23, 2024
@missinglink
Copy link
Member

This should be possible, you might have an issue accessing the pbf file in the container.

Please post your pelias.json and docker-compose.yml

@bmyashaswi
Copy link
Author

Hi,
Here is my pelias.json file
{
"esclient": {
"hosts": [{
"host": "localhost",
"port": 9200
}]
},
"imports": {
"openstreetmap": {
"datapath": "/media/tech/New Volume/Yashaswi/pelias/pelias_bindu_lite_v1.5.pbf",
"leveldbpath": "/media/tech/New Volume/Yashaswi/pelias/data/osm/leveldb",
"adminLookup": true,
"missingFilesAreFatal": false
}
},
"schema": {
"indexName": "pelias"
},
"api": {
"services": {
"placeholder": {
"url": "http://placeholder:4100"
},
"pip": {
"url": "http://pip:4200"
},
"interpolation": {
"url": "http://interpolation:4300"
}
}
}
}

and here is my docker-compose.yml file
version: '2.1'
services:
elasticsearch:
image: pelias/elasticsearch:7.10.2
environment:
- discovery.type=single-node
- cluster.name=pelias
ports:
- "9200:9200"
volumes:
- pelias_elasticsearch:/usr/share/elasticsearch/data
placeholder:
image: pelias/placeholder
environment:
- PELIAS_CONFIG=/code/pelias.json
volumes:
- /media/tech/yashaswi_pelias/pelias.json:/code/pelias.json
pip:
image: pelias/pip
environment:
- PELIAS_CONFIG=/code/pelias.json
volumes:
- /media/tech/yashaswi_pelias/pelias.json:/code/pelias.json
interpolation:
image: pelias/interpolation
environment:
- PELIAS_CONFIG=/code/pelias.json
volumes:
- /media/tech/yashaswi_pelias/pelias.json:/code/pelias.json
api:
image: pelias/api
environment:
- PELIAS_CONFIG=/code/pelias.json
ports:
- "4000:4000"
volumes:
- /media/tech/yashaswi_pelias/pelias.json:/code/pelias.json
schema:
image: pelias/schema
environment:
- PELIAS_CONFIG=/code/pelias.json
volumes:
- /media/tech/yashaswi_pelias/pelias.json:/code/pelias.json
openstreetmap:
image: pelias/openstreetmap
environment:
- PELIAS_CONFIG=/code/pelias.json
volumes:
- /media/tech/yashaswi_pelias/pelias.json:/code/pelias.json
- /media/tech/yashaswi_pelias:/data
command: import
volumes:
pelias_elasticsearch:

can you tell me the folder structure and procedure to run pelias geocoder in docker without external data.

@missinglink
Copy link
Member

missinglink commented Jul 24, 2024

This mapping - /media/tech/yashaswi_pelias:/data says that the directory on the left will be known as the directory on the right within the container.

So you can leave your docker-comose.yml as it is but change the paths in the pelias.json to point to /data eg:

Change this:

"datapath": "/media/tech/New Volume/Yashaswi/pelias/pelias_bindu_lite_v1.5.pbf"
to:

"datapath": "/data/pelias_bindu_lite_v1.5.pbf" and make sure the file is located at /media/tech/yashaswi_pelias/pelias_bindu_lite_v1.5.pbf on your local machine.

@bmyashaswi
Copy link
Author

I did changes after that I am getting like this error

pelias-api_1 | Elasticsearch WARNING: 2024-07-25T04:38:05Z
pelias-api_1 | Unable to revive connection: http://localhost:9200/
pelias-api_1 |
pelias-api_1 | Elasticsearch WARNING: 2024-07-25T04:38:05Z
pelias-api_1 | No living connections
pelias-api_1 |
pelias-api_1 | 2024-07-25T04:38:05.994Z - error: [api:type_mapping_discovery] message=No Living connections, stack=Error: No Living connections
pelias-api_1 | at sendReqWithConnection (/code/pelias/api/node_modules/elasticsearch/src/lib/transport.js:266:15)
pelias-api_1 | at next (/code/pelias/api/node_modules/elasticsearch/src/lib/connection_pool.js:243:7)
pelias-api_1 | at processTicksAndRejections (node:internal/process/task_queues:78:11)
pelias-api_1 | 2024-07-25T04:38:05.997Z - info: [api] pelias is now running on http://:::3100
pelias-api_1 | 2024-07-25T04:40:12.050Z - info: [api] Pelias API shutting down
pelias-api_1 | 2024-07-25T04:40:23.548Z - warn: [pip] pip service disabled
pelias-api_1 | 2024-07-25T04:40:23.549Z - warn: [placeholder] placeholder service disabled
pelias-api_1 | 2024-07-25T04:40:23.549Z - warn: [language] language service disabled
pelias-api_1 | 2024-07-25T04:40:23.549Z - warn: [interpolation] interpolation service disabled
pelias-api_1 | 2024-07-25T04:40:23.549Z - warn: [libpostal] libpostal service disabled
pelias-api_1 | 2024-07-25T04:40:23.549Z - warn: [libpostal] libpostal service disabled
pelias-api_1 | Elasticsearch ERROR: 2024-07-25T04:40:23Z
pelias-api_1 | Error: Request error, retrying
pelias-api_1 | POST http://localhost:9200/_search?request_cache=true&timeout=10s => connect ECONNREFUSED 127.0.0.1:9200
pelias-api_1 | at Log.error (/code/pelias/api/node_modules/elasticsearch/src/lib/log.js:239:56)
pelias-api_1 | at checkRespForFailure (/code/pelias/api/node_modules/elasticsearch/src/lib/transport.js:298:18)
pelias-api_1 | at HttpConnector. (/code/pelias/api/node_modules/elasticsearch/src/lib/connectors/http.js:171:7)
pelias-api_1 | at ClientRequest.wrapper (/code/pelias/api/node_modules/lodash/lodash.js:4991:19)
pelias-api_1 | at ClientRequest.emit (node:events:513:28)
pelias-api_1 | at Socket.socketErrorListener (node:_http_client:494:9)
pelias-api_1 | at Socket.emit (node:events:513:28)
pelias-api_1 | at emitErrorNT (node:internal/streams/destroy:157:8)
pelias-api_1 | at emitErrorCloseNT (node:internal/streams/destroy:122:3)
pelias-api_1 | at processTicksAndRejections (node:internal/process/task_queues:83:21)
pelias-api_1 |
pelias-api_1 | Elasticsearch WARNING: 2024-07-25T04:40:23Z
pelias-api_1 | Unable to revive connection: http://localhost:9200/
pelias-api_1 |
pelias-api_1 | Elasticsearch WARNING: 2024-07-25T04:40:23Z
pelias-api_1 | No living connections
pelias-api_1 |
pelias-api_1 | 2024-07-25T04:40:23.571Z - error: [api:type_mapping_discovery] message=No Living connections, stack=Error: No Living connections
pelias-api_1 | at sendReqWithConnection (/code/pelias/api/node_modules/elasticsearch/src/lib/transport.js:266:15)
pelias-api_1 | at next (/code/pelias/api/node_modules/elasticsearch/src/lib/connection_pool.js:243:7)
pelias-api_1 | at processTicksAndRejections (node:internal/process/task_queues:78:11)
pelias-api_1 | 2024-07-25T04:40:23.573Z - info: [api] pelias is now running on http://:::3100
root@tech-HP-280-Pro-G6-Microtower-PC:/media/tech/New Volume/Yashaswi/pelias# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
051573476fee pelias/api:latest "./bin/start" 4 minutes ago Up 2 minutes 0.0.0.0:3100->3100/tcp, :::3100->3100/tcp pelias_pelias-api_1
a0a052f18b70 docker.elastic.co/elasticsearch/elasticsearch:7.15.0 "/bin/tini -- /usr/l…" 17 minutes ago Up 17 minutes 0.0.0.0:9200->9200/tcp, :::9200->9200/tcp, 9300/tcp pelias_elasticsearch_1
root@tech-HP-280-Pro-G6-Microtower-PC:/media/tech/New Volume/Yashaswi/pelias# docker ps ls
"docker ps" accepts no arguments.
See 'docker ps --help'.

Usage: docker ps [OPTIONS]

List containers
root@tech-HP-280-Pro-G6-Microtower-PC:/media/tech/New Volume/Yashaswi/pelias# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
051573476fee pelias/api:latest "./bin/start" 4 minutes ago Up 2 minutes 0.0.0.0:3100->3100/tcp, :::3100->3100/tcp pelias_pelias-api_1
a0a052f18b70 docker.elastic.co/elasticsearch/elasticsearch:7.15.0 "/bin/tini -- /usr/l…" 18 minutes ago Up 18 minutes 0.0.0.0:9200->9200/tcp, :::9200->9200/tcp, 9300/tcp pelias_elasticsearch_1
root@tech-HP-280-Pro-G6-Microtower-PC:/media/tech/New Volume/Yashaswi/pelias# ^C

@missinglink
Copy link
Member

This is a huge blob of text, please format your comments so that they are legible.

You've only posted part of the docker-compose.yml file above, please post all of it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants