Update does not work

[etresoft]

I tried to run an update task and received the following failure:

make update-osm area=planet

Starting postgres docker compose target using default image (no recreate if exists)
docker-compose run --rm --user=1000:1000 openmaptiles-tools sh -c 'pgwait && import-update'
Creating openmaptiles_openmaptiles-tools_run ... done
[2021-11-12T00:35:03Z] 0:00:00 [fatal] Unable to read last.state.txt:open /import/last.state.txt: no such file or directory
ERROR: 1
make: *** [Makefile:395: update-osm] Error 1

It seems odd that it requires a hard-coded path like /import. I tried creating it, and an empty last.state.txt but no change.

[TomPohys]

Hi, the import folder is defined in docker-compose.yml.

openmaptiles/docker-compose.yml

Line 51 in d7ea453

- ./data:/import

The last.state.txt should be in data folder. Did you adjust DIFF_MODE=true in .env before the first import data?

[etresoft]

No. I did not. Perhaps that should be added to the update documentation.

[TomPohys]

Yes, it could be. PR is very welcome.

[etresoft]

I'm not submitting a PR until I have it working. I have no idea if it is working or not. My update task has been running for about 19 days now. Is that wrong? Should it be doing something else? Does the update task actually end or is it continuous? I'm asking at a fundamental level here. I simply don't know. Has anyone else ever run update? If so, how long does it take? What does it look like? How does it behave? Is there some other setting that wasn't documented that I've missed? So many questions....

[amenk]

@etresoft What did you try so far? Which country are you downloading?

[amenk]

I did the following (working on Ethiopia)

• set DIFF_MODE=true in .env

• delete old data/ethiopia.osm.pbf

• make download area=ethiopia

• make import-osm

• make update-osm
  Getting message:

  [2022-01-15T15:12:46Z] 0:00:00 [info] Starting replication from https://download.geofabrik.de/africa/ethiopia-updates/ with 24h0m0s interval

Which output did you get? which interval did you use?

EDIT: Problem here is that geofabrik has only daily diffs for Ethiopia ... I would like to have minutely diffs

[amenk]

@etresoft Which behind-time does the script output? If this is much more then a day, probably something is wrong or your initial download was too old?

I tried it with Eritrea (I need still to find minutely diffs for Ethiopia) and it works flawlessly

[etresoft]

@amenk To answer your questions...

I have only tried what I mentioned above.
It was a planet update.
I don't remember the output. That was two months ago.
I didn't specify any interval. Either I didn't get that far or didn't know where or how this was supposed to be provided.
I don't remember any specific "behind-time" output for the script. If I recall, it was just zero values repeated forever. Nothing indicating any success, or failure for that matter.

It will be some time before I could attempt this again. I'm still processing a new planet import using the latest build.

I appreciate the updates to the instructions. I'll try it again later with a small country just so I can see what the output is supposed to look like. However, I do require planet-wide data. I don't need daily updates. Given that it takes a week to import planet-wide data, that is plenty of time to checksum tiles from the previous update to find any updated tiles.

[amenk]

@etresoft I think it if is zero values, it's all fine - it will just run forever. Maybe it already was working for you :-)

[etresoft]

@amenk The documentation specifically said, "After each run you should also have a list of tiles that have updated." Is that different now? Is there some kind of trigger for new tile availability?

[amenk]

@etresoft It writes some *.tiles files which you summarize to a tiles.txt, see here: https://github.com/OpenPlaceGuide/openmaptiles/blob/doc-updates/UPDATE.md#generate-changed-tiles

[2803media]

For some clarification when make start-update-osm is running do we need to launch make "generate-changed-tiles" after or do the mbtiles are already updated?

Thanks for this explanation

[TomPohys]

Hi @2803media, if the start-update-osm is running, it is not possible to generate-changed-tiles due to locks (https://github.com/openmaptiles/openmaptiles/blob/master/UPDATE.md#note)

The start-update-osm will store a list of updated tiles in text format (https://github.com/openmaptiles/openmaptiles/blob/master/UPDATE.md#keep-database-updated).

Then make generate-changed-tiles take a list, generates the tiles, and updates the existing MBTiles file. (https://github.com/openmaptiles/openmaptiles/blob/master/UPDATE.md#generate-changed-tiles).

I hope that helps.

If you feel, that more explanation is needed, please reopen this issue.