Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Postgres database update error by osm2pgsql #1571

Closed
fpintoa opened this issue Sep 17, 2021 · 6 comments
Closed

Postgres database update error by osm2pgsql #1571

fpintoa opened this issue Sep 17, 2021 · 6 comments

Comments

@fpintoa
Copy link

fpintoa commented Sep 17, 2021

We have an error situation in the update process.
At certain times, the update process crashes and does not proceed. The error message in the file /var/log/tiles/osm2pgsql.log is shown below:

Reading in file: /var/lib/mod_tile/changes.osc.gz
Using XML parser.
Processing: Node(10k 0.0k/s) Way(3k 3.66k/s) Relation(40 20.00/s) parse time: 407s
Node stats: total(10020), max(9090048720) in 404s
Way stats: total(3661), max(983093395) in 1s
Relation stats: total(81), max(13209476) in 2s
DB writer thread failed due to ERROR: result COPY_END for planet_osm_polygon failed: ERROR: invalid input > syntax for type integer: "2.75"
CONTEXT: COPY planet_osm_polygon, line 26, column layer: "2.75"

The message seems to indicate that a double value (2.75) is being written to the layer field of the planet_osm_polygon table, which is set to integer.

To work around this, we decreased the value of the maxInterval parameter in the configuration file /var/lib/mod_tile/.osmosis/configuration.txt
And we manually increment the sequenceNumber value of the /var/lib/mod_tile/.osmosis/state.txt file, so that it advances in the update. We believe there may be some information recorded wrong in the update files, which are available at https://planet.openstreetmap.org/replication/minute

However, this is impacting our environment as it requires a lot of technical intervention.

Our environment is Ubuntu 20.04 LTS. The osm2pgsql version is shown below:

osm2pgsql version 1.2.1 (64 bit id space)
Compiled using the following library versions:
Libosmium 2.15.4
Lua 5.2.4

We would appreciate if you can guide us on how to solve the problem.

Yours sincerely,

@fpintoa
Copy link
Author

fpintoa commented Sep 17, 2021

Attached is the graph with the occurrences of the problem in the last 7 days
crash-osm2pgsql-2021-09

@mmd-osm
Copy link
Contributor

mmd-osm commented Sep 17, 2021

It seems the map style you’re using is not fault tolerant enough to correctly deal with a tag „layer=2.75“

You need to fix your style file first before continuing. By the way, this kind of issue would probably be out of scope for this repository.

And we manually increment the sequenceNumber value

That’s not a good idea as it leads to loss of data. I really don’t recommend doing so to work around the issue in your style!

@fpintoa
Copy link
Author

fpintoa commented Sep 17, 2021

What do you mean by non-fault tolerant style? If the field in the planet_osm_polygon table is an integer type, I understand that someone is doing something wrong along the way. Can you point a way for me to solve the problem? Where do you think the problem might be?

Sorry, if the problem is outside the scope of the repository. Where do you suggest I can get help?

Yours sincerely,

@mmd-osm
Copy link
Contributor

mmd-osm commented Sep 17, 2021

Please try https://help.openstreetmap.org/ instead.

@fpintoa
Copy link
Author

fpintoa commented Sep 17, 2021

Thank you.

@mmd-osm
Copy link
Contributor

mmd-osm commented Sep 17, 2021

@joto joto closed this as completed Sep 18, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants