Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding osm extract #1545

Closed
akarsh opened this issue May 20, 2017 · 4 comments
Closed

Adding osm extract #1545

akarsh opened this issue May 20, 2017 · 4 comments

Comments

@akarsh
Copy link

akarsh commented May 20, 2017

I had followed the installation instructions and then i am stuck at adding osm extract.
I have a .osm file and how to add it ?
I had read the configuration steps but it was for .osm.pbf file

May 20, 2017 10:48:35 AM org.openstreetmap.osmosis.core.Osmosis run
INFO: Osmosis Version 0.45
May 20, 2017 10:48:35 AM org.openstreetmap.osmosis.core.Osmosis run
INFO: Preparing pipeline.
May 20, 2017 10:48:35 AM org.openstreetmap.osmosis.core.Osmosis run
INFO: Launching pipeline execution.
May 20, 2017 10:48:35 AM org.openstreetmap.osmosis.core.Osmosis run
INFO: Pipeline executing, waiting for completion.
May 20, 2017 10:48:35 AM org.openstreetmap.osmosis.core.pipeline.common.ActiveTaskManager waitForCompletion
SEVERE: Thread for task 1-read-xml failed
org.openstreetmap.osmosis.core.OsmosisRuntimeException: Unable to establish a database connection.
	at org.openstreetmap.osmosis.apidb.common.DatabaseContext.getPostgresConnection(DatabaseContext.java:113)
	at org.openstreetmap.osmosis.apidb.common.DatabaseContext.getConnection(DatabaseContext.java:82)
	at org.openstreetmap.osmosis.apidb.common.DatabaseContext.prepareStatement(DatabaseContext.java:346)
	at org.openstreetmap.osmosis.apidb.v0_6.ApidbWriter.initialize(ApidbWriter.java:351)
	at org.openstreetmap.osmosis.apidb.v0_6.ApidbWriter.process(ApidbWriter.java:1095)
	at org.openstreetmap.osmosis.xml.v0_6.impl.NodeElementProcessor.end(NodeElementProcessor.java:139)
	at org.openstreetmap.osmosis.xml.v0_6.impl.OsmHandler.endElement(OsmHandler.java:107)
	at org.apache.xerces.parsers.AbstractSAXParser.endElement(Unknown Source)
	at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanEndElement(Unknown Source)
	at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl$FragmentContentDispatcher.dispatch(Unknown Source)
	at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanDocument(Unknown Source)
	at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
	at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
	at org.apache.xerces.parsers.XMLParser.parse(Unknown Source)
	at org.apache.xerces.parsers.AbstractSAXParser.parse(Unknown Source)
	at org.apache.xerces.jaxp.SAXParserImpl$JAXPSAXParser.parse(Unknown Source)
	at org.apache.xerces.jaxp.SAXParserImpl.parse(Unknown Source)
	at javax.xml.parsers.SAXParser.parse(SAXParser.java:189)
	at org.openstreetmap.osmosis.xml.v0_6.XmlReader.run(XmlReader.java:90)
	at java.base/java.lang.Thread.run(Thread.java:844)
Caused by: org.postgresql.util.PSQLException: The server requested password-based authentication, but no password was provided.
	at org.postgresql.core.v3.ConnectionFactoryImpl.doAuthentication(ConnectionFactoryImpl.java:444)
	at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:173)
	at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:64)
	at org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:136)
	at org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:29)
	at org.postgresql.jdbc3.Jdbc3Connection.<init>(Jdbc3Connection.java:24)
	at org.postgresql.Driver.makeConnection(Driver.java:393)
	at org.postgresql.Driver.connect(Driver.java:267)
	at java.sql/java.sql.DriverManager.getConnection(DriverManager.java:678)
	at java.sql/java.sql.DriverManager.getConnection(DriverManager.java:229)
	at org.openstreetmap.osmosis.apidb.common.DatabaseContext.getPostgresConnection(DatabaseContext.java:103)
	... 19 more

May 20, 2017 10:48:35 AM org.openstreetmap.osmosis.core.Osmosis main
SEVERE: Execution aborted.
org.openstreetmap.osmosis.core.OsmosisRuntimeException: One or more tasks failed.
	at org.openstreetmap.osmosis.core.pipeline.common.Pipeline.waitForCompletion(Pipeline.java:146)
	at org.openstreetmap.osmosis.core.Osmosis.run(Osmosis.java:92)
	at org.openstreetmap.osmosis.core.Osmosis.main(Osmosis.java:37)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:547)
	at org.codehaus.plexus.classworlds.launcher.Launcher.launchStandard(Launcher.java:328)
	at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:238)
	at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:408)
	at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:351)
	at org.codehaus.classworlds.Launcher.main(Launcher.java:31)
@zerebubuth
Copy link
Contributor

Osmosis is able to read both XML and PBF files, but you need to change the command line to use --read-xml rather than --read-pbf. For example:

osmosis --read-xml greater-london-latest.osm \
  --write-apidb host="localhost" database="openstreetmap" \
  user="openstreetmap" password="" validateSchemaVersion="no"

However, the error in your example is:

The server requested password-based authentication, but no password was provided.

Which would suggest that you need to check that your database authentication is set up correctly. The most common reason for this is that the database user has no password set up at all and is using "ident" authentication over a local socket. Please read the PostgreSQL documentation on authentication and make sure you can login using psql -h localhost -u openstreetmap openstreetmap, then use the same credentials for Osmosis.

Please also have a look at the conversation around #282, which might have some useful tips. And, if you plan to edit the data afterwards, then update the primary key sequences to avoid issues such as #1542.

@akarsh
Copy link
Author

akarsh commented May 21, 2017

@zerebubuth
I changed to
osmosis --read-xml file=/home/test/Desktop/changedfiles/m1_fokus_e00_changed.osm \ --write-apidb host="localhost" database="openstreetmap" \ user="openstreetmap" password="" validateSchemaVersion="no"

and i cannot login
test@linuxtest:~/openstreetmap-website$ psql -h localhost -U openstreetmap openstreetmap Password for user openstreetmap: psql: fe_sendauth: no password supplied
Did not find helpful information from the two issues.

@tomhughes
Copy link
Member

This is really a question for IRC or the mailing lists and not this bug tracker as it isn't an issue with the code, you just need help with using osmosis, which is nothing to do with this repository.

@akarsh
Copy link
Author

akarsh commented Jul 1, 2017

I solved the issue with the database.
It was mainly due to the role & username.

Now i am facing the issue of key id already exists; when i am trying to populate the database.
`aseggemu data $ osmosis --read-xml fokus_e00.osm \

--write-apidb host="localhost" database="openstreetmap"
user="openstreetmap" password="openstreetmap" validateSchemaVersion="no"
Jul 01, 2017 2:38:23 PM org.openstreetmap.osmosis.core.Osmosis run
INFO: Osmosis Version 0.45
Jul 01, 2017 2:38:23 PM org.openstreetmap.osmosis.core.Osmosis run
INFO: Preparing pipeline.
Jul 01, 2017 2:38:23 PM org.openstreetmap.osmosis.core.Osmosis run
INFO: Launching pipeline execution.
Jul 01, 2017 2:38:23 PM org.openstreetmap.osmosis.core.Osmosis run
INFO: Pipeline executing, waiting for completion.
Jul 01, 2017 2:38:25 PM org.openstreetmap.osmosis.core.pipeline.common.ActiveTaskManager waitForCompletion
SEVERE: Thread for task 1-read-xml failed
org.openstreetmap.osmosis.core.OsmosisRuntimeException: Unable to load current nodes.
at org.openstreetmap.osmosis.apidb.v0_6.ApidbWriter.populateCurrentNodes(ApidbWriter.java:935)
at org.openstreetmap.osmosis.apidb.v0_6.ApidbWriter.populateCurrentTables(ApidbWriter.java:1036)
at org.openstreetmap.osmosis.apidb.v0_6.ApidbWriter.complete(ApidbWriter.java:1070)
at org.openstreetmap.osmosis.xml.v0_6.XmlReader.run(XmlReader.java:92)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.postgresql.util.PSQLException: ERROR: duplicate key value violates unique constraint "current_nodes_pkey1"
Detail: Key (id)=(-39113) already exists.
at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2103)
at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1836)
at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:257)
at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:512)
at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:388)
at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:381)
at org.openstreetmap.osmosis.apidb.v0_6.ApidbWriter.populateCurrentNodes(ApidbWriter.java:932)
... 4 more

Jul 01, 2017 2:38:25 PM org.openstreetmap.osmosis.core.Osmosis main
SEVERE: Execution aborted.
org.openstreetmap.osmosis.core.OsmosisRuntimeException: One or more tasks failed.
at org.openstreetmap.osmosis.core.pipeline.common.Pipeline.waitForCompletion(Pipeline.java:146)
at org.openstreetmap.osmosis.core.Osmosis.run(Osmosis.java:92)
at org.openstreetmap.osmosis.core.Osmosis.main(Osmosis.java:37)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.codehaus.plexus.classworlds.launcher.Launcher.launchStandard(Launcher.java:330)
at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:238)
at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
at org.codehaus.classworlds.Launcher.main(Launcher.java:47)`

I read through some of the issues. But, the solution was not applicable to mine. As, in the osm file there is one node i.e. id="-39113"
Here is the fokus_e00.osm file

@openstreetmap openstreetmap locked and limited conversation to collaborators Jul 1, 2017
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants