Parsing issue #5

Closed
tyczj opened this Issue Jul 20, 2012 · 2 comments

Comments

Projects
None yet
3 participants
@tyczj

tyczj commented Jul 20, 2012

I am trying to figure out how to use the java parser with a .pbf file from geofafrik.de but I am having a problem getting the nodes correctly. I give the BlockInputStream the file and the parser sends back the HeaderBlock to the header method and such but when the parseDense method gets called with the nodes it does not give me the all the nodes correctly.

when I loop through the list of nodes to get the id, latitude and longitude the very first node is correct but after that it starts giving me numbers that make no sense and are not even in the .pbf file as a node. I can confirm that by using osmconvert to convert the .pbf file to a .osm xml file so I can see what is in the file.

to give you an example of what I mean, the first "node" of each iteration will give me something like this

id="38707804"
lat="41.8179677"
lon="-71.3800996"

which would be correct

but the next "node" in the list will be this

id="1"
lat="235862"
lon="-765386"

which is not correct and never appears in the .osm xml that I can look at. The next time the parseDense method gets called the same thing happens where the first "node" will be correct but everyone after that wont be.

also the parseNodes method never gets called for me only the parseDense method.

Is there something I am doing wrong or some extra steps I have to do to get all of the nodes?

@Diomas

This comment has been minimized.

Show comment Hide comment
@Diomas

Diomas Dec 11, 2012

I see parseNodes() is called few times while parsing. But I don't get any items inside parseNodes() too. So all nodes data I get from parseDense() calls.

As for values you get in parseDense() - these are not exact values but their offsets from the previous values. So parseDense() supposed to be something like this:

            protected void parseDense(DenseNodes nodes) {
                long id = 0;
                long lat = 0;
                long lon = 0;
                int count = nodes.getIdCount();
                for( int i = 0; i < count; i ++ )
                {
                    id += nodes.getId(i);
                    lat += nodes.getLat(i);
                    lon += nodes.getLon(i);
                    System.out.println("\tid: " + id + " lat: " + lat + " lon: " + lon);
                }
            }

So the question is: why do we need parseNodes at all?

Diomas commented Dec 11, 2012

I see parseNodes() is called few times while parsing. But I don't get any items inside parseNodes() too. So all nodes data I get from parseDense() calls.

As for values you get in parseDense() - these are not exact values but their offsets from the previous values. So parseDense() supposed to be something like this:

            protected void parseDense(DenseNodes nodes) {
                long id = 0;
                long lat = 0;
                long lon = 0;
                int count = nodes.getIdCount();
                for( int i = 0; i < count; i ++ )
                {
                    id += nodes.getId(i);
                    lat += nodes.getLat(i);
                    lon += nodes.getLon(i);
                    System.out.println("\tid: " + id + " lat: " + lat + " lon: " + lon);
                }
            }

So the question is: why do we need parseNodes at all?

@joto

This comment has been minimized.

Show comment Hide comment
@joto

joto Sep 2, 2013

Collaborator

There are two different ways of storing nodes in PBF files, usually "dense nodes" is used, because it is more space efficient. You need parseDense() for those "dense nodes" and parseNodes() for "normal nodes".

Collaborator

joto commented Sep 2, 2013

There are two different ways of storing nodes in PBF files, usually "dense nodes" is used, because it is more space efficient. You need parseDense() for those "dense nodes" and parseNodes() for "normal nodes".

@joto joto closed this Sep 2, 2013

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment