Skip to content
This repository has been archived by the owner. It is now read-only.

Invalid character in GPX track description or changeset comments produces bad XML in API response #4382

Open
openstreetmap-trac opened this issue Jul 23, 2021 · 11 comments

Comments

@openstreetmap-trac
Copy link

@openstreetmap-trac openstreetmap-trac commented Jul 23, 2021

Reporter: don-vip
[Submitted to the original trac issue database at 2.42pm, Saturday, 28th April 2012]

Hi,

We have a JOSM bug (#josm7648) that should be fixed on OSM server side as it will impact all applications processing XML API responses.

This track contains an invalid character in its description:
http://www.openstreetmap.org/user/CornyJoe/traces/1209300

(0x001A is not the expected German character). This cause XML parsers to fail when downloading GPX data in this area.

Is it possible to ensure GPX tracks descriptions replied by the API contain only valid XML characters ?

Thanks

@openstreetmap-trac
Copy link
Author

@openstreetmap-trac openstreetmap-trac commented Jul 23, 2021

Author: simon04
[Added to the original trac issue at 8.04am, Tuesday, 8th May 2012]

The linked track does not contain the <desc> element. Instead, consider http://api.openstreetmap.org/api/0.6/trackpoints?bbox=10.038045799999999,49.7493614,10.0412616,49.750925599999995&page=0 and see <desc>Schnellstrae (15.0m)</desc> (around line 1246) and note the Ux001A character.

Notice that Ux001A is not an allowed XML character.

@openstreetmap-trac
Copy link
Author

@openstreetmap-trac openstreetmap-trac commented Jul 23, 2021

Author: bastik
[Added to the original trac issue at 9.56pm, Saturday, 15th September 2012]

Any update on this one?

@openstreetmap-trac
Copy link
Author

@openstreetmap-trac openstreetmap-trac commented Jul 23, 2021

Author: TomH
[Added to the original trac issue at 11.01pm, Saturday, 15th September 2012]

If there was I imagine the person fixing it would have commented here.

@openstreetmap-trac
Copy link
Author

@openstreetmap-trac openstreetmap-trac commented Jul 23, 2021

Author: stoecker
[Added to the original trac issue at 10.13am, Sunday, 14th April 2013]

We get lots of reports on this issue.

@openstreetmap-trac
Copy link
Author

@openstreetmap-trac openstreetmap-trac commented Jul 23, 2021

Author: TomH
[Added to the original trac issue at 11.33am, Sunday, 14th April 2013]

Bumping the priority is not going to produce a magic code elf to fix it though - if it's important to you then a patch is what will get things changed, not a metadata change on a bug.

@openstreetmap-trac
Copy link
Author

@openstreetmap-trac openstreetmap-trac commented Jul 23, 2021

Author: don-vip
[Added to the original trac issue at 11.50am, Sunday, 14th April 2013]

We just have implemented a workaround, as it appears unlikely to see this bug fixed someday.

@openstreetmap-trac
Copy link
Author

@openstreetmap-trac openstreetmap-trac commented Jul 23, 2021

Author: stoecker
[Added to the original trac issue at 3.57pm, Sunday, 14th April 2013]

Replying to [comment:5 TomH]:

Bumping the priority is not going to produce a magic code elf to fix it though - if it's important to you then a patch is what will get things changed, not a metadata change on a bug.

Increasing priority of duplicate reports actually is normal.

We now have a workaround, but this issue still is not minor. The server delivers invalid XML.

@openstreetmap-trac
Copy link
Author

@openstreetmap-trac openstreetmap-trac commented Jul 23, 2021

Author: TomH
[Added to the original trac issue at 4.52pm, Sunday, 14th April 2013]

Well it turns out that this is very odd in fact... We are using libxml to return this so I find it hard to believe that it isn't doing the right escaping, but that does seem to be what is happening. Here is our code:

https://github.com/openstreetmap/openstreetmap-website/blob/master/app/controllers/api_controller.rb#L71

which is just invoking << on an XML::Node object, which is documented here:

http://xml4r.github.io/libxml-ruby/rdoc/classes/LibXML/XML/Node.html#method-i-3C-3C

in turn that calls the xmlNodeAddContent function in libxml:

http://xmlsoft.org/html/libxml-tree.html#xmlNodeAddContent

according to which the content being added should be "raw text, so unescaped XML special chars are allowed" which suggests that it should do the escaping.

@openstreetmap-trac
Copy link
Author

@openstreetmap-trac openstreetmap-trac commented Jul 23, 2021

Author: TomH
[Added to the original trac issue at 4.55pm, Sunday, 14th April 2013]

Of course I also don't understand why that trace has U+1A there... I assume it is meant to be but I can't imagine that any character set would have that at 0x1a.

@openstreetmap-trac
Copy link
Author

@openstreetmap-trac openstreetmap-trac commented Jul 23, 2021

Author: don-vip
[Added to the original trac issue at 4.38pm, Thursday, 30th January 2014]

There's another example of bad xml delivered by OSM API. Try to get history of node 415524175:

http://www.openstreetmap.org/node/415524175/history

You will see that comment text of changeset 1424555 (created by Potlatch 1.0) contains exotic characters. Then our XML parser is unable to parse it:

https://josm.openstreetmap.de/ticket/9647

Is there hope to see this bug fixed inside OSM API or do we need again to find a workaround ?

@openstreetmap-trac
Copy link
Author

@openstreetmap-trac openstreetmap-trac commented Jul 23, 2021

Author: mmd
[Added to the original trac issue at 6.15pm, Tuesday, 19th May 2020]

Issue is fixed on JOSM side already: https://josm.openstreetmap.de/ticket/9647

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
1 participant