Proposal of a Common Opencaching GPX extension, see #344 #511

following5 · 2017-09-14T17:45:17Z

No description provided.

following5 · 2017-09-14T18:05:01Z

@opencaching/opencaching-pl-lead-programmers
@teiling88
This is a proposal for an own Opencaching GPX extension, which would allow to put Opencaching-specific information into GPX files. It starts with an OKAPI implementation of log uuids (OKAPI issue #344) and a "requires password" field for caches (see OC.de forum topic). The latter one would allow to show password requirement e.g. in c:geo, when the cache was imported via GPX.

I suggest do deposit the XML schema definition at https://opencaching.pl/okapi/static/ocgpx. So it would be maintained in the OKAPI project, which ensures stability and compatibility across all OC sites and versions, while the reference file is at the PL site which is always most current due to automatic OKAPI deployment.

Would you support this OC GPX extension on your websites, i.e. in native GPX generation in OCPL / OCDE code? I may also do the implementations for that.

Do you have any comments or suggestions? I am not experienced in creating XML Schemas, just collected the information needed from tutorials and from existing GPX extensions.

following5 · 2017-09-14T18:11:57Z

Example output:

<gpx
...
    xmlns:oc="https://opencaching.pl/okapi/static/ocgpx/1"
    xsi:schemaLocation="
...
        https://opencaching.pl/okapi/static/ocgpx/1
        https://opencaching.pl/okapi/static/ocgpx/1/ocgpx.xsd
>
<wpt lat="45.020567" lon="7.037033">
    <groundspeak:cache archived="False" available="True" id="1">
        <groundspeak:name>My first cache</groundspeak:name>
...
        <groundspeak:log id="33">
            <groundspeak:date>2017-09-02T21:26:00+02:00</groundspeak:date>
            <groundspeak:type>Didn't find it</groundspeak:type>
...
            <oc:log>
                <oc:uuid>426A3DF7-CF33-D83F-EB68-63711932E09F</oc:uuid>
            </oc:log>
        </groundspeak:log>
    </groundspeak:cache>
    <oc:cache>
        <oc:req_passwd>true</req_passwd>
    </oc:cache>

following5 · 2017-09-14T18:32:06Z

@rsudev, @SammysHP:
Would you support this in c:geo? E.g. evaluate the OC req_passwd flag from GPX files?

following5 · 2017-09-14T19:03:55Z

okapi/static/ocgpx/1/ocgpx.xsd

+
+    <xsd:complexType name="cache_type">
+        <xsd:sequence>
+            <xsd:element name="logpw" type="xsd:boolean" minOccurs="1" maxOccurs="1" />


This must be renamed to "req_passwd".

BTW - worth considering - perhaps we could replace this be a more generic A-code attribute element, and introduce a new A-code for this purpose?

Since A-codes are quite solid and backward compatible, developers could implement this logic by checking if a particular A-code is present. Also, the users of application which already display attributes, would get this information independently displayed, without the need of the developer implementing support for the new A-code.

(I was quite sure we already had an A-code for "requires passoword", but I checked, and it seems we don't.)

Some things to consider:

There is already a way to deliver OC attributes in GPX, which is understood by some applications; it's called attrs=gc_ocde:attrs in OKAPI.

The only major OKAPI App - c:geo - already processes the password requirement as indicated by the req_passwd flag of services/caches/geocache(s). Don't know how they display it.

It may be a good idea to show the password requirement in a more prominent way than an attribute. Because if you are in the field and forget to write down the password, you will not be able to log the cache later. OCPL therefore prominently inserst a text into the GPX description text which points out the password reqirement.

It would be the first OKAPI-only attribute, because passwords are stored as strings, while OC attributes are stored as boolean flags. So it would need some change in the OKAPI attribute logic, and it could not be set in the same way like other attributes (see the new services/caches/edit method).

The user's requests that lead to this proposal was to transfer the "requires password" flag from www.opencaching.de via GPX to c:geo.

All things considered, I would prefer to go for the quick solution and replicate the req_passwd flag from services/caches/geocaches to GPX. Creating a new GPX attributes standard (besides of the already existing GC and OCDE standards) might be persued separately. Will do no harm if the "requires password" information would be available redundantly.

Will do no harm if the "requires password" information would be available redundantly.

Yes, sure.

It's a pity I never found the time to finish the development of A-codes and S-codes. Probably never will... Despite this, my mind keeps finding new applications for them. :) If designed correctly, they would make such an elegant and forward-compatible model for expressing so many things...! ;)

wrygiel · 2017-09-15T13:39:52Z

okapi/static/ocgpx/1/ocgpx.xsd

+<?xml version="1.0" encoding="utf-8"?>
+<xsd:schema
+    xmlns:xsd="http://www.w3.org/2001/XMLSchema"
+    xmlns="https://opencaching.pl/okapi/static/ocgpx/1"


Perhaps it's safer to use xmlns in OKAPI's github namespace? See here for example. GitHub URLs are highly unlikely to change, and we don't convey a bad impression that "this is for OCPL only".

I would even suggest to put this XSD outside of the OKAPI repository - we could make a separate repository solely for that single XSD (e.g. https://github.com/opencaching/gpx-extension), and introduce semantic versioning similar to the one I used in the example linked above (stable-v1 branch). I have designed tons of XSDs in this fashion in recent years and it seems to be working very well, so far.

I am fine with https://github.com/opencaching/gpx-extension.

(Had in mind that there was a case when the Github path changed for such a document and OKAPI needed an adjustment. But maybe I am confusing this with something else.)

Even if GitHub paths change (they didn't, since the time I started using GitHub), they still redirect properly. In other words, the URL in xmlns will always work properly. This is not necessarily true for the "raw XSD" links used in xsi:schemaLocation, such as this one, but these links can break, so it's not a big deal (in fact, we don't even need to provide xsi:schemaLocation, if we don't want to).

wrygiel · 2017-09-15T13:46:11Z

okapi/static/ocgpx/1/ocgpx.xsd

+        </xsd:documentation>
+    </xsd:annotation>
+
+    <xsd:element name="cache" type="cache_type" />


If we're only trying to add attributes to an existing groundspeak:geocache element, then I guess we shouldn't declare our own cache element. Instead, we should declare only specific elements and annotate them properly (e.g. "this element can be put only as the child of the groundspeak:geocache element").

Similar is true for the log element below.

Agree for the logs. They depend on groundspeak, because logs are added to GPX by the groundspeak extension.

oc:cache (as proposed) is independent from groundspeak:cache; it's directly attached to the wpt element, which represents a geocache (it has <sym>Geocache</sym>). There will probably be more Opencaching-specific cache attributes be added, so it may make sense to group them as oc:cache.

Okay, sounds reasonable. wpt can also represent "additional waypoint", but we may use a different element for extending this one.

Source: opencaching/okapi#511

wrygiel · 2017-09-15T15:54:26Z

See here:

https://github.com/opencaching/gpx-extension

Bananeweizen · 2017-09-16T05:37:31Z

I'm not an expert in XML stuff, but I would leave away the GPX and Groundspeak namespace declarations from your extension, e.g. lines 4 and 5 of https://github.com/opencaching/gpx-extension/blob/master/schema.xsd#L4. You are not really using them in your schema, but you are fixing their version number by doing so. This is a problem because Groundspeak GPX files come in different versions, and therefore some of those would not fit your extension.

At least in theory, in practice basically no one actually verifies using the schemas from the namespace. :)

wrygiel · 2017-09-16T14:29:38Z

I am referring to them, but only in the annotations, not the schema itself. You're right that they can be removed, but there's also nothing wrong with leaving them there, I think, and people who know XML Schema well, tend to look for prefix references there.

This is a problem because Groundspeak GPX files come in different versions

Are you sure? If I remember correctly, they attempted to introduce a new version, but they have backed out of it quite quickly, because it turned out that there were some applications which were xmlns-dependant. (BTW, it is correct for the app to be xmlns-dependant, this is exactly why XML namespaces were created for, after all. But many developers don't understand XML namespaces, hence the problems.)

following5 · 2017-09-16T16:01:51Z

In the GC user profile, there is the choice of GPX version 1.0 and 1.0.1:

GSAK offers the option of GPX "1.0" (which actually produces 1.0.1 files) and 1.1. If you select 1.1, it shows a warning:

OC sites only produce GPX 1.0.1.

following5 · 2017-09-16T16:05:14Z

The version 1.1 GSAK refers to is Topografix 1.1: http://www.topografix.com/GPX/1/1/gpx.xsd

wrygiel · 2017-09-16T18:33:47Z

If I understood you correctly, you're saying that:

GC still allows users to download GPX with groundspeak's 1.0 schema (but I guess that the default option is to use the 1.0.1 one, right?)
GSAK strongly suggests groundspeak's 1.0.1 schema, doesn't allow 1.0 schema, and warns about using 1.1 schema.
Not sure what you means by "refers to", when you're saying that GSAK refers to is Topografix 1.1? Does it allow to generate GPX files in Topografix 1.1 format?

I don't gave a GC account so I might be missing something, but I was quite sure that "Topografix 1.0" + "Groundspeak 1.0.1" was the only true geocaching GPX standard, for many years now. And that the attempt to introduce "Groundspeak 1.1" was a failure.

following5 · 2017-09-16T19:14:33Z

Sorry, I confused Topografix and Groundspeak versions. To sort things:

GC produces Topografix 1.0 GPX with either Groundspeak 1.0 oder Groundspeak 1.0.1 extension. Yes, the default setting for new GC accounts is 1.0.1, I guess for about 6 or 7 years now.
GSAK produces either Topografix 1.0 or 1.1 GPX, both with Groundspeak 1.0.1 extension, and strongly recommends Topografix 1.0.

I was quite sure that "Topografix 1.0" + "Groundspeak 1.0.1" was the only true geocaching GPX standard

Surely it is the prevailing standard today, while some GC users will still download Topografix 1.0 + Groundspeak 1.0 (which ~~doesn't know about geocache attributes~~ doesn't know about negative geocache attributes like "not at night"). I don't see any newer version of Groundspeak extension out there.

Started as the result of the discussion in this thread: opencaching/okapi#511

wrygiel · 2017-09-16T19:56:00Z

This deserves an article :)

https://github.com/opencaching/gpx-extension/blob/master/all-these-namespaces.md

Bananeweizen · 2017-09-17T07:03:43Z

BTW, it is correct for the app to be xmlns-dependant, this is exactly why XML namespaces were created for, after all. But many developers don't understand XML namespaces, hence the problems

I wish everyone would understand that, and not blame cgeo for not importing GPX files, which are - from an XML point of view - not really GPX files. We meanwhile filter the underlying byte stream to get rid of some nasty things that never ever should appear in XML files. :)

wrygiel · 2017-09-17T12:32:15Z

okapi/services/caches/formatters/gpx/gpxfile.tpl.php

@@ -15,7 +15,7 @@
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xmlns:groundspeak="http://www.groundspeak.com/cache/1/0/1"
    xmlns:ox="http://www.opencaching.com/xmlschemas/opencaching/1/0"
-    xmlns:oc="https://opencaching.pl/okapi/static/ocgpx/1"
+    xmlns:oc="https://raw.githubusercontent.com/opencaching/gpx-extension/stable-v1"


opencaching/gpx-extension-v1#6

wrygiel · 2017-09-17T12:32:21Z

okapi/services/caches/formatters/gpx/gpxfile.tpl.php

@@ -27,8 +27,8 @@
        http://www.opencaching.com/xmlschemas/opencaching/1/0
        https://raw.githubusercontent.com/opencaching/okapi/master/etc/nsox.xsd

-        https://opencaching.pl/okapi/static/ocgpx/1
-        https://opencaching.pl/okapi/static/ocgpx/1/ocgpx.xsd
+        https://raw.githubusercontent.com/opencaching/gpx-extension/stable-v1


teiling88 · 2017-09-17T19:35:35Z

Would you support this OC GPX extension on your websites, i.e. in native GPX generation in OCPL / OCDE code? I may also do the implementations for that.

do it :-)

andrixnet · 2017-09-18T10:21:49Z

OCPL also has support for log password.

andrixnet · 2017-09-18T10:31:09Z

Furthermore, if OC specific information is considered to be included as having own namespace in GPX files, then I would recommend the following as well:

OC has different / more cache types. One could declare such cache types using the namespaces according to Groundspeak 1.0.1 which is the most compatible, also in doing so, allows showing in apps that do not know about OC cache types a compatible equivalent, and define a separate OC namespace with the OC cache type. This should be treated by clients as: if present and understood by the app, OC namespace definition to replace the generic definitions.

Like:

<type>Geocache|Unknown Cache</type>
<groundspeak:type>Unknown Cache</groundspeak:type>
<oc:type>Moving Cache</oc:type>

Non-OC aware clients will show "unknown cache".
OC aware clients will show "moving cache".

Same goes for cache sizes, log types, waypoint types and especially attributes.
OC values that can be equivaleted have both namespaces in exported GPX, OC values that have no equivalent (for example attribute OCUK's "rated on handicaching.com") be shown only by OC aware apps.

following5 · 2017-09-18T10:52:04Z

OCPL also has support for log password.

OCPL inserts a localized text into the description text:

"WARNING! You need a PASSWORD to log this cache. You have to either find in the cache (and remember to write it down) or obtain it according to the description!!!"

This is something different than a "requires password" field in the GPX. The latter one will e.g. allow c:geo to prompt for a password then logging the cache. Currently you cannot log "found" GPX-imported caches at all with c:geo if they require a password.

andrixnet · 2017-09-18T10:54:43Z

Yet the presence of this field is one step towards making that happen, right?

SammysHP · 2017-09-18T11:58:29Z

The problem is that some users use tools to manage their caches like GSAK. The additional information will be lost as long as the tools do not work with the extended schema.

following5 · 2017-09-18T12:05:29Z

@SammysHP
What method would you prefer to transfer the information "requires password" from an OC site through GPX to c:geo?

SammysHP · 2017-09-18T16:09:03Z

With the proposed GPX extension and a request to other tools. ;)

wrygiel · 2017-09-19T21:46:07Z

The problem is that some users use tools to manage their caches like GSAK. The additional information will be lost as long as the tools do not work with the extended schema.

In general, external tools should avoid editing/removing XML elements they don't understand. Perhaps that's something that can be reported to GSAK authors as a feature-request?

SammysHP · 2017-09-20T05:50:45Z

All tools that I know store the data in some sort of internal data structure that has a fixed set of fields. Some tools that should be notified: GSAK, Cachewolf, c:geo (already done), QLandkarte GT, QMapShack, Locus. I don't know any other tool that works explicitly with OpenCaching.

wrygiel · 2017-09-20T09:46:31Z

All tools that I know store the data in some sort of internal data structure that has a fixed set of fields.

I think so too. Because this is the easiest approach. It can be done in a better way, but it requires some work (in particular, the internal data structure would need to be able to hold "unknown XML elements"). And most authors probably won't be willing to do this work, but still, perhaps some good ones might :)

…ak:log

added Common Opencaching GPX extension

014011b

following5 commented Sep 14, 2017

View reviewed changes

wrygiel reviewed Sep 15, 2017

View reviewed changes

wrygiel added a commit to opencaching/gpx-extension-v1 that referenced this pull request Sep 15, 2017

Move the XSD from the original draft contents

32d367d

Source: opencaching/okapi#511

wrygiel added a commit to opencaching/gpx-extension-v1 that referenced this pull request Sep 16, 2017

Add all-these-namespaces.md

adaa0d9

Started as the result of the discussion in this thread: opencaching/okapi#511

updated proposed implementation for OC GPX extension

4d0284f

following5 force-pushed the feature/ocgpx branch from 5ad607f to 4d0284f Compare September 17, 2017 10:44

wrygiel reviewed Sep 17, 2017

View reviewed changes

andrixnet mentioned this pull request Sep 18, 2017

Adding an OC size element opencaching/gpx-extension-v1#5

Open

following5 mentioned this pull request Sep 18, 2017

OC log types opencaching/gpx-extension-v1#7

Open

following5 mentioned this pull request Sep 18, 2017

OC attributes in GPX files opencaching/gpx-extension-v1#9

Open

updated OC GPX extension implementation to the current draft

28c70ec

following5 mentioned this pull request Sep 27, 2017

XML validation errors opencaching/gpx-extension-v1#13

Closed

fixed oc:cache elements order; moved OC log data outside of groundspe…

bd67717

…ak:log

following5 force-pushed the feature/ocgpx branch from 6b59805 to bd67717 Compare October 2, 2017 03:19

following5 merged commit 1d66caa into opencaching:master Oct 6, 2017

following5 deleted the feature/ocgpx branch October 8, 2017 16:42

Proposal of a Common Opencaching GPX extension, see #344 #511

Proposal of a Common Opencaching GPX extension, see #344 #511

Uh oh!

Conversation

following5 commented Sep 14, 2017

Uh oh!

following5 commented Sep 14, 2017

Uh oh!

following5 commented Sep 14, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

following5 commented Sep 14, 2017

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

wrygiel Sep 15, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

wrygiel commented Sep 15, 2017

Uh oh!

Bananeweizen commented Sep 16, 2017

Uh oh!

wrygiel commented Sep 16, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

following5 commented Sep 16, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

following5 commented Sep 16, 2017

Uh oh!

wrygiel commented Sep 16, 2017

Uh oh!

following5 commented Sep 16, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

wrygiel commented Sep 16, 2017

Uh oh!

Bananeweizen commented Sep 17, 2017

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

teiling88 commented Sep 17, 2017

Uh oh!

andrixnet commented Sep 18, 2017

Uh oh!

andrixnet commented Sep 18, 2017

Uh oh!

following5 commented Sep 18, 2017

Uh oh!

andrixnet commented Sep 18, 2017

Uh oh!

SammysHP commented Sep 18, 2017

Uh oh!

following5 commented Sep 18, 2017

Uh oh!

SammysHP commented Sep 18, 2017

Uh oh!

following5 commented Sep 14, 2017 •

edited

Loading

wrygiel Sep 15, 2017 •

edited

Loading

wrygiel commented Sep 16, 2017 •

edited

Loading

following5 commented Sep 16, 2017 •

edited

Loading

following5 commented Sep 16, 2017 •

edited

Loading

wrygiel commented Sep 19, 2017 •

edited

Loading