Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Proposal of a Common Opencaching GPX extension, see #344 #511

Merged
merged 4 commits into from Oct 6, 2017

Conversation

Projects
None yet
6 participants
@following5
Copy link
Contributor

commented Sep 14, 2017

No description provided.

@following5

This comment has been minimized.

Copy link
Contributor Author

commented Sep 14, 2017

@opencaching/opencaching-pl-lead-programmers
@teiling88
This is a proposal for an own Opencaching GPX extension, which would allow to put Opencaching-specific information into GPX files. It starts with an OKAPI implementation of log uuids (OKAPI issue #344) and a "requires password" field for caches (see OC.de forum topic). The latter one would allow to show password requirement e.g. in c:geo, when the cache was imported via GPX.

I suggest do deposit the XML schema definition at https://opencaching.pl/okapi/static/ocgpx. So it would be maintained in the OKAPI project, which ensures stability and compatibility across all OC sites and versions, while the reference file is at the PL site which is always most current due to automatic OKAPI deployment.

Would you support this OC GPX extension on your websites, i.e. in native GPX generation in OCPL / OCDE code? I may also do the implementations for that.

Do you have any comments or suggestions? I am not experienced in creating XML Schemas, just collected the information needed from tutorials and from existing GPX extensions.

@following5

This comment has been minimized.

Copy link
Contributor Author

commented Sep 14, 2017

Example output:

<gpx
...
    xmlns:oc="https://opencaching.pl/okapi/static/ocgpx/1"
    xsi:schemaLocation="
...
        https://opencaching.pl/okapi/static/ocgpx/1
        https://opencaching.pl/okapi/static/ocgpx/1/ocgpx.xsd
>
<wpt lat="45.020567" lon="7.037033">
    <groundspeak:cache archived="False" available="True" id="1">
        <groundspeak:name>My first cache</groundspeak:name>
...
        <groundspeak:log id="33">
            <groundspeak:date>2017-09-02T21:26:00+02:00</groundspeak:date>
            <groundspeak:type>Didn't find it</groundspeak:type>
...
            <oc:log>
                <oc:uuid>426A3DF7-CF33-D83F-EB68-63711932E09F</oc:uuid>
            </oc:log>
        </groundspeak:log>
    </groundspeak:cache>
    <oc:cache>
        <oc:req_passwd>true</req_passwd>
    </oc:cache>

@following5

This comment has been minimized.

Copy link
Contributor Author

commented Sep 14, 2017

@rsudev, @SammysHP:
Would you support this in c:geo? E.g. evaluate the OC req_passwd flag from GPX files?


<xsd:complexType name="cache_type">
<xsd:sequence>
<xsd:element name="logpw" type="xsd:boolean" minOccurs="1" maxOccurs="1" />

This comment has been minimized.

Copy link
@following5

following5 Sep 14, 2017

Author Contributor

This must be renamed to "req_passwd".

This comment has been minimized.

Copy link
@wrygiel

wrygiel Sep 15, 2017

Member

BTW - worth considering - perhaps we could replace this be a more generic A-code attribute element, and introduce a new A-code for this purpose?

This comment has been minimized.

Copy link
@wrygiel

wrygiel Sep 15, 2017

Member

Since A-codes are quite solid and backward compatible, developers could implement this logic by checking if a particular A-code is present. Also, the users of application which already display attributes, would get this information independently displayed, without the need of the developer implementing support for the new A-code.

(I was quite sure we already had an A-code for "requires passoword", but I checked, and it seems we don't.)

This comment has been minimized.

Copy link
@following5

following5 Sep 15, 2017

Author Contributor

Some things to consider:

  • There is already a way to deliver OC attributes in GPX, which is understood by some applications; it's called attrs=gc_ocde:attrs in OKAPI.
  • The only major OKAPI App - c:geo - already processes the password requirement as indicated by the req_passwd flag of services/caches/geocache(s). Don't know how they display it.
  • It may be a good idea to show the password requirement in a more prominent way than an attribute. Because if you are in the field and forget to write down the password, you will not be able to log the cache later. OCPL therefore prominently inserst a text into the GPX description text which points out the password reqirement.
  • It would be the first OKAPI-only attribute, because passwords are stored as strings, while OC attributes are stored as boolean flags. So it would need some change in the OKAPI attribute logic, and it could not be set in the same way like other attributes (see the new services/caches/edit method).
  • The user's requests that lead to this proposal was to transfer the "requires password" flag from www.opencaching.de via GPX to c:geo.

All things considered, I would prefer to go for the quick solution and replicate the req_passwd flag from services/caches/geocaches to GPX. Creating a new GPX attributes standard (besides of the already existing GC and OCDE standards) might be persued separately. Will do no harm if the "requires password" information would be available redundantly.

This comment has been minimized.

Copy link
@wrygiel

wrygiel Sep 15, 2017

Member

Will do no harm if the "requires password" information would be available redundantly.

Yes, sure.

It's a pity I never found the time to finish the development of A-codes and S-codes. Probably never will... Despite this, my mind keeps finding new applications for them. :) If designed correctly, they would make such an elegant and forward-compatible model for expressing so many things...! ;)

<?xml version="1.0" encoding="utf-8"?>
<xsd:schema
xmlns:xsd="http://www.w3.org/2001/XMLSchema"
xmlns="https://opencaching.pl/okapi/static/ocgpx/1"

This comment has been minimized.

Copy link
@wrygiel

wrygiel Sep 15, 2017

Member

Perhaps it's safer to use xmlns in OKAPI's github namespace? See here for example. GitHub URLs are highly unlikely to change, and we don't convey a bad impression that "this is for OCPL only".

I would even suggest to put this XSD outside of the OKAPI repository - we could make a separate repository solely for that single XSD (e.g. https://github.com/opencaching/gpx-extension), and introduce semantic versioning similar to the one I used in the example linked above (stable-v1 branch). I have designed tons of XSDs in this fashion in recent years and it seems to be working very well, so far.

This comment has been minimized.

Copy link
@following5

following5 Sep 15, 2017

Author Contributor

I am fine with https://github.com/opencaching/gpx-extension.

(Had in mind that there was a case when the Github path changed for such a document and OKAPI needed an adjustment. But maybe I am confusing this with something else.)

This comment has been minimized.

Copy link
@wrygiel

wrygiel Sep 15, 2017

Member

Even if GitHub paths change (they didn't, since the time I started using GitHub), they still redirect properly. In other words, the URL in xmlns will always work properly. This is not necessarily true for the "raw XSD" links used in xsi:schemaLocation, such as this one, but these links can break, so it's not a big deal (in fact, we don't even need to provide xsi:schemaLocation, if we don't want to).

</xsd:documentation>
</xsd:annotation>

<xsd:element name="cache" type="cache_type" />

This comment has been minimized.

Copy link
@wrygiel

wrygiel Sep 15, 2017

Member

If we're only trying to add attributes to an existing groundspeak:geocache element, then I guess we shouldn't declare our own cache element. Instead, we should declare only specific elements and annotate them properly (e.g. "this element can be put only as the child of the groundspeak:geocache element").

Similar is true for the log element below.

This comment has been minimized.

Copy link
@following5

following5 Sep 15, 2017

Author Contributor

Agree for the logs. They depend on groundspeak, because logs are added to GPX by the groundspeak extension.

oc:cache (as proposed) is independent from groundspeak:cache; it's directly attached to the wpt element, which represents a geocache (it has <sym>Geocache</sym>). There will probably be more Opencaching-specific cache attributes be added, so it may make sense to group them as oc:cache.

This comment has been minimized.

Copy link
@wrygiel

wrygiel Sep 15, 2017

Member

Okay, sounds reasonable. wpt can also represent "additional waypoint", but we may use a different element for extending this one.

wrygiel added a commit to opencaching/gpx-extension-v1 that referenced this pull request Sep 15, 2017

@wrygiel

This comment has been minimized.

Copy link
Member

commented Sep 15, 2017

@Bananeweizen

This comment has been minimized.

Copy link
Contributor

commented Sep 16, 2017

I'm not an expert in XML stuff, but I would leave away the GPX and Groundspeak namespace declarations from your extension, e.g. lines 4 and 5 of https://github.com/opencaching/gpx-extension/blob/master/schema.xsd#L4. You are not really using them in your schema, but you are fixing their version number by doing so. This is a problem because Groundspeak GPX files come in different versions, and therefore some of those would not fit your extension.

At least in theory, in practice basically no one actually verifies using the schemas from the namespace. :)

@wrygiel

This comment has been minimized.

Copy link
Member

commented Sep 16, 2017

I am referring to them, but only in the annotations, not the schema itself. You're right that they can be removed, but there's also nothing wrong with leaving them there, I think, and people who know XML Schema well, tend to look for prefix references there.

This is a problem because Groundspeak GPX files come in different versions

Are you sure? If I remember correctly, they attempted to introduce a new version, but they have backed out of it quite quickly, because it turned out that there were some applications which were xmlns-dependant. (BTW, it is correct for the app to be xmlns-dependant, this is exactly why XML namespaces were created for, after all. But many developers don't understand XML namespaces, hence the problems.)

@following5

This comment has been minimized.

Copy link
Contributor Author

commented Sep 16, 2017

In the GC user profile, there is the choice of GPX version 1.0 and 1.0.1:

gcprofile

GSAK offers the option of GPX "1.0" (which actually produces 1.0.1 files) and 1.1. If you select 1.1, it shows a warning:

gsak

OC sites only produce GPX 1.0.1.

@following5

This comment has been minimized.

Copy link
Contributor Author

commented Sep 16, 2017

The version 1.1 GSAK refers to is Topografix 1.1: http://www.topografix.com/GPX/1/1/gpx.xsd

@wrygiel

This comment has been minimized.

Copy link
Member

commented Sep 16, 2017

If I understood you correctly, you're saying that:

  • GC still allows users to download GPX with groundspeak's 1.0 schema (but I guess that the default option is to use the 1.0.1 one, right?)
  • GSAK strongly suggests groundspeak's 1.0.1 schema, doesn't allow 1.0 schema, and warns about using 1.1 schema.
  • Not sure what you means by "refers to", when you're saying that GSAK refers to is Topografix 1.1? Does it allow to generate GPX files in Topografix 1.1 format?

I don't gave a GC account so I might be missing something, but I was quite sure that "Topografix 1.0" + "Groundspeak 1.0.1" was the only true geocaching GPX standard, for many years now. And that the attempt to introduce "Groundspeak 1.1" was a failure.

@following5

This comment has been minimized.

Copy link
Contributor Author

commented Sep 16, 2017

Sorry, I confused Topografix and Groundspeak versions. To sort things:

  • GC produces Topografix 1.0 GPX with either Groundspeak 1.0 oder Groundspeak 1.0.1 extension. Yes, the default setting for new GC accounts is 1.0.1, I guess for about 6 or 7 years now.

  • GSAK produces either Topografix 1.0 or 1.1 GPX, both with Groundspeak 1.0.1 extension, and strongly recommends Topografix 1.0.

I was quite sure that "Topografix 1.0" + "Groundspeak 1.0.1" was the only true geocaching GPX standard

Surely it is the prevailing standard today, while some GC users will still download Topografix 1.0 + Groundspeak 1.0 (which doesn't know about geocache attributes doesn't know about negative geocache attributes like "not at night"). I don't see any newer version of Groundspeak extension out there.

wrygiel added a commit to opencaching/gpx-extension-v1 that referenced this pull request Sep 16, 2017

Add all-these-namespaces.md
Started as the result of the discussion in this thread:
opencaching/okapi#511
@wrygiel

This comment has been minimized.

Copy link
Member

commented Sep 16, 2017

@Bananeweizen

This comment has been minimized.

Copy link
Contributor

commented Sep 17, 2017

BTW, it is correct for the app to be xmlns-dependant, this is exactly why XML namespaces were created for, after all. But many developers don't understand XML namespaces, hence the problems

I wish everyone would understand that, and not blame cgeo for not importing GPX files, which are - from an XML point of view - not really GPX files. We meanwhile filter the underlying byte stream to get rid of some nasty things that never ever should appear in XML files. :)

@following5 following5 force-pushed the following5:feature/ocgpx branch from 5ad607f to 4d0284f Sep 17, 2017

@@ -15,7 +15,7 @@
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:groundspeak="http://www.groundspeak.com/cache/1/0/1"
xmlns:ox="http://www.opencaching.com/xmlschemas/opencaching/1/0"
xmlns:oc="https://opencaching.pl/okapi/static/ocgpx/1"
xmlns:oc="https://raw.githubusercontent.com/opencaching/gpx-extension/stable-v1"

This comment has been minimized.

Copy link
@wrygiel

wrygiel Sep 17, 2017

Member

Bad xmlns

This comment has been minimized.

@@ -27,8 +27,8 @@
http://www.opencaching.com/xmlschemas/opencaching/1/0
https://raw.githubusercontent.com/opencaching/okapi/master/etc/nsox.xsd
https://opencaching.pl/okapi/static/ocgpx/1
https://opencaching.pl/okapi/static/ocgpx/1/ocgpx.xsd
https://raw.githubusercontent.com/opencaching/gpx-extension/stable-v1

This comment has been minimized.

Copy link
@wrygiel

wrygiel Sep 17, 2017

Member

Here too.

@teiling88

This comment has been minimized.

Copy link
Contributor

commented Sep 17, 2017

Would you support this OC GPX extension on your websites, i.e. in native GPX generation in OCPL / OCDE code? I may also do the implementations for that.

do it :-)

@andrixnet

This comment has been minimized.

Copy link
Contributor

commented Sep 18, 2017

OCPL also has support for log password.

@andrixnet

This comment has been minimized.

Copy link
Contributor

commented Sep 18, 2017

Furthermore, if OC specific information is considered to be included as having own namespace in GPX files, then I would recommend the following as well:

OC has different / more cache types. One could declare such cache types using the namespaces according to Groundspeak 1.0.1 which is the most compatible, also in doing so, allows showing in apps that do not know about OC cache types a compatible equivalent, and define a separate OC namespace with the OC cache type. This should be treated by clients as: if present and understood by the app, OC namespace definition to replace the generic definitions.

Like:

<type>Geocache|Unknown Cache</type>
<groundspeak:type>Unknown Cache</groundspeak:type>
<oc:type>Moving Cache</oc:type>

Non-OC aware clients will show "unknown cache".
OC aware clients will show "moving cache".

Same goes for cache sizes, log types, waypoint types and especially attributes.
OC values that can be equivaleted have both namespaces in exported GPX, OC values that have no equivalent (for example attribute OCUK's "rated on handicaching.com") be shown only by OC aware apps.

@following5

This comment has been minimized.

Copy link
Contributor Author

commented Sep 18, 2017

OCPL also has support for log password.

OCPL inserts a localized text into the description text:

"WARNING! You need a PASSWORD to log this cache. You have to either find in the cache (and remember to write it down) or obtain it according to the description!!!"

This is something different than a "requires password" field in the GPX. The latter one will e.g. allow c:geo to prompt for a password then logging the cache. Currently you cannot log "found" GPX-imported caches at all with c:geo if they require a password.

@andrixnet

This comment has been minimized.

Copy link
Contributor

commented Sep 18, 2017

Yet the presence of this field is one step towards making that happen, right?

@SammysHP

This comment has been minimized.

Copy link

commented Sep 18, 2017

The problem is that some users use tools to manage their caches like GSAK. The additional information will be lost as long as the tools do not work with the extended schema.

@following5 following5 referenced this pull request Sep 18, 2017

Open

OC log types #7

@following5

This comment has been minimized.

Copy link
Contributor Author

commented Sep 18, 2017

@SammysHP
What method would you prefer to transfer the information "requires password" from an OC site through GPX to c:geo?

@SammysHP

This comment has been minimized.

Copy link

commented Sep 18, 2017

With the proposed GPX extension and a request to other tools. ;)

@wrygiel

This comment has been minimized.

Copy link
Member

commented Sep 19, 2017

The problem is that some users use tools to manage their caches like GSAK. The additional information will be lost as long as the tools do not work with the extended schema.

In general, external tools should avoid editing/removing XML elements they don't understand. Perhaps that's something that can be reported to GSAK authors as a feature-request?

@SammysHP

This comment has been minimized.

Copy link

commented Sep 20, 2017

All tools that I know store the data in some sort of internal data structure that has a fixed set of fields. Some tools that should be notified: GSAK, Cachewolf, c:geo (already done), QLandkarte GT, QMapShack, Locus. I don't know any other tool that works explicitly with OpenCaching.

@wrygiel

This comment has been minimized.

Copy link
Member

commented Sep 20, 2017

All tools that I know store the data in some sort of internal data structure that has a fixed set of fields.

I think so too. Because this is the easiest approach. It can be done in a better way, but it requires some work (in particular, the internal data structure would need to be able to hold "unknown XML elements"). And most authors probably won't be willing to do this work, but still, perhaps some good ones might :)

@following5 following5 referenced this pull request Sep 27, 2017

Closed

XML validation errors #13

@following5 following5 force-pushed the following5:feature/ocgpx branch from 6b59805 to bd67717 Oct 2, 2017

@following5 following5 merged commit 1d66caa into opencaching:master Oct 6, 2017

@following5 following5 deleted the following5:feature/ocgpx branch Oct 8, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.