Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Already on GitHub? Sign in to your account

Cache size values and future-compatibility #155

Closed
GoogleCodeExporter opened this Issue Jul 19, 2015 · 29 comments

Comments

Projects
None yet
1 participant
OKAPI documentation currently states that size has values between 1 and 5, i.e. 
micro to x-large. At Opencaching.de there are two additional sizes:

"unknown", which is database value 1 and will be returned as 0 by current OKAPI 
implementation

"nano", which is currently in master branch for testing and may be released 
with next OC.de code update. Database value is 7.

Regarding nano, a solution might be to return 1 and an addtional flag 
nano=true, so it will not break existing applications.

Original issue reported on code.google.com by following09 on 24 Oct 2012 at 5:18

OKAPI returns floats, so other values are also an option ("1.5", etc.).

"unknown" should be null, I guess (all virtual caches have null there).

Original comment by rygielski on 25 Oct 2012 at 5:58

Original comment by rygielski on 25 Oct 2012 at 5:58

  • Added labels: Priority-High
  • Removed labels: Priority-Medium
Returning 0 is a bug, since OKAPI states the value should be a float between 1 
and 5. I think we should return 1 for nano, 1.5 for micro, and the rest should 
stay the same.

Original comment by rygielski on 25 Oct 2012 at 6:11

Sorry, I was in error about number 7.

1 = "other size"
7 = "no container", which is set for virtual caches and webcam caches.
8 = nano

null is the correct return value for "no container". null must NOT be returned 
for "other size", because "other size" means that there IS a container. "other 
size" is in use for miscellaneous purposes:

- for nano caches (some are listed as "micro" and some "other size")
- for caches whose owners consider naming the cache size as a spoiler 
- by some owners who are not sure what is the correct size spec for their 
container
- for some event caches... maybe a reastaurant or whatever is even too large 
for "xtra large" :)
- maybe there are even containers with varying size?

There are about 350 active "other size" caches on OC.de.

Original comment by following09 on 28 Oct 2012 at 3:49

That's a bit weird. There is no way to return such size in OKAPI (we cannot 
break backward compatibility, also all other attributes are in the 1.0..5.0 
range, so it's neat).

Since "other size" does not have a specific meaning, I will return it as "null" 
and update the docs saying something like this:

"""
Note: Some OC sites may use the "null" value to represent caches which do have 
a container, but its size is unknown or hidden (i.e. considered a spoiler). 
There is no way of clearly differentiating such scenarios. Users should be 
directed to the cache description for more info.
"""

In the future, if it turns out to be really important (users complaining), we 
can revise this solution. But this would require us to add some more fields 
("size" MUST stay the same, but we may add other fields with other semantics).

Original comment by rygielski on 28 Oct 2012 at 7:56

Just retuning null for "unknown" would break replication and search functions 
for OC.de. Users should be able to select/filter either caches with or without 
container, which is not possible if null can mean both.

Generally, OC.de is in active development, and any sort of new options may be 
added to the database; we cannot restrict this to the capabilities of existing 
OKAPI clients. What about versioning the interface, e.g. by adding an optional 
"version" parameter to services/caches/geocache ? Clients could request higher 
versions if they understand additional features.

Regarding nano: I think OKAPI should behave the same for all OC.de notes. Am I 
getting it right that you would reserve 1.0 for nano, so all existing micro 
caches would change from 1.0 to 1.5? This could break applications which use 
icons to visualize sizes and have a switch statment like 1.0 = micro, 2.0 = 
small ... It also would need an replication update mechanism which changes 1.0 
to 1.5 for all micros.

Original comment by cgoo...@online.de on 30 Oct 2012 at 2:58

> Just retuning null for "unknown" would break replication and search
> functions for OC.de.

Hmm, true. 

> Generally, OC.de is in active development, and any sort of new
> options may be added to the database; we cannot restrict this to the
> capabilities of existing OKAPI clients.

There are some applications which are not prepared to handle things
like "unknown size". API must be prepared to deal with such
applications gracefully. That's why the semantics of the original
"size" parameter must stay the same.

But from your next comment, I can see you understand that :)
(Thanks Gods!)

> What about versioning the interface, e.g. by adding an optional
> "version" parameter to services/caches/geocache ? Clients could
> request higher versions if they understand additional features.

We don't need to version the entire method in this case. We want to
version only one field, so we may add "size2" field (in the geocache
method) which would include additional "unknown" value and a new
"size2" parameter (in the search methods) which would allow searching
the "unknown size" caches. This should work.

> Regarding nano: I think OKAPI should behave the same for all OC.de
> notes. Am I getting it right that you would reserve 1.0 for nano, so
> all existing micro caches would change from 1.0 to 1.5?

Yes.

> This could break applications which use icons to visualize sizes and
> have a switch statment like 1.0 = micro, 2.0 = small ... It also
> would need an replication update mechanism which changes 1.0 to 1.5
> for all micros.

Well, in theory, OKAPI states (from the beginning) that "size" is a
float value in 1-5 range. So, better developers should be prepared for
that. However, you may be right - many don't even read the docs. Maybe
it's better to include this change in the "size2" field and leave the
"size" field as it was.

BUT, this would also mean that we would want to do the same every time
someone adds a new cache size (like, super-hyper-extra-fucking-huge
one)... And we don't want want to release newer version of this
parameter every year or so. We should design it to be
future-compatible.

Original comment by rygielski on 30 Oct 2012 at 4:41

Fuck, did I say "unknown size"? Its called "other size", not unknown.

> BUT, this would also mean that we would want to do the same every time
> someone adds a new cache size (like, super-hyper-extra-fucking-huge
> one)... And we don't want want to release newer version of this
> parameter every year or so. We should design it to be
> future-compatible.

What about defining size2 as "may be any floating point value. Sizes which are 
unknown to the application should be stored and passed on with their original 
value, but presented to the user as 'other size'"?

0 = other size
0.5 = nano
1 = micro
...
5 = extra large

Original comment by cgoo...@online.de on 30 Oct 2012 at 5:02

> What about defining size2 as "may be any floating point value. Sizes
> which are unknown to the application should be stored and passed on
> with their original value, but presented to the user as 'other
> size'"?

All other attributes (difficulty, terrain, rating) are on the 1..5
scale, so it would be best if size was too.

Maybe we should adapt the scale used by opencaching.com? I know they
are our enemy, in some weird way, but their scale IS becoming a
de-facto standard in the world (or at leasy, much more than our scales
are). The current names-values mapping used by opencaching.com is:

1.0-1.5 - nano
1.6-2.5 - micro
2.6-3.5 - small
3.6-4.5 - regular
4.6-5.0 - large

As to the textual representation - the best way would be to expose
separate fields called "size_text", "rating_text" etc. These fields
would be localized (dependant on the "lang" parameter). This way,
developers won't have to hardcode their values (they will simply ask
for them with geocache?cache_code=ABC&fields=size_text). We will guide
them not to hardcode their own mappings while using the "size2" field.

So, I propose the following values for OUR sizes:

1.3 - nano (at this point, on OCDE only)
2.0 - micro
3.0 - small
3.8 - regular
4.6 - large
4.9 - very large

This way:

- we will be allowed to change these values in the future.
- the "size2" scale will be compatibile with the current scale 
  used by opencaching.com.
- it won't affect OCPL/OCDE sites which are currently NOT using a 
  "1..5 rating" for size - OCPL/OCDE use textual values currently
  and our users are OK with them,
- developers which would want to display the size in a 1..5 scale
  will be allowed to do so (caches marked with "large" and "very 
  large" would both have 5 "stars").
- developers which would want to display a textual representation 
  (compatibile with OCPL/OCDE) will use the "size_text" field.

Original comment by rygielski on 31 Oct 2012 at 6:53

Original comment by rygielski on 31 Oct 2012 at 6:53

  • Changed title: Cache size values and future-compatibility
Also, "size" would become deprecated (in favor of "size2"), but fully 
functional.

Original comment by rygielski on 31 Oct 2012 at 2:00

Also, merging large and very large into one category seems much better idea 
than merging nano and micro:

mysql> select cs.en, count(*) from caches c, cache_size cs where cs.id=c.size 
group by cs.en order by count(*) desc;
+--------------+----------+
| en           | count(*) |
+--------------+----------+
| Small        |     8313 |
| Micro        |     7139 |
| Regular      |     6020 |
| No container |     1218 |
| Large        |      731 |
| Very large   |      148 |
+--------------+----------+
6 rows in set (0.36 sec)

Original comment by rygielski on 31 Oct 2012 at 2:08

There is one alternative: Maybe you'd want to completely *replace* the "very 
large" size with "nano" one? This way, we could still have integer values (1 
for nano, 5 for large).

On the other hand, it would also mean that all "very large" caches would become 
"large" ones, and definition of large cache would have to be altered.

Damn, there is no clear way to deal with this...

Original comment by rygielski on 2 Nov 2012 at 4:34

Could you paste your current definitions of sizes? We should compare OCDE and 
OCPL definitions before we proceed. I guess we should have the same definitions 
if we want to have a common API.

OCPL:

micro < 0.1
0.1 <= small < 1.0
1.0 <= regular < 3.0
3.0 <= large < 10.0
10.0 <= very large.

Original comment by rygielski on 2 Nov 2012 at 4:54

I cannot decide about creation or abandoning cache sizes - thats a team 
decision. But distinction between large and very large may be crucical for 
existing caches, as the sizes gives a hint what to look for (is there a big box 
hidden in the cave, or is the cave itself the container?). So I don't think 
that we can abandon it, as it can break existing listings. Also it would 
handicap OC<->GC data exchange. 

Regarding numeric representation: The real-world size step between large and 
very large is the biggest of all, even in a logarithmic scale, as it is open 
ended. Therefore I think compressing the long end to very similar values for 
both would be unintuitive for the user, either if numeric values are directly 
displayed, or if they are visualized as a graph bar.

Why not just go beyond the OX scale? The only drawback I can imagine is that if 
you transfer data from an OC database to OX and then back to an OC database, 
information would get lost. But who should want to do that? By introducing nano 
we give up 100% interchangeability with GC, so why bothering with OX? I am glad 
about any OC feature that other platforms do *not* have, so it makes OC 
distinguishable instead of exchangeble with them.

But that's just my personal opinion. Maybe we should consult other developers 
at http://forum.geocaching-network.com/index.php?board=26.0 ?

Regarding future extensions, I can well imagine few-millimeter-sized electronic 
"pico" caches at the low end - but hardly a need to differentiate the rarely 
used "very large" even further to something like "hyper large". So it seems 
reasonable to reserve some space at the low end.


Regarding your suggestions in #9: It think that combining a localized textual 
field with variable numeric fields would break any replication or matching of 
different cache databases as soon as values are redefined, so there should be 
some fixed values for well-known sizes.

Regarding OC.de size scale: Until a few weeks ago, there has be no 
definition/explanation of sizes. Now we have one in the wiki, which is same as 
yours @ OCPL, with exception of small/regular limit being changed to 0.75 
litres. This looks reasonable as it is very common here that containers 
somewhat below 1 liter are flagged as regular. But this is not written in 
stone, but just the result of a discussion of two wiki users.

Original comment by cgoo...@online.de on 3 Nov 2012 at 12:48

(On the other hand, I also understand your point of expanding highly used parts 
of the scale and compressing rarely used parts. It's a difficult decision, and 
I could live with any solution which allows complete replication of known OC.de 
sizes.)

Original comment by cgoo...@online.de on 3 Nov 2012 at 12:53

Okay then, no merging/compressing. I think it's safer too.

> Regarding your suggestions in #9: It think that combining a
> localized textual field with variable numeric fields would break any
> replication or matching of different cache databases as soon as
> values are redefined, so there should be some fixed values for
> well-known sizes.

We have these fixed values currently, but it's clearly not what we're
looking for, since we cannot add the "nano" to a *fixed* list.

If we hold on to that solution, we will have to release a new API
version every time a new cache size is added (size2, size3, size4,
etc...). It's an easy solution, but it seems somewhat lame...

Also, your "other size" breaks the implicit order of this scheme. If
user wants all "regular or bigger" caches, should we include the
"other size" in the result? I guess that the next version of API
should not allow to ask such queries anymore...

The "other size" example shows us clearly, that "size" is becoming an
option-based field instead of a numeric one. Most developers (me
included) haven't anticipated such scenario.

> It's a difficult decision, and I could live with any solution which
> allows complete replication of known OC.de sizes.

If not for the "other size", I would go with the 1..5 float approach
in comment#9. But "other size" still cannot be represented this way. 

It would seem that we have to change the semantics of "size", so it
resambles "type". If I knew you had this "other size" option, I guess
I would male all size-related fields and params would look completely
different.

To sum up:

I am thinking about adding the "size2" field with possible values of
"nano", "micro", "small", "regular", "large", "xlarge", "other". And a
comment "other sizes may appear in the future, you should be
preparated for that and treat them as 'other'".

What do you think?

Original comment by rygielski on 3 Nov 2012 at 10:54

The comment mentioned above ("other sizes may appear in the future,
you should be preparated for that and treat them as 'other'") may
affect backward-compatibility later, if user tried to search caches
with size2=other (we wouldn't know if to include "pico" caches in such
search). It's better to skip this comment and add "size3" if we ever
wanted "pico" sizes in the future (I hope we won't) ;)

So, anyhow, this stands:

> I am thinking about adding the "size2" field with possible values of
> "nano", "micro", "small", "regular", "large", "xlarge", "other".

Also for searching, e.g. "...search/all?size2=regular|large|xlarge|other".

Original comment by rygielski on 3 Nov 2012 at 11:08

> I am thinking about adding the "size2" field with possible values of
> "nano", "micro", "small", "regular", "large", "xlarge", "other".

This is the GC.com solution; I think it's ok. (I just notice that I was in 
error about GC and "very large" - they don't have it. But I still think we 
should keep it.) 

> if user tried to search caches with size2=other (we wouldn't know if to 
include 
> "pico" caches in such search 

This is an inherent problem of both the "other" option and of adding new sizes, 
and it already applies to nanos. Currently, nanos are either listed as micro or 
as "other" at OC.de (I think most are micro); now we add the third variant 
"nano". All three will coexist in the future, as many existing listings won't 
be updated. So we must live with the fact that there is and will be no clear 
way to search for nanos. For the next 5-10 years, the nano size will mainly be 
an additional hint in the listing what container size to look for.

(Btw., opencaching.nl also has the "other" size, so this problem already exists 
in OKAPI.)

So all this is already fucked up enough that adding "pico" sometime in the 
future would not do any relevant harm to search functions. Still it might break 
some apps where the "default" case was forgotton in a size switch statement. 

Original comment by cgoo...@online.de on 3 Nov 2012 at 3:30

One more question: How do you handle the "other size" in exported GPX files? 
AFAIK, there is no documented counterpart for this.

Original comment by rygielski on 3 Nov 2012 at 3:32

> One more question: How do you handle the "other size" in exported GPX files? 
AFAIK, > there is no documented counterpart for this.

GC.com and GPX know "Other" size, therefore it is returned as "Other".

Original comment by cgoo...@online.de on 3 Nov 2012 at 3:37

> there is and will be no clear way to search for nanos. For the next
> 5-10 years, the nano size will mainly be an additional hint in the
> listing what container size to look for.

You could send an email to all authors of micro caches asking them to
make the switch, if necessary.

> (Btw., opencaching.nl also has the "other" size, so this problem
> already exists in OKAPI.)

Oops.

> So all this is already fucked up enough that adding "pico" sometime
> in the future would not do any relevant harm to search functions.
> Still it might break some apps where the "default" case was
> forgotton in a size switch statement.

I think you misunderstood my intention. My thought was: if we ever
wanted to add "pico", this would require a new ("size3") version of
this API. Both "size" and "size2" would stay compatibly with their 
previous limitations.

So, in case of "size2" this list is fixed and cannot change in the
future: "none", "nano", "micro", "small", "regular", "large",
"xlarge", "other". Next version of this list will be published as
"size3", so all app developers may catch up if they wanted.

Hopefully, we will stay at "size2" for a long time.

For backward compatibility, "search?size=1-1" will return both nano
and micro caches. "search?size=1-5" won't return "other size" caches,
but "search?size=1-5|X" will. "size" parameter will be deprecated,
since it can't distinguish between "other size" and "no container".

Original comment by rygielski on 3 Nov 2012 at 3:45

> You could send an email to all authors of micro caches asking them to
> make the switch, if necessary.

Good idea. However, about 30 percent of OC.de caches have inactive owners, and 
a good deal of active-owners' caches are automatically synchronized from GC.com 
listings (wich don't know Nano) using the Ocprop tool. So I would expect less 
than halve of listings to be fixed after a mailing. The problem will stay.

> I think you misunderstood my intention. My thought was: if we ever
> wanted to add "pico", this would require a new ("size3") version of
> this API. Both "size" and "size2" would stay compatibly with their 
> previous limitations.

Yes. And your argument to do so (instead of allowing e.g. "pico" to be added to 
size2 later) was that otherwise search function would be broken. I just tried 
to point out that search function is already broken, so this needs not to be a 
concern (while there still may be other reasons to restrict "size2" to today's 
known sizes).

The benefit of keeping "size2" open for new options would be that a new size 
would be immediately be displayable and replicable by all properly designed 
applications, so it would be easier to introduce such a new feature. I expect 
that some years from now, paperless caching using OKAPI will be the predominant 
way of doing OC geocaching, so it would be nice if we could have some 
flexibility in OKAPI which allows us to introduce new features without 
depending on App developers to update their applications. If I get it right, 
this flexibility already exists with cache types and attributes ...?

But as there are other good reasons to chose the size2/size3 solution, and I am 
glad that you support OC.de at all, I won't try to stop you. :) 

Original comment by following09 on 3 Nov 2012 at 4:29

> I just tried to point out that search function is already broken, so
> this needs not to be a concern (while there still may be other
> reasons to restrict "size2" to today's known sizes).

I will be adding a new "size2" parameter to the search functions which 
will allow to find "other" and "nano" caches.

> The benefit of keeping "size2" open for new options would be that a
> new size would be immediately be displayable and replicable by all
> properly designed applications, so it would be easier to introduce
> such a new feature.

True, but we didn't find any way of properly allowing such option. All
our ideas except this one had some logical flaws. (Or, did I miss
something?)

> I expect that some years from now, paperless caching using OKAPI
> will be the predominant way of doing OC geocaching, so it would be
> nice if we could have some flexibility in OKAPI which allows us to
> introduce new features without depending on App developers to update
> their applications. If I get it right, this flexibility already
> exists with cache types and attributes ...?

Yes, but it has flaws too. Currently, cache type can by *any* string.
Whenever you add a new cache type, all applications:

1. will treat it as "other" when reading cache data.
2. will not be able to filter by it using the "type" attribute.

We might need to release a common listing of all OC cache types. OKAPI
could expose such listing in a separate method. All OKAPI instances
would return the same listing. The base copy of the listing will be 
held in OKAPI repository, the same way as the installations list is 
kept now:

http://code.google.com/p/opencaching-api/source/browse/trunk/etc/installations.x
ml

If we'd like to continue this sujbect, maybe it's better if we start a
new issue ;)

> But as there are other good reasons to chose the size2/size3
> solution, and I am glad that you support OC.de at all, I won't try
> to stop you. :)

Ok. I will post the details here, once implemented.

Original comment by rygielski on 3 Nov 2012 at 4:47

Commited a preview: http://code.google.com/p/opencaching-api/source/detail?r=490
(documentation is missing)

Original comment by rygielski on 5 Nov 2012 at 11:50

And here are the missing docs: 
http://code.google.com/p/opencaching-api/source/detail?r=491

Original comment by rygielski on 5 Nov 2012 at 12:31

This issue was closed by revision r493.

Original comment by rygielski on 10 Nov 2012 at 6:59

  • Changed state: Fixed
To sum up:

"size" will continue working, but developers are encouraged to upgrade to 
"size2" which allows to differentiate "nano" vs. "micro" and "none" vs. "other".

Let us hope there won't be any more "size extensions", at least for a while. We 
would have to release "size3" in such case.

Original comment by rygielski on 10 Nov 2012 at 1:08

Thanks! I will go on testing this as soon as Oliver has merged it into master.

I have chosen a similar size/size2 solution for the nano type in OC.de XML 
interface now (opencaching.de/doc/xml) ...

Original comment by cgoo...@online.de on 10 Nov 2012 at 4:57

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment