Allow selection of multiple values per key from a list of options #103

PaulUithol · 2015-12-09T18:25:08Z

Proposal: to allow selection of multiple values for a single key (multiselect), yielding a semicolon delimited list of values. I'm not sure about how the semantics in the form, but that's something to discuss here ;)

Example: from a list of possible values for the network key on an ATM (as defined in the ODK/OMK form), select MasterCard, Visa, MTN, and Airtel (this is one also offering mobile money services), resulting in network=MasterCard;Visa;Airtel;MTN.

The main use case currently for us is that in Uganda, we'd need to map the payment networks accessible through a single mobile money agent (as indicated above, also applies to ATMs).

Using semicolons in OSM keys is not always the right solution (it often isn't); but in this case, it most probably would be: both network values apply to the same POI (it shouldn't be split into multiple entities, as if it were four ATMs), and all values should be present (we can't just go with one or the other). Using all possible values as keys (say "MasterCard=yes", etc) wouldn't be the way to go either. Also see http://wiki.openstreetmap.org/wiki/Semi-colon_value_separator.

Here's a few more examples using semicolons to indicate multiple networks or another key:

Tanzania mobile money (we'll be adjusting our tagging for Uganda, since we need to distinguish more types of financial services there; also, use network instead of operator):

  <node id="3766196144" lat="-6.8225067" lon="39.2773854">
    <tag k="amenity" v="money_transfer"/>
    <tag k="operator" v="Airtel-Money;M-Pesa;Tigo-Pesa"/>
  </node>

Belgian bus lines:

  <node id="2619430757" lat="50.8492063" lon="4.3258571">
    <tag k="bus" v="yes"/>
    <tag k="highway" v="bus_stop"/>
    <tag k="name" v="Quatre-Vents - Vier Winden"/>
    <tag k="name:fr" v="Quatre-Vents"/>
    <tag k="name:nl" v="Vier Winden"/>
    <tag k="network" v="DLVB;IBXL"/>
    <tag k="operator" v="De Lijn;STIB/MIVB"/>
    <tag k="public_transport" v="platform"/>
    <tag k="railway" v="tram_stop"/>
    <tag k="ref:De_Lijn" v="301066"/>
    <tag k="route_ref" v="82"/>
    <tag k="route_ref:De_Lijn" v="126;127;128"/>
    <tag k="tram" v="yes"/>
    <tag k="zone:De_Lijn" v="20"/>
  </node>

London under/overground:

  <node id="11249226" lat="51.3971534" lon="-0.0746248">
    <tag k="name" v="Norwood Junction"/>
    <tag k="naptan:AtcoCode" v="9100NORWDJ"/>
    <tag k="network" v="National Rail;London Overground"/>
    <tag k="railway" v="station"/>
    <tag k="ref" v="NWD"/>
    <tag k="source_ref" v="http://en.wikipedia.org/wiki/London_Overground#Stations"/>
    <tag k="wikipedia" v="en:Norwood Junction railway station"/>
  </node>

The text was updated successfully, but these errors were encountered:

PaulUithol · 2015-12-28T22:20:20Z

It's also a feature in JOSM preset files btw, with semi-colon as the default separator; see http://wiki.openstreetmap.org/wiki/Customising_JOSM_Presets#Multiselect

hallahan · 2016-01-05T00:45:02Z

Do you have any ideas on how we want to specify in the XLS Form that a given OSM question can select multiple values?

cc/ @dalekunce @PaulUithol

hallahan · 2016-01-05T02:03:19Z

This sort of constraint needs to be specified in the XLS document. I'd rather not have to modify pyxform and JavaRosa further. They are bloated and cumbersome libraries that are hard to hack. Also, we will be having constraints beyond just this one, and I think there will be a fair amount of new logic we need to add to the XLS.

Two options:

XLS to JSON in NodeJS

We can use js-xlsx to convert the XLS document to JSON. This JSON can then be sent to the phone and read by OpenMapKit Android.

Directly read XLS in OpenMapKit Android

This approach seems a bit more attractive to me, because we don't have another redundant file to store. We can use Apache POI to directly read the XLS file in OpenMapKit Android. Both libraries look good, however, the Java one looks mature and has stood the test of time. Also, it is very feature rich, not that that is important...

It looks like maven has POI, so I should be able to just declare I want it in the gradle file.

Challenge

The thing we have to figure out is how we get the XLS file to the phone. Currently, ODK Collect downloads the XForm XML, but it does not get the XLS precursor. We don't want to change ODK Collect to be able to do this. That sort of approach would likely not be something UW maintainers would want to pull back in.

That leads me to think we should provide the XLS file as part of a given deployment. Because we are downloading other things we need from the server, we might as well also get the XLS file too. This will then able us to begin building in constraint functionalities like this one.

By having this feature be part of a deployment, this will mean that this feature will only work if we are using OpenMapKit Server. With other ODK Aggregates, we will have to fall back to not having new constraint features. I'm ok with this.

This means we need to have another deployment sprint that implements getting the XLS file into a deployments directory on the phone.

Thoughts?

hallahan · 2016-01-05T02:05:12Z

If HOT is time constrained, one thing we can do is provide them their own APK that just enables multiple selections for all multiple choice questions.

PaulUithol · 2016-01-05T05:12:58Z

For the moment & from my point of view, I don't actually care about the conversion process. I'm fine with just adding an attribute or something like that to the xml forms.

So just having the facility in parsing the xml forms, and OMK itself would be absolutely fine with me. Then we can figure out how to represent this in the xls forms and convert this to xml later.

For example:

      <tag key="network" type="multiple-select">
        <label>Network</label>
        <item>
          <label>Airtel Money</label>
          <value>Airtel Money</value>
        </item>
    etc, etc

hallahan · 2016-01-05T05:37:57Z

I like your idea, however, this solution would still require mods to JavaRosa, because ODK Collect actually uses JavaRosa to parse and create objects from the XForms.

Would enabling multiple select for everything be OK short term? I think parsing out XLS in OMK will be needed for the project as a whole.

I could also parse XForms XML in OpenMapKit, but parsing XLS with Apache POI is super easy actually.

PaulUithol · 2016-01-05T07:00:33Z

Right. Discussed it here, and that would work for us indeed. We have some attributes/keys where this isn't useful (and can lead to people accidentally selecting multiple), but we can catch that quite easily in data review.

On the rest of your approach text: provided you can still simply copy the (XLS) forms to the phone instead of being required to use OMK server, that sounds fine to me. It would mean you'd need to copy the xml to odk, and the corresponding xls to OMK though? Or can we avoid that?

mberg · 2016-01-05T07:10:52Z

Right now XLSForms support select_one for OSM tags. Why not just add
support for select_multiple? Or is this something different that I'm not
understanding properly.

I think keeping OMK ODK compliant as much as possible is really important
for the broader adoption of the tool.

On Tue, Jan 5, 2016 at 10:00 AM, Paul Uithol notifications@github.com
wrote:

Right. Discussed it here, and that would work for us indeed. We have some
questions where this isn't useful (and can lead to people accidentally
selecting multiple), but we can catch that quite easily in data review.

On the rest of your approach text: provided you can still simply copy the
(XLS) forms to the phone instead of being required to use OMK server, that
sounds fine to me. It would mean you'd need to copy the xml to odk, and the
corresponding xls to OMK though? Or can we avoid that?

—
Reply to this email directly or view it on GitHub
#103 (comment)
.

PaulUithol · 2016-01-05T07:50:27Z

As far as I'm aware, that applies to 'regular' odk survey questions - those have a constraints column that gets parsed. However, within a single osm/OMK question, you'll be collecting data for a number of attributes/keys - and there's no way to define constraints per attribute/key at the moment. See the attached examples, two of the forms we're using (out of 12). So this is about seeing how we can define constraints per attribute/key; making one value the default one, select one/multiple, numeric/text input, etc.

hot_03_atm.xlsx
hot_01_mobile_money.xlsx

mberg · 2016-01-05T08:46:10Z

Ah you're right sorry.

Nick - would this need a change in javarosa or just pyxform? If it's
pyxform we can help make the change if we can figure out the right syntax.

On Tue, Jan 5, 2016 at 10:50 AM, Paul Uithol notifications@github.com
wrote:

As far as I'm aware, that applies to 'regular' odk survey questions -
those have a constraints column that gets parsed. However, within a single
osm/OMK question, you'll be collecting data for a number of attributes/keys

and there's no way to define constraints per attribute/key at the moment.
See the attached examples, two of the forms we're using (out of 12). So
this is about seeing how we can define constraints per attribute/key;
making one value the default one, select one/multiple, numeric/text input,
etc.

hot_03_atm.xlsx
https://github.com/AmericanRedCross/OpenMapKitAndroid/files/78123/hot_03_atm.xlsx
hot_01_mobile_money.xlsx
https://github.com/AmericanRedCross/OpenMapKitAndroid/files/78124/hot_01_mobile_money.xlsx

—
Reply to this email directly or view it on GitHub
#103 (comment)
.

hallahan · 2016-01-05T17:53:39Z

@mberg - Basically, adding support for select_multiple would require work in:

pyxform
JavaRosa
ODK Collect.
OpenMapKit Android

That's a lot of repos and different pieces to tackle for a feature like this. This can be done for sure, but this is not a good architecture for maintenance.

@PaulUithol - I think you fully understand what's up regarding:

However, within a single osm/OMK question, you'll be collecting data for a number of attributes/keys - and there's no way to define constraints per attribute/key at the moment.

You are also correct that parsing the XLS for constraints does indeed require the XLS and XML to be on the phone. OMK Android would then directly parse the XLS.

Basically, there is a lot of motivation to cut out all of these middle men.

hallahan · 2016-01-05T18:17:15Z

I think I'm going to work on this problem from the other end right now. I'm going to create a SelectMultipleTagValueFragment based off of SelectOneTagValueFragment.

I can implement this UI and write semi-colon delimited values for OSM tags. I'll make a custom APK where this takes place of the radio buttons for HOT temporarily. This is the easy part, and with it in place, we'll be in a better spot to make the right decisions regarding how to get proper constraint parameters to OpenMapKit Android.

Basically, we need to build the checkbox select multiple interface, and the approach on how we ship constraints to it is not going to affect this aspect of the implementation.

mberg · 2016-01-05T18:26:35Z

Nick - could we define a standard/endpoint for serving up the form? that's
something that ODK compliant servers could also probably support relatively
easily.

I still think option 1 could make sense but i see how it adds a lot of
complexity for something that is OMK specific

On Tue, Jan 5, 2016 at 9:17 PM, Nicholas Hallahan notifications@github.com
wrote:

I think I'm going to work on this problem from the other end right now.
I'm going to create a SelectMultipleTagValueFragment based off of
SelectOneTagValueFragment
https://github.com/AmericanRedCross/OpenMapKitAndroid/blob/master/app/src/main/java/org/redcross/openmapkit/tagswipe/SelectOneTagValueFragment.java
.

I can implement this UI and write semi-colon delimited values for OSM
tags. I'll make a custom APK where this takes place of the radio buttons
for HOT temporarily. This is the easy part, and with it in place, we'll be
in a better spot to make the right decisions regarding how to get proper
constraint parameters to OpenMapKit Android.

Basically, we need to build the checkbox select multiple interface, and
the approach on how we ship constraints to it is not going to affect this
aspect of the implementation.

—
Reply to this email directly or view it on GitHub
#103 (comment)
.

hallahan · 2016-01-05T18:33:43Z

@mberg - That's up to you. Please keep in mind that it is too prohibitive for me to work with proprietary software in context to this project.

mberg · 2016-01-05T19:01:29Z

None of what i'm talking about is proprietary. This is about trying to
build things upon open standards that will ensure a lot of different
platforms can leverage what OMK brings to the table.

If you are going to go direct to XLSForms then you are modifying the
XLSForm standard. If you don't support the corresponding xform that comes
out then you are restricting OMK use to tools that use XLSForms. If you
are going to passing excel directly to OMK you are creating a situation
where you might have a different version of excel on ODK vs. what's in OMK
depending on how the server implements it. If you are proposing
supporting downloading XLSForms directly to OMK then it would probably be
good to try and define it as a standard that people can support. Keeping
this within javarosa and the different ways of transferring forms, while
tedious, offers a lot of advantages.

On Tue, Jan 5, 2016 at 9:33 PM, Nicholas Hallahan notifications@github.com
wrote:

@mberg https://github.com/mberg - That's up to you. Please keep in mind
that it is too prohibitive for me to work with proprietary software in
context to this project.

—
Reply to this email directly or view it on GitHub
#103 (comment)
.

PaulUithol · 2016-01-05T19:15:08Z

Out of curiosity on my part - to what extent is the osm sheet/question part of the XLSForms standard to begin with? Xlsform.org (which as far as I've been able to ascertain should be the 'specification', along with the spreadsheet) doesn't even mention it.

mberg · 2016-01-05T19:31:40Z

xlsform.org is a community managed site and it's just that the OSM tag
documentation haven't been added yet. If anyone wants to they can do so
here,

https://github.com/XLSForm/xlsform.github.io

The current OSM features are both supported by pyxforms and javarosa. What
that means is any platform that supports both can serve up OMK compliant
forms.

On Tue, Jan 5, 2016 at 10:15 PM, Paul Uithol notifications@github.com
wrote:

Out of curiosity on my part - to what extent is the osm sheet/question
part of the XLSForms standard to begin with? Xlsform.org (which as far as
I've been able to ascertain should be the 'specification', along with the
spreadsheet) doesn't even mention it.

—
Reply to this email directly or view it on GitHub
#103 (comment)
.

mberg · 2016-01-05T19:38:58Z

The other important thing to consider is there other odk complaint clients
like enketo where creating an omk like web widget would be super valuable.

Adding that would be pretty much impossible to support or you abandon the
underlying xform.

On Tuesday, January 5, 2016, Matt Berg mberg@ona.io wrote:

xlsform.org is a community managed site and it's just that the OSM tag
documentation haven't been added yet. If anyone wants to they can do so
here,

https://github.com/XLSForm/xlsform.github.io

The current OSM features are both supported by pyxforms and javarosa.
What that means is any platform that supports both can serve up OMK
compliant forms.

On Tue, Jan 5, 2016 at 10:15 PM, Paul Uithol <notifications@github.com
javascript:_e(%7B%7D,'cvml','notifications@github.com');> wrote:

Out of curiosity on my part - to what extent is the osm sheet/question
part of the XLSForms standard to begin with? Xlsform.org (which as far as
I've been able to ascertain should be the 'specification', along with the
spreadsheet) doesn't even mention it.

—
Reply to this email directly or view it on GitHub
#103 (comment)
.

…d to sort that out #103

hallahan · 2016-05-11T19:18:17Z

I'm re-opening this issue, because it needs to be integrated with the constraint.json mechanism. Right now it's hard-coded and requires a custom build. Obviously, we want to do this right.

hallahan · 2016-05-11T19:36:28Z

Another would be opening hours

opening_hours=Mo 10:00-12:00,12:30-15:00; Tu-Fr 08:00-12:00,12:30-15:00; Sa 08:00-12:00

…raint #103

hallahan · 2016-05-30T17:56:18Z

Ok, this feature is now working properly with constraints JSON.

hallahan added the contraints label Dec 10, 2015

hallahan added this to the HOT Task Order milestone Dec 30, 2015

hallahan mentioned this issue Jan 5, 2016

Constraints #117

Closed

hallahan added a commit that referenced this issue Jan 6, 2016

basic fragment #103

92a5055

hallahan added a commit that referenced this issue Jan 6, 2016

tag key and tag label text views are populated #103

397215f

hallahan added a commit that referenced this issue Jan 6, 2016

checkboxes are populated with values #103

0aac969

hallahan added a commit that referenced this issue Jan 6, 2016

almost there, i'm not getting unique IDs for the checkboxes, so i nee…

227471c

…d to sort that out #103

hallahan added a commit that referenced this issue Jan 6, 2016

semi colon delimited values working #103

c09068e

hallahan added a commit that referenced this issue Jan 6, 2016

writing semicolon values in xml almost working correctly #103

4a2237c

hallahan closed this as completed in 8dfc51a Jan 6, 2016

hallahan reopened this May 11, 2016

hallahan added a commit that referenced this issue May 28, 2016

#117 Some default JSON for select_multiple #103

3b0dc8c

hallahan added a commit that referenced this issue May 30, 2016

#151 Select Multiple view shows up based off of select_multiple const…

b65342a

…raint #103

hallahan closed this as completed May 30, 2016

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Allow selection of multiple values per key from a list of options #103

Allow selection of multiple values per key from a list of options #103

PaulUithol commented Dec 9, 2015

PaulUithol commented Dec 28, 2015

hallahan commented Jan 5, 2016

hallahan commented Jan 5, 2016

hallahan commented Jan 5, 2016

PaulUithol commented Jan 5, 2016

hallahan commented Jan 5, 2016

PaulUithol commented Jan 5, 2016

mberg commented Jan 5, 2016

PaulUithol commented Jan 5, 2016

mberg commented Jan 5, 2016

hallahan commented Jan 5, 2016

hallahan commented Jan 5, 2016

mberg commented Jan 5, 2016

hallahan commented Jan 5, 2016

mberg commented Jan 5, 2016

PaulUithol commented Jan 5, 2016

mberg commented Jan 5, 2016

mberg commented Jan 5, 2016

hallahan commented May 11, 2016

hallahan commented May 11, 2016

hallahan commented May 30, 2016

Allow selection of multiple values per key from a list of options #103

Allow selection of multiple values per key from a list of options #103

Comments

PaulUithol commented Dec 9, 2015

PaulUithol commented Dec 28, 2015

hallahan commented Jan 5, 2016

hallahan commented Jan 5, 2016

XLS to JSON in NodeJS

Directly read XLS in OpenMapKit Android

Challenge

hallahan commented Jan 5, 2016

PaulUithol commented Jan 5, 2016

hallahan commented Jan 5, 2016

PaulUithol commented Jan 5, 2016

mberg commented Jan 5, 2016

PaulUithol commented Jan 5, 2016

mberg commented Jan 5, 2016

hallahan commented Jan 5, 2016

hallahan commented Jan 5, 2016

mberg commented Jan 5, 2016

hallahan commented Jan 5, 2016

mberg commented Jan 5, 2016

PaulUithol commented Jan 5, 2016

mberg commented Jan 5, 2016

mberg commented Jan 5, 2016

hallahan commented May 11, 2016

hallahan commented May 11, 2016

hallahan commented May 30, 2016