Layers - Pitch #2461
The idea is to make available a tool that makes contributing and maintaining data easy and efficient to do on a survey, focussing on one aspect of the map at a time.
On the smartphone, there are three issues that further challenge usability:
StreetComplete is so easy and quick to use, because amongst other things, it severely limits what data the user can see and change.
For mappers who want to map/update "everything" on a survey, there is and there will probably never be a more efficient method than taking a lot of photos (or for mapillary) and then adding all the info visible on the photos to the map via JOSM.
The target audience for this idea are mappers who want to focus on completing and keeping up-to-date one aspect of OpenStreetMap efficiently at a time, i.e. per survey.
This requires the app to be able to
Rough UI Concept
The UI / and input form can and should be customized for each layer as long as it enhances the overview and makes the input easier.
Switching to this other part of the app would be possible through the main menu. It would lead to a list of available layers, one can select a layer, that list slides to the left to reveal the map with the appropriate layer.
For all the layers, there should be a consistent color scheme at least for "no" and "not specified yet". I am thinking, it should be...
I did some prototyping with Tangram and found out that:
Too bad for point 1, so the display will be less fancy. However, this means it could maybe be streamlined so that it is easier to add new layers.
Why layers at all, if focussed contribution and maintenance is already possible with the quest system? One can simply deactivate all quests except ones where one wants to do the survey (f.e. only cycleway quests). Well:
So, there are at least two additional prerequisites for layers:
Certainly there are many more, just covering the most valued ones.
These are the most important because these are things that can only be effectively recorded on-site:
This is less important because while they can often only be effectively recorded on-site, it is not necessary to see the big picture:
These are also less important because while it might be very helpful to see the big picture, they do not necessarily need to be recorded on-site:
Now comes the hard part. The really really hard part. Thinking how the UI is best going to be is nice and dandy, but the technical part must work basically so different from the rest of the app that it could basically be another app.
Current data flow
So what this means is, that the answers the user gives does not result in a change to the data, only after upload. This is okay, since that changed data is not shown anywhere. The downside that can already now be observed with this approach is that quests are not immediately (re)created after splitting a way (offline) and quests that conditionally follow after another quest has been answered are also not created until upload.
Required data flow
Above section in a nutshell: Downloaded OSM data is not changed until upload. This approach will not work for the layers as the user wants to see his change immediately, not after upload. Also, he'd want to immediately continue after splitting a way.
So, what is necessary is
An MVP would be to have at least one layer available at all, reachable through the UI. Selecting and viewing the current data should work.
It would be nice to have the layer definition in some kind of streamlined fashion so that it is easier to add new layers, but it is not necessary for a first version.
So, a good MVP should have basic editing capabilities, in a pinch online-only would do, but editing is simply necessary to see if this idea gets any followers. Both MapContrib and MapComplete, who have similar approaches, did not really lift off. So before sinking too much time into this idea, it would be best to get a rudimentary prototype going ASAP and then seeing how the idea is being embraced or not.
The text was updated successfully, but these errors were encountered:
Also, if someone knows that data become outdated (big redesign of cycleways, changes in lighting system) then it would enable to update it.
It would also allow to review what was added by others.
Please feel free to comment. Also if you think you'd use it and in what way you would use it. If you can think of other layers to add, or an efficient interface etc..
It is not 100% clear yet if I do this or not, since the technical part is somewhat of an Herculean task and I am not sure if I still got enough time for it. Positive feedback can help. I did plan to do this as part of the BMBF grant but only got time to plan and occupy myself (with the details) it since this week.
I don't doubt that this is useful, but…
In this case, why not actually make it a different app?
All that said, I think I see a less extreme solution than what you are proposing, that can fit within StreetComplete more easily.
*By the way, this feature is exactly the type of thing I am talking about in #1654 when I have said I want to think deeply about quest presets and make sure we have considered ALL of the use cases. Same for #1914, and other issues mentioned at
Anyway, these are the goals you said we need:
I think appropriate presets would accomplish this, perhaps along with an indicator of what quests are available, as discussed in #124 (comment).
It is not quite as nice as what you propose, but I think it is a much better ratio of effort to reward.
I would certainly use it for micro-mapping tasks when I want to complete some streets / neighborhoods.
My first use case is always the same thing that I like doing : bicycle parking. It could work well for benches, fyre hydrants or any other urban furniture.
Right now, I've customized OsmAnd+ UI for that purpose (= adding the missing bicycle parking POIs), and I use SC to survey the existing POIs where some tags (bicycle_parking, covered, capacity) are missing.
To give an idea, this is how my OsmAnd+ looks right now
I use a lot the feature to save geolocated photo/audio notes in the app and then process them at home on my computer.
Also, the ultimate thing that I'm missing is a way to quickly visualise if and when a street or area was "completed" for a certain type of POI (= 100% of POI of type X from this street/area are supposed to be in OSM), and also define & assign survey areas as tasks to different people, like some kind of task manager to organize the surveying of a city for example, with the ultimate goal of making it 100% complete & accurate on one specific aspect of the map.
Well, that's just not in the OSM data. But anyway, you can assign survey tasks, indirectly: Just open a note and phrase it as a question (add a "?"). StreetComplete users will then see the note and contribute to it. F.e. you could ask for photos to be taken etc.
The OsmAnd thing is interesting. Can you actually tap on the bicycle parkings there and change the data? How does the interface look? Can this be done with OsmAnd also for non-POIs? (i.e., (cycle)ways)
Indeed, I was just dreaming up about the ultimate task management tool to organized perfect "cartoparties" (as we call them in french) between small groups of highly motivated users. But I don't believe it's StreetComplete's mission.
I don't believe the app allows editing of non-POI data, the menu of Osmand I'm using to display these is called 'POI overlay', and the native app plugin to edit osm data refers specifically to POIs.
You can find the app on F-droid to try it out if you want, it gives you access for free to the 'OsmAnd live' feature to refresh osm data as much as you want (on Play store, you either have to go with the free version of the app that retrieves osm data updates monthly, or pay a subscription to get updates more frequently). In the app, you need to activate the plugin 'OpenStreetMap editing'. The plugin 'Audio/video notes' is pretty interesting too and pleasant to use.
I suppose that's fair, although I wonder how big a barrier it really is given that this feature seems targeted at power users.
An alternative is that (according to this very old stackoverflow question, at least) it is possible for one app to put multiple entries in the launcher. I think if this were set up with care (making sure they always use different windows), it could behave to the user as two separate apps, which share some information.
So, if I get this correctly, the idea is basically having Quest "presets" (#1654) renamed to "layers" with minimal number of quests per layer and with each layer having it's own (CSS-alike) styling (or am I missing something crucial here)?
If so, I'd instead suggesting separate user-customizable presets (so user can put one or more quests in it, as they wish to map), and separate (also user-customizable) styling. OsmAnd does it is similar way, as that part of it IMHO works very nicely (OsmAnd ships with few presets and styles filled with defaults, but users are free to modify them and/or add new ones customized to their liking).
BTW if implemented, could multiple layers be used (and customized?) at once, in true GIS-style (thus increasing complexity), or only one-layer-at-the-time (thus highly decreasing usability)? The first (complex) one would be useful to me (unlike the limited one), but I'm afraid majority of current SC users (non-expert-OSM/GIS users) who are there for simplicity would not be happy with the idea of turning SC into such a GIS system.
Also, I see few problems with implementing this idea, and some notes:
My suggestion was a different app, which does not necessarily mean a new app. I already mentioned this in passing but I want to emphasize it: I think this idea would be a great UI for Vespucci.
@westnordost As you rightly point out, Vespucci suffers from showing too much data on screen at once. The ability to filter to different "layers" would make it so much more usable. It would be amazing. I would reinstall the app.
But StreetComplete already has a concept to limit and simplify the data shown, and that concept is the Quest. Everything in StreetComplete is quests, from its data structures to its UI. You've been reluctant to implement even multi-part quests because of this (#2127 (comment) and related), and so it is quite confusing to me that you now propose an even larger departure.
Forgot to mention, it seems like this feature is targeted at "power users" — historically not StreetComplete's target demographic, but they are Vespucci's demographic. The two apps are not really in competition, except to the extent that Vespucci's complicated UX makes it hard to use even for power users… who then turn to StreetComplete. A usability improvement to Vespucci would help SC too, because then power users will have less need of it, so SC can focus on being the best "introduction to OSM" app that it can be.
And I want to add that I totally sympathize with the technical side of things — SC's code base is nice to work in, and it's much easier to work with what you know or build green field than jump into an unfamiliar code base. It just doesn't sway my opinion about which is the right design decision :)
@mnalis Well, no not really it is not the same as quest presets. As @smichel17 noted, it is quite a departure from the quest concept. But StreetComplete is not defined by its quests, its defined by its ease of use. The new concept is still easy to use, while covering a use case that is more attractive for power/craft mappers because it is more efficient.
The quest system has its limits of what can be done because for each quest it requires an algorithm to determine whether a data point is missing or not. This is simple in some cases (
Taking the example of housenumbers, an interface like shown in the mockup above would be by far more efficient and unproblematic than the quest version of it (i.e. not dependent on building type bein set first, not disabled in certain countries, not having a complex algorithm to determine if a housenumber is really missing etc ...) because of one reason: The user can have this overview himself, and see by himself if something is missing, no algorithm required. This is one big advantage all the other OSM editors have over StreetComplete.
@smichel17 In regards to Vespucci - it wouldn't surprise me if you could already do that in Vespucci somehow, after all you can also add custom presets etc. as far as I know. The general UX approach of Vespucci (and JOSM) is to grant the user maximum flexibility. So, if this would be implemented there, it would probably look and work very similarly to filters in JOSM:
The problem about Vespucci is that it can do so many things, that even if such a feature would be added, it would easily be buried amongst the many other settings and customizations one can do.
This is why I initially wrote that such a layers feature would need to have such a "handcrafted" UI and smart filters just like the quests: To preserve the ease of use and to not confront the users with such customization.
I am just going to tag @pietervdvn , author of MapComplete here because the approach is quite similar, maybe he is interested in reading this and might have an opinion about it. Same with @GuillaumeAmat, author of MapContrib. @zlant created the parking lanes viewer, which is a very similar concept as the one here.
You can already do that in Vespucci. There are two types of setting you can use to filter things. Tag filter allows particular tags to be shown. Preset filter allows presets or groups of presets to be shown as a filter. With the beautified JOSM preset used in Vespucci there are plenty of possibilities.
For example if you filter by ways sub-group of the highways group it will show paths, tracks, dedicated footways, steps, dedicated cycleways, segregated foot/cycleways, unsegregated foor/cycleways and dedicated bridleways only.
It's very best for working with POIs since you can both add new POIs and edit existing ones in all ways. The problem with ways and areas is that whilst you can move the whe way/area around and edit the tagging you cannot edit the position of individual nodes that are part of a matching way/area unless that individual node also matches the preset or preset group being filtered.
So it's circumscribed by the presets loaded and impossible to use to improve the geometry of ways and nodes because of the nodes issue mentioned earlier. However the presets issue can be worked round to some degree because Vespucci is compatible with JOSM presets. So in theory you could create your own JOSM preset file to define exactly what you'd like to filter, subject to that nodes issue.
At least iD, JOSM and Vespucci can filter the displayed data and substantially reduce what data is displayed.
Vespucci, since way back in version 0.9.9 (and JOSM has this for a while too now), doesn't just support a tag based filter, you can simply select a preset or a group of presets and only matching items will be displayed (if you only have one preset selected the preset will be automatically applied to any new objects created).
So for example if you want to map bus stops you just select the bus stop preset and you will
a) only see bus stops
the advantage is naturally that you can always turn the filter off and get "everything".
In any case I would be careful about doing this the other way around, that is by just downloading objects that correspond to certain criteria, this is just asking for things to go wrong because relevant data is missing in what is being edited.
@davidpnewton has just pointed the above too. It is true that modifying way geometry may not be possible in preset filter mode, but that would be easy to fix, just as in tag filter mode which has a flag that indicates if way nodes should be included for matching ways or not. It is simply a case of nobody asking for the feature up to now.
I don't want to waste too much time on something for which the outcome is already predetermined, but I do want to point out that surveying one feature (type) at a time isn't efficient at all. The time consuming aspect of surveying is getting there (and back) and actually surveying, not the data entry (within reason naturally). so a well prepared survey that can capture everything of interest is always going to be far more efficient overall than a single topic undertaking.
You have to press on the "..." button in the lower right corner, then select "Preset Filter". A small button appears in the vertical center of right side of the screen. You have to press on it to be brought to a preset selection screen. I did not find out how to filter by for example "all roads", it seemed to require me to click on the most precise one.
The functionality is very similar to JOSM. There is no special highlighting of values of certain properties (f.e. lit=yes/no), seeing and editing the data one is after requires to go into the edit-tags-screen. Showing the data directly on the ways for the big picture and editing them in a simple UI that does not require to change the screen would be significant parts of my idea.
Here is a screencast, for reference.
This? https://vespucci.io/tutorials/data_styling/#validation-styling seems to require that I write such a XML file on the computer and then put it into a certain directory on my smartphone. Then, I can somewhere in the settings activate that styling instead of the default one.
The styling is for certain kinds of validation issues (and for data in general), for example you could build a style that uses a different colour to highlight unjoined way nodes. But it is not something that users customize on the fly.
Some of the validation rules are hardwired, some are not, in particular the missing tag validation works by highlighting objects that are missing a (required) tag that is in the matching preset.
To use the example at hand, assume you want to highlight all objects that 'should' have a 'lit' tag you simply add 'lit' to the list of keys in the validation configuration and any object that matches a preset that requires 'lit' but doesn't have it gets highlighted.
The advantage of doing it this way is that the knowledge which tags are used for which objects is concentrated in one place, the presets and doesn't require duplication all over the place. Which is also why preset filters are an elegant solution to the filter problem. Yes there is some duplication of this in the data styling which is annoying, but is not easily avoidable.
But this issue isn't about Vespucci design, so over and out now.
By now, I have read through the entire thread and recognized a lot in my work on mapcomplete.
The core idea of mapcomplete is exactly this "one specific theme of interest with easy update capabilities". This is not because of a power mapper who wants to update a certain feature, but to appeal to a certain target audience, e.g. a map for cyclists, a map for climbing, a map for vegans, The core idea is to go to the cyclists/climbers/... and tell them Hey, I have made a map specifically for you! It shows all those things you care about!. When they answer yeah, but you missed POI xyz/xyz is outdated, we answer them It's openstreetmap! Create an account and update it yourself! Then it'll even show up in OsmAnd, maps.me, and much more!. It is basically luring them in ;) . In other words, we give all those communities the means to (help) maintain the data of their interest. Also, the potential users of this are way bigger then the power mappers.
At the same time, MapComplete is setup to grow together with the new contributor. The editor unlocks further depending on the number of changesets the logged in contributor has! Around changeset 50, the applied tags show up in grey; around changeset 100 those small tags become hyperlinks towards the wiki; at change 200 an introduction panel on MapComplete is added and upon changeset 500 the custom theme generator is unlocked).
I have the feeling that StreetComplete is with that respect the opposite of MapComplete. StreetComplete feels more like OpenStreetMap is cool! You can help out by answering all these random quests! It is like catching pokemon, just as fun but this one is useful! Now, I must admit, I have edited the StreetComplete-app more then once to create a custom quest (first time to survey sett types in bruges, second time to determine road widths). And SC has a few technical capabilities (is there a sidewalk/parking lane/cycle lane on the left/right side; the 'cut a street') which are top notch for those... If I can, I want to implement something similar in MC.
Furthermore, I would like to add two important, general thoughts. First of all, there is always the concern of 'simple to use' and 'lots of features'. Lots of features implies lots of buttons, lots of things to learn, so the inherent complexity of an app with a lot of capabilities will always bleed out. We can try to simplify things with a good UI and avoiding as much as accidental complexity. IMHO, we are having editors all along the pareto front for all those niches, from powerful but difficult to easy to use we have Level0, Vespucci, JOSM, ID, MapComplete, StreetComplete, ...
The second thought is on the balance between showing everything versus showing a filtered view. I had the case that a total newbie added a new bicycle pump with MapComplete. I later updated some tags with iD, after which I got an angry message from mapper X that the bicycle pump was tagged twice. Turns out that mapper X mapped the pump with a totally different tagging years ago, tagging which we all updated before rolling out the map (but I missed this one - twice).
The anecdote is to indicate that, if we decide to not show all the data, the integration between the data might suffer. If we for example only show cycle paths in iD, and we align it with the imagery, we might miss that it now crosses a building and that we have to align the buildings too!
This implies that there is some inherent complexity to handle here; we cannot get around this with clever UI-tricks. Either we allow all operations (editing geometry) or we simplify the UI by not showing certain objects, but we choose not to allow everything. For MapComplete, one can add a new POI or change tags on points and ways, but I draw the line at modifying geometries which'll never be possible (in a future version, splitting a road might be possible though, but I do not consider this changing the geometry.) Even this causes some problems already, e.g. when a user 'fat-fingers' and accidentally adds a new POI in a building while it is outside, just around the corner, ... (Now, this is solvable by UI by having to zoom in even more, but I still have to fix that)
These were my general thoughts on the subject, let me expand a bit on MapComplete specifically...
With MapComplete, I
So, yes, it is possible to easily create your own theme - like OpenMapKit. One can create their own theme, the complete configuration for the theme (a json file) is then encoded into the URL and can be sent of. There is a list of such unofficial, user generated themes on the wiki. Some of those themes are brought back into the official mapcomplete, if they are well worked out and useful. Others have already made their own themes and contributed them back or setup their own forks. (I have to admit that the custom theme generator isn't very user friendly for now).
And of course, for the power mappers who are interested in a lot of features to survey them all at once, there is the [mapcomplete.osm.be/personal](personal theme) which allows one to enable layers from all other themes, including previously visited unofficial themes! (Yes, installing an user-created theme is as simple as visiting it once. The theme configuration is saved into the user preferences of the currently logged in user).
To summarize, MapComplete is basically pretty much everything discussed here in this issue, except for a few missing technical features. Implementing this issue into MapComplete would boil down to enabling a multiselect, but , to be honest, the moment a contributor needs to edit information on multiple nodes at the same time, I think they'd have outgrown MapComplete and have to move on to iD or even JOSM.
Phew, that was quite a wall of text!
So, @westnordost : I hope that my insights will help you to make a decision. In my personal opinion, I would not take StreetComplete on this path. First of all, you would get into exactly the same usecase as MapComplete, and second it would complicate the UI a lot for other users. At last, it is more useful to have a thematic map as website (as it can be emmbedded and easily shared) then as an application - especially because MapComplete has a webmanifest and can be "installed" too (it is but a glorified bookmark on the home page which opens without browser UI elements, but it does the trick!).
Thank you for your insights, @pietervdvn . I will read through it later in more detail and also give a detailed response, maybe post some feeback / improvement suggestions on your issue tracker. For now, I just tried out one layer, here is a video for reference:
That is an excellent example of a layer showing 'red' when the data is missing. It is an unofficial layer, made by a spanish person (the english can be improved here and there). But, I'm not fond of the preset where you can indicate how you perceive the 'lit' at a certain point... It shouldn't be in OSM.
Second, the bug that one is not being able to exit the preset screen is what I'm fixing right now
The idea proposed in this ticket works for data that can be recorded and verified with aerial imagery just as well as for data that can only be verified on-site. So it makes sense to build an app that would be capable of this as a web-app - so it can run on a smartphone while being on-site and also on the desktop PC. So, it fits quite well that MapComplete is a web-app.
There are a few decisions you took that I'd do differently in StreetComplete. Conceptually,
From the UI perspective (some may be bugs or missing features, or a matter of taste):
There still is room for some highly specialized UI-components though, which can be parametrized too so that they can write different tags:
A drawback of having multiple themes is that people can get confused when switching themes. On the other hand, it is perfectly possible to create a 'batteries included theme' which questions a lot of stuff (i.e. a remake of StreetComplete) and which does not show the links to other themes.
About your UI-remarks: jup, there is still quite some work to do. Keep in mind that MC is pretty young - development started only around 8 months ago and I didn't prior webdev experience (lots of backend experience though), so it's been quite an adventure...
Thank you for all the feedback, you all, especially @pietervdvn !
I've decided to try to do this.
In November, I interviewed a dozen mappers who regularly contribute data on the ground to explore whether my idea would be something that could find its way into (craft-)mappers surveying workflow. Such a qualitative survey is of course not representative, it served me just to explore whether the idea is worth pursuing further and get a very rough understanding which use case might fly - how it would need to work UX-wise. The feedback overall was quite positive.
To sum it up, of the interviewed, for field mapping
A bit over half the interviewees answered that if they are on a survey, they usually try to map "everything".
My overall impression was that the layers approach would not be adopted that much by those who want to map "everything" and are already accustomed to using the photo-mapping + JOSM approach. After all, measured by data added vs time used, I think this is amongst the most efficient approaches and always will be.
So, first I'll only implement it as an experimental feature to see if it is picked up. So, I will also severely limit the number of layers which will be available. As mentioned, the plan would be to have - even later - just a couple of layers, with only such data that must be collected on-site and where it is really necessary to see a big picture (f.e. networks of things, relation to nearby things).
If done right, it is a massive effort. Right, in my book, would be that download and upload continues to work automatically (by default), that the downloaded data is persisted as well as the changes made, that it continues to work offline, including that changes made are visualized directly (that's the biggest chunk). Stretch goals are that splitting ways and deleting works, including offline too (hello again, big chunk), undo function.
As you know, I am currently funded by the Prototype Fund. As the name implies, it should serve to try new things - they may or may not be so successful - but that's the purpose of it, to be able to do innovation. So, when to try something new if not boosted by such a fund? Never. So, I'll do this now.
If done right, the massive restructuring effort will also have a very positive effect on how the quests work: New quests will immediately be unlocked even before the map is downloaded anew and even before the answer is uploaded. This missing feature is the reason for countless bug reports I have to close as will-not-fix due to the known technical reasons (explained in the first post).