Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Guidelines for adding new features #1630

Open
gravitystorm opened this issue Jul 3, 2015 · 115 comments
Open

Guidelines for adding new features #1630

gravitystorm opened this issue Jul 3, 2015 · 115 comments

Comments

@gravitystorm
Copy link
Owner

In a different issue @kocio-pl says:

"AFAIK there are currently no policy on the amount of tag uses yet. It's just a matter of individual taste and feeling."

I don't want thinks to be unpredictable, so we should come to a collective decision rather than it depending one individual taste. But it's a tricky subject, and causes a huge amount of bad feelings, since every mapper wants every tag to show up somewhere, and unfortunately for us, that somewhere is almost always here.

I have two guidelines that I'd like to see implemented:

  • For every new feature added, (at least) one feature is removed
  • For any feature to be considered for this style, it must already be on one other osm.org front-page style

Basically, numbers-of-features based rules aren't usable. Just because there are millions of X doesn't mean we should put them on this map style. And just because only a few hundred Y have been mapped doesn't make them unsuitable.

When it comes to adding new things - well, there are roughly hundreds of thousands - perhaps millions - of different "things" in OpenStreetMap and they almost all could be added to this style. If we add things and don't remove things, then the map style will show hundreds of thousands of different things, and that's not tenable. It stops being a map and is just a complex, unreadable, useless and ugly visualisation of the database. So we need to have a plan to remove things if we keep adding more and more. I know people disagree with me here, and claim that there's no downside to adding more features (or new features won't be drawn in the same place as existing ones, or can't see the problem with having millions of different icons) but they are wrong. And the way we are doing it at the moment - where we add new features every week since "one or two more can't hurt" is unsustainable.

For the second point, if a feature is so obscure that no other rendering - even including specialist renderings - contain that feature, then I think it's inappropriate to add it to a general style. It should be the case that the main map style is a good summary of the data available, and then if you want to see more information on a topic, you can switch to another, perhaps specialist, map. Having features appearing only on the general style and not elsewhere is completely back-to-front.

Other proposals are, of course, welcome!

@kocio-pl
Copy link
Collaborator

kocio-pl commented Jul 3, 2015

I also think this will be hard to reach general agreement, but anyway the problem is worth discussing.

We have different backgrounds for a start - for example you're the lead developer of the whole style (and not only this one), so you tend to see universal things, while I work mainly as a micromapper, so I see the world mainly through the z17+ zoom levels.

For me even the most crowded POI space according to some mappers is just mildly saturated with informations on z19 with a big hole in the center (and I see missing icon for Fired Earth shop, because the name tells me nothing, while we have needed info in our database). But on the other hand I don't remember this style may be used on other servers and for other purposes than only main OSM website.

This is not a bad thing per se - quite the reverse: I thing it's rather healthy. The more different people, the more chance to strike the balance. That makes everything much harder and longer, but that's the price of trying to do the best we can.

I would ask some questions before we start drawing the lines:

  1. What are the most important goals of general map?

I would say it should show the things common people wants to find (and the most visible things, especially on micromapping level). On the lowest levels that would be physical view and readable country borders and names, with capitals visible early enough, on the medium level that could be readable roads, cities, touristic attractions, the borders of campuses and many other institutions and nature forms (IMO we excel at this and are still getting better, unlike on the lower and higher levels), and on the highest zoom levels that should be POIs (readable icon shapes and names when possible), highways shapes, trees, flower beds, overground pipelines and maybe even fire hydrants (red dot at z18+ is comparably important to black dot for bollard, which we have a lot by now). It's just a rough list, but I think you get the vision.

For me the rule of removing as much as we add may be the most correct at the medium level (however maybe that's too strict even there). For the lowest levels it's rather reworking things than adding/removing anything, and for the highest levels there's a lot of essential things which are not visible yet, so I hardly see the need to remove anything for now.

  1. The numbers on Taginfo is one thing, but what about Wiki status? If we don't take the numbers blindly, maybe we should somehow look also if the tagging is "deprecated", "approved" or "de facto"?

@imagico
Copy link
Collaborator

imagico commented Jul 3, 2015

  • For every new feature added, (at least) one feature is removed
  • For any feature to be considered for this style, it must already be on one other osm.org front-page style

I would be for following the first - not necessarily in a strict 'quid pro quo' sense but strongly suggesting that anyone who suggests or creates PRs for adding new features should also make similar contributions w.r.t. removing.

But i would be strictly against the second. This is a typical 'preserving the status quo for the sake of conservation itself' rule. Innovation and progress need to start somewhere and since the standard style is the only one that is based on at least partly democratic control and does not have a strict thematic focus this rule - when obeyed strictly - will seriously cripple the ability to develop the style in a way that reflects mapping priorities in OSM.

In general i think formal rules are not very beneficial in this context, especially since people tend to read them the other way round - like if i create a PR that adds a feature that is already shown in another map and removes another one it will have to be accepted.

It seems to me that much of this matter is centered around POI icons which make up a large portion of recent PRs. I think these are a special case in the much broader field of map design in general and many of the problems here are related to the technical constraints that currently exist. Let's not implement global rules on map design just based in this specialized matter. In general improving readability of the map has often very little to do with how many types of features are shown but with how they are shown (and POI icons are often simply a fairly wasteful way of showing things).

Instead of a set of rules/guidelines i would create a checklist of questions that are to be considered to decide if a new feature is to be rendered. What comes to mind here is:

  • is the tagging to be rendered the established tagging in OSM for this kind of feature, i.e. are there no competing tags?
  • is use of this tag reasonably widespread compared to the occurrence of features that could be tagged, in particular consider the geographic distribution of features and the number of mappers using it?
  • is the quality of mapping of this type of feature sufficiently high on average (like accuracy of position and level of detail and accuracy of the geometry)?
  • is the meaning of the tagging sufficiently clearly defined and documented on the wiki?
  • is the feature itself of sufficient interest for the average map user to deserve being shown?
  • is the average map user sufficiently familiar with the nature of the feature to understand it at least with the help of a map key?
  • is what distinguishes this feature from other similar features that are either already rendered or not rendered but tagged differently sufficiently clear to the average map user?
  • is this feature in distinct form shown in some other kind of map (not necessarily OSM based, not necessarily digital)?
  • is it feasible to render the feature together with other similar features in a common styling?
  • could the feature be rendered as a styling variation of another feature that is already in the style instead of being rendered as a separate map element?
  • does the proposed styling work reasonably well on all zoom levels it is shown in all regions the feature occurs in?
  • does the suggested styling well represent differences and similarities to other features rendered?

The answers to these questions of course depend on your point of view, such list would not have the purpose of making the decision for you, it would be a helper for those in a position of making the decision (making a PR or merging a PR) and you could use your detailed answers to these questions to support your decisions.

And i said before and will repeat here: One of the most important aspects of style development i see much too little here is the critical review of changes after they have been rolled out. This in particular applies to the addition of new features.

@matkoniecz
Copy link
Contributor

@gravitystorm

For every new feature added, (at least) one feature is removed

I agree that we are at point where removing features may be better than adding more - for example in my road redesign I decided to display highway=motorway and highway=trunk using nearly the same style (I am copying it from Humanitarian style, the only planned difference is zoom level where road appears). I also want to get rid of highway=proposed.

Though it would be tricky to define how features are counted. 100 shop types displayed as a purple dot - is it a single feature or 100 (my bet that it is single)? Lets say that there are 27 road types that may be displayed with red dashes or not marking whatever road is private. Is it 54 features or 28? 27 road types where each may be private/public and paved/unpaved. Method for displaying access and surface status is the same across roads. Is it 29 features or 108? I think that it would not work as hard rule, but "removing features is not only acceptable but also welcomed" guideline would be great.

Also, features are not equal. Displaying shop=toys and shop=kiosk differently has lower impact than displaying highway=motorway and highway=trunk differently. Different icons for shops are quickly increasing number of features - but impact of making map more confusing is not high as these shop icons are quite intuitive. And in the worst case icon colour should make it clear that it is some kind of shop. Impact of #1239, #775 or #720 would be much more significant (as measured by increasing complexity of map) than one more shop icon. My feeling is that removing highway=proposed and adding 50 new shop icons would be balanced.

Basically, numbers-of-features based rules aren't usable. Just because there are millions of X doesn't mean we should put them on this map style. And just because only a few hundred Y have been mapped doesn't make them unsuitable.

I think that any request for rendering tag with less than 1000 occurrences and without really good explanation why it should be exempt from this rule may be rejected without discussion. "not used widely, therefore no rendering" allows to close majority of requests for new features without pointless discussion. It is especially helpful as it is common that people proposing to render rare tags are ready to debate endlessly. For example in case of #1239 effort to discuss, implement and test rendering is greater than effort that was needed to map existing occurrences of that tag.

And what may be a good explanation - for example one for rendering natural=shingle from #1217 ("In the code you can see that i included natural=shingle to be rendered like natural=scree to avoid mappers feeling additionally inclined to tag natural=scree despite this being technically wrong."). capital=yes would be clearly exempt as useful tag to describe really rare feature. But "this tag should be promoted" is not a good reason.

For any feature to be considered for this style, it must already be on one other osm.org front-page style

I would expand it to "style with rendering for the whole planet, updated at least once a month". That would include for example French-style renderer.

@dieterdreist
Copy link

2015-07-03 16:01 GMT+02:00 Christoph Hormann notifications@github.com:

  • For every new feature added, (at least) one feature is removed

  • For any feature to be considered for this style, it must already be
    on one other osm.org front-page style

    I would be for following the first - not necessarily in a strict 'quid
    pro quo' sense but strongly suggesting that anyone who suggests or creates
    PRs for adding new features should also make similar contributions w.r.t.
    removing.

+1, sounds reasonable. Making the removal a hard requirement might lead to
nonsense for the sake of following the rules.

But i would be strictly against the second. This is a typical 'preserving

the status quo for the sake of conservation itself' rule. Innovation and
progress need to start somewhere and since the standard style is the only
one that is based on at least partly democratic control and does not have a
strict thematic focus this rule - when obeyed strictly - will seriously
cripple the ability to develop the style in a way that reflects mapping
priorities in OSM.

+1, it is also somehow arbitrary what is showcased on the front page, and
there are requirements like global availability that reduce the number of
potential new maps there mostly to people doing business with OSM (because
of the required ressources to offer an up to date global rendering that can
cope with the number of requests from osm.org).

Generally we should keep in mind that this is the main style of a global
project, and that the project is likely still "young" in a certain sense.
"We" are naturally already biased towards the "western world" and under the
influence of western culture / point of view (inevitably, just have a look
where we come from and where "we" live), and so are all our maps on the
front page (incl. HOT). With this requirement we would make this situation
even less changeable / open to changes that don't fit into or a needed in
the western world (and often don't matter because the features in questions
don't happen to occur in the western world).

In general i think formal rules are not very beneficial in this context,

+1

@pnorman
Copy link
Collaborator

pnorman commented Jul 3, 2015

Basically, numbers-of-features based rules aren't usable. Just because there are millions of X doesn't mean we should put them on this map style. And just because only a few hundred Y have been mapped doesn't make them unsuitable.

While I agree that they aren't always suitable, for shop and similar POIs where we have a generic fallback, I think it's a reasonable first pass.

More throughts from me later

@joostschouppe
Copy link

Definitely agree with the general idea. Really like the idea to use rendering questions as a means to improve documentation as @imagico seemed to suggest.
It might make sense to introduce some kind of pop poll for suggesting additions and removals. Then popular slots for addition could be freed up only if and when some removal suggestion reaches critical mass.This would avoid having to devise logical rules, and would just leave it to the community.

@kocio-pl
Copy link
Collaborator

kocio-pl commented Jul 5, 2015

Also, features are not equal. Displaying shop=toys and shop=kiosk differently has lower impact than displaying highway=motorway and highway=trunk differently. Different icons for shops are quickly increasing number of features - but impact of making map more confusing is not high as these shop icons are quite intuitive. And in the worst case icon colour should make it clear that it is some kind of shop. Impact of #1239, #775 or #720 would be much more significant (as measured by increasing complexity of map) than one more shop icon. My feeling is that removing highway=proposed and adding 50 new shop icons would be balanced.

While I agree with this conclusion, it reminds me of important hint we should consider crafting (not just adding or removing) any feature on the map:

  • Mind the scale.

The most general scale guide would be like:

  1. Big scale - worldwide features, mainly natural ones (like continents and the oceans), but also country borders and capitals.
  2. Medium scale - roughly speaking: anything from country to city level.
  3. Small scale - anything from part of the city to street/building level (so called micromapping).

They are different beasts, really: most important big scale objects are areas, most important small scale objects are points, and medium scale is associated mainly with lines (however there is also mix of important areas and points involved).

Now - we tend to focus on medium scale and see virtually every feature through these "lenses". However the impact of displaying highway=motorway and highway=trunk differently is far less on both extremes than:

  • displaying the country borders differently than state/province borders or showing as many capitals and country names as it's possible (on the big scale)
  • displaying one kind of the shop differently than the others or displaying the real shapes of highways (on the small scale).

We may even treat the medium scale as a special case and take more care of it, but if something is less visible or just doesn't belong there, we should - metaphorically speaking - change our regular glasses for telescope or microscope, respectively.

@gravitystorm
Copy link
Owner Author

Instead of a set of rules/guidelines i would create a checklist of questions that are to be considered to decide if a new feature is to be rendered. What comes to mind here is:

My point about using other styles as a yardstick is to avoid exactly this outcome. You've listed 12 perfectly reasonable points that should be considered, but you're advocating that all those decisions should be made in this one style in this one repository. But we're already overwhelmed by conversation, and I'm trying to diffuse the amount of work and angst that goes into this style, not increase it further.

If you're unhappy about the number of styles on the front page, then we should either solve that (by increasing the number of styles) or change the rule (to include a wider acceptance criteria, as @matkoniecz suggested). But openstreetmap-carto will probably collapse under the weight of discussion if we need to go into such depths ourselves. I would like to expand this scrutiny effort, using the distributed knowledge of all the other people working on their own renderings, not just concentrate everything here.

@imagico
Copy link
Collaborator

imagico commented Jul 6, 2015

My suggestion was not meant to advocate more discussion. It is meant as a helper for (a) contributors to check their requests for adding new features to be viable before making them and therefore avoiding fruitless discussion and (b) style maintainers to make fast yet transparent decisions on closing issues/rejecting PRs without a pointless discussion.

I completely understand your desire to keep style development here focused on the important things and avoid the style degenerating in quality and readability - i just don't think this particular regulation would have a positive effect.

One thing that aggravates the problem of excessive unproductive discussions here is that there is no forum for generic OSM map styling discussion elsewhere. A lot of discussions started here are on a fairly generic level that is not really suited for this place. Discussions on how a certain POI tag can in principle be visualized with an icon for example are something not specific to this style and could IMO be separated from actual style development. Managing discussion here in a way that decidedly directs subjects on a level that is not specific to this style to a different place would help a lot. This certainly covers a lot of new feature discussions. If on the other hand someone has a concrete plan how a certain new feature could be shown in this style based on extensive research into the possibilities and suggests this here it would IMO be very bad if he/she would be turned down based on the formal argument that this feature is not shown in any other qualified OSM map without considering the merits of the suggestion itself.

I would like to expand this scrutiny effort, using the distributed knowledge of all the other people working on their own renderings, not just concentrate everything here.

That is a good idea, if however you put the barrier that high (for example like @matkoniecz: global maps with regular updates) the primary effect would be removing a possibility for style developers to contribute to OSM when new features are involved, especially those with limited ressources (who cannot run their own global map) and those not from regions with a large and well organized local community who have their own local focus style (like in Germany or France).

@brycenesbitt
Copy link

A problem: There is no truly "open" rendering style in OpenStreetMap. The database is open: people can and do add whatever they want. There's no equivalent on the rendering side: renderings are like castles with moats, with the hordes massing outside banging on the gates. The maintainers inside protecting the integrity of the rendering, and the masses outside complaining, are a hard dynamic for everyone.

The way out may be more top level maps, including a true "bagel with everything" approach map with few rules, another new map focused explicitly on motorists, and an open community version of cycling, public transport and political maps. Organize this all on a new mailing list called rendering@openstreetmap.org, so osm-carto does not become the only focal point for discussion debate, hopes and dreams. Offer a mix of "curated" maps created with the integrity of a single voice (such as OpenCycleMap), and "open" maps created via a messier process.


@gravitystorm "For every new feature added, (at least) one feature is removed" is a particularly moat-like rule that ignores the fact that rendering is not a zero sum game. A small community could get it's feature rendered with little to zero impact on anyone else. Deciding to render trees or fire hydrants or road signs is a big deal, because there are so many. Other features (and here "motorhome toilet dump stations" or "farm product stands" come to mind) have essentially zero impact on anyone else, because they are located in areas that are otherwise empty on the map. Very close to exactly nobody is overwhelmed by clutter for such items, as there is no clutter.

@matthijsmelissen
Copy link
Collaborator

@gravitystorm I'm happy you started this discussion. My point of view:

For every new feature added, (at least) one feature is removed
For any feature to be considered for this style, it must already be on one other osm.org front-page style

I don't think I'd be personally in favour of either of these guidelines.

the map style will show hundreds of thousands of different things, and that's not tenable.

I think we can distinguish three (related but distinct) issues here:

  1. displaying too many different features
  2. using too many different symbols
  3. displaying those symbols on too early zoom levels

@gravitystorm Which of these do you think are problems with our style?

In my opinion the problem is 3., and a bit of 2. I don't think 1. is a problem.

For example, rendering things like hackerspaces would be no problem to me at all if we render them with text only (no icon, addressing 2), and render them only on high zoom levels (addressing 3).

It stops being a map and is just a complex, unreadable, useless and ugly visualisation of the database.

I think that's the core issue. We do really bad at the issues you mention. I would expect though that we can do a lot by simply changing minimum zoom levels, and maybe even by toning down colours.

I think having concrete examples makes things easier to discuss.
@gravitystorm, what do you think of the example given by @kocio-pl? Too me on z19 the map is readable, useful and not terribly ugly. Would you prefer a reduction in the number of different icons, of even a reduction in the number of different displayed POI, here too?

@kocio-pl
Copy link
Collaborator

kocio-pl commented Jul 6, 2015

@brycenesbitt +1

I like the democratic, open aspect of this style very much, even if it's limited by many aspects. But I hate feeling like a "hordes agent" (?) inside the castle, just because I want to try some things rather than just keep the house tidy. I would be happy to work with some "development"/"working"/"for mappers" version of this style (less strict, with more features and testing stuff), but when I recently asked if new hardware donations could be used for this, @pnorman said it won't.

We definitely should have some place to talk about more general things related to rendering. Mailing list or maybe subforum on OSM forum would be easy to set. This alone could not resolve the problem of "open" rendering style, of course (talking without a chance of getting the real output will be harsh exercise in frustration), but that would be a good start.

The big question is how to make the rendering in OSM more open, collaborative process in a peaceful way, without anybody feeling ignored or being under pressure.

@pnorman
Copy link
Collaborator

pnorman commented Jul 7, 2015

I have two guidelines that I'd like to see implemented:

For every new feature added, (at least) one feature is removed
For any feature to be considered for this style, it must already be on one other osm.org front-page style

The style is currently in a place where removing features is more valuable than adding them, but I'm not sure that this is the most effective way reign this issue in. On the other hand, I don't really have an alternative suggestion.


I get the feeling that some people view this style as an OpenStreetMap project to design a style for osm.org. I do not consider it that. I consider it a project started by Andy for a style that shows off OSM data, and which is shown on osm.org. I have an interest in the latter, I would probably contribute much less to the former. Remember, Andy chose to put this project under @gravitystorm, not @openstreetmap.

Part of showing off OSM data is good cartography. Part of good cartography is being selective in what to render, which means not continually adding new features to the map.

@dieterdreist
Copy link

sent from a phone

Am 07.07.2015 um 10:32 schrieb Paul Norman notifications@github.com:

I consider it a project started by Andy for a style that shows off OSM data, and which is shown on osm.org.

this is not a project started by Andy AFAIK, but the continuation of the OpenStreetMap main map (mapnik, the former "Chilton"-style), the only one published on osm.org that is generated with OSM(F) resources.

@matthijsmelissen
Copy link
Collaborator

Part of good cartography is being selective in what to render, which means not continually adding new features to the map.

Do you have any concrete suggestions of features that you'd prefer not to render?

@imagico
Copy link
Collaborator

imagico commented Jul 7, 2015

@pnorman - yes, this is Andy's project but being the style that is rendered on the main OSMF infrastructure comes with responsibilities to fairly deal with the wishes from the community. The OSM community needs to have a say in what is rendered on the project infrastructure and what is featured as the main map on the project website. Allowing everyone to directly edit the style would not work of course, so this influence has to be indirect. The current construct with a group of style maintainers responsible for channeling and structuring the requests and suggestions seems to work quite well though. (slight critique here: style maintainers currently all have a fairly technical background and approach to things. This is natural to some extent - merging pull requests and reviewing changes in a primarily technical task. But a more cartographic/geographic view of things can be helpful sometimes. I am not saying this needs to be changed or even that it could be changed, just an observation).

@math1985 - the primary problem about POIs IMO is the lack of a reliable prioritization system. As soon as not all POIs can be shown selection gets arbitrary.

And yes, there are areas where z=19 is overcrowded with POI icons or at least would be if those were fully mapped in these areas:

http://www.openstreetmap.org/#map=19/38.71272/-9.13924
http://www.openstreetmap.org/#map=19/1.29943/103.85460

I for one think one of the most pressing needs in this style is improving overall consistency in styling - features that are similar in reality should be similar in rendering and things that are different should be shown differently. This is often not the case currently but it is very hard to to improve since even small changes in styling can have enormous side effects on the overall look and readability. It is much easier to just add a new feature than to change existing styling without causing a mess. This is probably one of the main causes of feature creep here.

Maybe it would be a good idea to open a new separate issue to collect some ideas for what features could be well removed from the style.

@kocio-pl
Copy link
Collaborator

kocio-pl commented Jul 7, 2015

Remember, Andy chose to put this project under @gravitystorm, not @openstreetmap.

I guess this is exactly where most of the tension comes from: the "private" style (designed mostly by few people with strong opinion how should it look like) being default visualisation for the community project. This community is rapidly growing, so I think the clash is unavoidable.

And this is not so clear from the beginning: the license is open, the GitHub makes participation easy, and there are many leaders in FLOSS world, so using private namespace for the project may suggest just the strong leadership. Nothing besides that really warns that you are on kind of the "private" ground and while the leaders are polite, they don't really like being "leaders" at all - they have their own clear vision and just allow others to participate to some extent.

Part of showing off OSM data is good cartography. Part of good cartography is being selective in what to render, which means not continually adding new features to the map.

I understand it. But on the other hand - part of community GIS project is having community influence on all parts of the process.

Try to imagine Wikipedia with lot of people writing things as they like it ("any tags you like"), but what gets showed at the main website are only some featured articles carefully selected by the few "super-editors". Some other thematic selections can be found, but most articles are not shown even if they could be, just because super-editors think the readers would be overwhelmed - no other encyclopaedia contains so detailed articles, anyway, so what's the problem?

Even Wikipedia has some limits what it can contain and what are formal requirements, but if it conforms these general rules, nobody limits it anymore - all the articles are visible.

@mboeringa
Copy link

One thing that is really missing in this discussion is the option of the real time creation of "toggable" on/off thematic overlay layers of certain types of objects on the OpenStreetMap Rails web application via Overpass turbo API calls instead of trying to pre-render everything in a style.

This would circumvent the impossible task of trying to display everything, as @gravitystorm justly points out.

This would allow dedicated, specialized thematic rendering, without cluttering up the base map.

There are several existing implementations of this already, but unfortunately, on websites separate from the main Rails web application, and therefore not found or known by the average user. A good example is Marc Zoutendijk's OpenPoiMap:

http://openpoimap.org/

I really think adding such an option to the main Rails application, would lessen the burden on carto and other styles displayed on the main website, and potentially sharply reduce the number of requests for features being rendered.

Of course, this is not a solution for all types of objects, especially the ones needing layered rendering, but POI icons are a definite candidate.

@gravitystorm
Copy link
Owner Author

Nothing besides that really warns that you are on kind of the "private" ground and while the leaders are polite, they don't really like being "leaders" at all

This is quite insulting, but I'm not sure if you are aware of how much so? I mean, I find this DEEPLY insulting.

I put a lot of effort into the initial port, and I made sure that the whole project - not just the code - was open to other people to get involved. I've built a whole team here: I've added multiple maintainers - all of whom have the same authority as I do - and we've accepted pull requests from many different people and put a whole ton of effort into helping dozens of new people - including you - get started, learn how things work, get your changes processed and incorporated. But you boldly state that this is still a "private" project with insufficient "community influence"?

And you make these statements on a thread where I've taken one of your complaints and instead of just decreeing new rules I even open things up for discussion and stated "we should come to a collective decision"?

Really?

@matkoniecz
Copy link
Contributor

"private" ground

It is enough to check what is the proportion of proposed pull requests to rejected to see that it is really far away from truth https://github.com/gravitystorm/openstreetmap-carto/pulls?q=is%3Apr+is%3Aclosed And note that many marked as rejected by Github were merged manually or included in other PR.

@dieterdreist
Copy link

sent from a phone

Am 07.07.2015 um 12:43 schrieb mboeringa notifications@github.com:

One thing that is really missing in this discussion is the option of the real time creation of "toggable" on/off thematic overlay layers of certain types of objects on the OpenStreetMap Rails web application via Overpass turbo API calls instead of trying to pre-render everything in a style.

this is really a different issue, useful for searching stuff, while prerendering them is the basis for exploring the area (aka see "what it is like")

@kocio-pl
Copy link
Collaborator

kocio-pl commented Jul 7, 2015

This is quite insulting, but I'm not sure if you are aware of how much so? I mean, I find this DEEPLY insulting.

I am surprised... and I'm sorry. I still don't feel there was any rudeness involved, because I don't even see it as a personal problem and I don't look for anybody to blame for it.

I already remember one time you felt urged by me and angry because of this, while I had no slightest intention to make anybody feel bad - I was just trying to get some decision because nobody (including you) told anything. How rude is it to ask anybody to just tell anything and discuss things? I didn't expect anyone will "do the work as I said" - "I have no time" or "I decided that I won't fix it" are also perfectly valid answers one can discuss further, if needed. But no answer at all in community project - how do you think a newbie like me feels trying to get to know anything at all? I can make my homework - it's not a shame when you have to learn - but no documentation, no rules and no answers makes learning hard experience, requiring a lot of patience, guessing and being very persistent. I don't like it this way - I prefer open disagreement over being quiet "just in case".

This time is no different: for the start, just to be VERY clear - I respect you personally, your work and your point of view, I just disagree with the degree of being conservative regarding to rendering. I may be totally wrong in how I see your attitude toward leadership - you may love this, I don't know! - but for me it doesn't really look like this. If I put the words between quotation marks and it's not quotation, it means "as if" - I just try to show the problem by comparing, not to judge you or anybody else! I even wrote exactly what I mean by this: (designed mostly by few people with strong opinion how should it look like).

If you don't like the word and find it insulting, I can drop it and describe the problem I see in different words. I think the leadership of community project is HARD - it's "more like herding cats" (as Linus Torvalds said it once) and we all know how obedient cats are... I didn't claim you've done nothing important for this style and project - simply because I don't think this is the case at all! I just want to say the formal things (like the open license and so on) are not everything for a community project to be healthy. The feeling you're welcome and can speak in the open way and being taken seriously is also important part of it. Lack of some important answers (you never answered why you generally don't like more icons, even if it was not just me who asked - at least I'm not aware of it) is not nice, nor constructive. Even in this thread @math1985 asked you personally a few questions, which are important to resolve some things - but unfortunately you omitted them again and instead answered me...

It may be hard to measure the effect of collaborative atmosphere, but let's look how many people are involved in mapping, how many discussion you can find on mailing lists about crafting the right tagging, and how many people discuss the rendering here. Of course, there are more technical difficulties when dealing directly with the code, but I believe if everybody get the message "you can also learn to code or even just discuss the rendering", there would be much more people involved, because many people care for rendering even more than for tagging. Yet they tend to tag for renderer rather than complain here. For me it's at least telling us something. And THIS shows the core of the problem for me - do you see it also?

I'm very happy you started this thread/issue and appreciate it a lot. I also try my best to not make things personal when the problem lies somewhere else IMO - and even this is not enough sometimes, because I clearly failed at showing it the way I really see it, so sorry once again, Andy. Please, try to answer more questions and show us more of the reasons you believe are true, because I see no other way to resolve general problems we have.

@kocio-pl
Copy link
Collaborator

kocio-pl commented Jul 7, 2015

@imagico

And yes, there are areas where z=19 is overcrowded with POI icons or at least would be if those were fully mapped in these areas:

http://www.openstreetmap.org/#map=19/38.71272/-9.13924
http://www.openstreetmap.org/#map=19/1.29943/103.85460

Most of big cities in the world will have a lot of shops, but now there are only some places well micromapped and I consider the issue the problem of tomorrow. And even then they will be crowded, but still well readable (so not "overcrowded" in my view) - you have showed probably the highest physically possible density of shops in the city (excluding some marketplaces, I guess), so it won't be worse than that at the highest zoom levels.

There are some things unreadable and overcrowded in the middle scale, however. If we should start with cleaning this style, I would start here for sure. The main roads should be much better soon, thanks to the work of @matkoniecz, but the only other things you can recognize there, are the sea/ocean, airports and city names. It's just much worse than anything related to POIs I ever saw!

But if we remove anything mainly/just to make the most dense places look better, we will hide some things in many more less dense places. The former ones are just a small percent of the latter ones, so effectively we will loose more than we will gain this way.

@daganzdaanda
Copy link

@kocio-pl , it sounds as if you would like more leadership, i.e. more rules or decisions from Andy? Or do you just want to hear his opinion more?
I think this repo works quite well at the moment, especially since more people have stepped forward as maintainers to share the burden and the workload. Andy is a "leader", but not the only one here, and he's certainly not imposing his will above everybody else.
Of course there are some issues that have been open long. But I think that's mostly because everybody here is doing this on their free time, and can't look at all the issues at once. I don't think any issue will be forgotten.

@daganzdaanda
Copy link

Back on topic:
I agree with most of what @imagico wrote.
Something like his checklist could be used to remind people of what a good addition should bring to the style."Scoring" low on that checklist should be the main reason to say no to a new addition.
If something has been proven to work in any other multi-purpose style, that should be a plus point, but not a reason to automatically copy the feature into carto.
Also, he is right about reminding us to review changes after roll-out. But if something throws up new problems, we usually catch them on the second try.

About icons:
I don't see many different icons as a problem. It's definitely better to have the icons than just a dot or the housenumber as before. In very dense situations I'd like to see a z20 someday, or density-based rules. But until then, the one thing that bugs me more is that very high dx between generic dot and Label. I'll open an issue about that.

@gravitystorm
Copy link
Owner Author

this is not a project started by Andy AFAIK, but the continuation of the OpenStreetMap main map (mapnik, the former "Chilton"-style), the only one published on osm.org that is generated with OSM(F) resources.

This isn't quite accurate - I know it's nitpicking, but it's important to get right. This is a fork of the "Chilton" style and obviously shows that great cartographic heritage. But I chose to use a different technology, different code hosting, a different way of working (via pull requests) and managing things (i.e. a team of maintainers), and I made new goals for this project, for example reusability of the stylesheets in other forks. This project was running for many months independently, before it also became the default cartography on osm.org

The OSMF is free to choose whatever stylesheets it likes to run on its hardware. It currently runs one style, and it currently runs this one. But at some point there will be a better option, and things will move on again, and a different style will be the default. Compare with the default online editor, for example - there have now been at least four. For the map styles, there have been at least three, if you count white-lines-on-landsat.

I feel that being the default stylesheet does come with responsibilities towards the wider OSM community, and I certainly bear those in mind (e.g. mapper feedback loop), but I wouldn't go as far as to say we're obliged to do things that we disagree with, even if they are popular with non-contributors. For this project, the final decisions are with the maintainers.

Of course the lines between "this project" and the wider community are deliberately blurred - just see my presentations, the workshops that we've run, the ability for anyone to comment or to make PRs! Even the choice of maintainers is open and flexible.

@gravitystorm
Copy link
Owner Author

  1. displaying too many different features
  2. using too many different symbols
  3. displaying those symbols on too early zoom levels

@gravitystorm Which of these do you think are problems with our style?

  1. is the biggest problem for me. A map becomes incomprehensible when it is too complicated or requires too much learning to make use of. Even I struggle to recognise features from our map due to the complexity of what we try to show - http://www.openstreetmap.org/#map=18/51.50275/0.00323 showing a wide grey line with white dashes confused me for a while this morning, and I resorted to using the query tool to figure out what the feature actually was.

    It's also a burden when we're trying to change things. For example, by showing so many different landuse colours, when we want to change the building colours there's a big list of things to consider. It's not the best example, but I want to make sure people are aware that there is a non-zero burden on stylesheet development for every feature that is added to the style. It's not like we merge the PR and then there's no other consequences.

  2. Is the second biggest problem. Having so many different icons makes it hard to read the map, since in crowded areas they all need viewing to figure out what they are and it's a huge list of things to (subconsciously) re-recognise when you pan the map around. Having a mixture of a few icons and generic symbols was my preferred approach, but I realise that most people preferred lots of icons so we've gone with that approach. It doesn't mean that I like it though :-)

  3. is not a big problem, since it's easily solved. So this is why I'm focussed on the first two issues.

@gravitystorm
Copy link
Owner Author

Andy is a "leader", but not the only one here, and he's certainly not imposing his will above everybody else.

I've been trying really hard not to impose my will here over the last few years. I know that I could, since making all the decisions myself works for my other map styles! But a big part of what is important to me is to make this a collaborative project rather than just "Andy's style". So in a lot of discussions I take a back seat, or keep out of it entirely, or wait for a while to see what other people are thinking before I give my own opinions.

i.e. more rules or decisions from Andy? Or do you just want to hear his opinion more?

Good questions.

@matkoniecz
Copy link
Contributor

by showing so many different landuse colours, when we want to change the building colours there's a big list of things to consider

I may confirm that it is a problem. For example it is surprisingly hard to find road colours that will ensure that (a) road is clearly visible across all possible backgrounds (b) rendering is not terrible.

@matthijsmelissen
Copy link
Collaborator

Some analysis, but not particularly related to this repository:
www.openstreetmap.org/user/Math1985/diary/39404

@matthijsmelissen
Copy link
Collaborator

Since usage has almost doubled over a short period I suspect something else beyond normal additions by mappers.

Somebody did a series of mechanical edits, retagging thousands of instances of mountain_pass into saddle.
http://www.openstreetmap.org/changeset/28122761

@Ircama
Copy link
Contributor

Ircama commented Sep 1, 2016

Somebody did a series of mechanical edits, retagging thousands of instances of mountain_pass into saddle.
http://www.openstreetmap.org/changeset/28122761

Interesting analysis.
By curiosity I quickly checked some nodes randomly and almost all of them (not all anyway) do not cross a road and are expected to be saddles.

@Ircama
Copy link
Contributor

Ircama commented Sep 2, 2016

I see I can run it online on http://taghistory.raifer.tech/

Just FYI, it broadly supports browsers (e.g., Firefox, Chrome, ...) but does not look to run on IE11.

@kocio-pl
Copy link
Collaborator

Another quite interesting analytic tool from the same person - OSM tile access logs viewer (so we know which areas are currently the most interesting for viewers):

http://osm-tile-access-log-viewer.raifer.tech
https://www.openstreetmap.org/user/tyr_asd/diary/39434

@tyrasd
Copy link
Contributor

tyrasd commented Oct 18, 2016

[… http://taghistory.raifer.tech/ …] Does it take into account deleted objects?

Yes, it does.

@kocio-pl
Copy link
Collaborator

We ultimateley failed to find common rules, therefore I close it now. If any new ideas would appear, it can be reopened, of course.

@imagico
Copy link
Collaborator

imagico commented Apr 24, 2018

I made an update on the stats i did in #1630 (comment) - for all PRs merged since the beginning of 2017:

category count
a1 (fully new features) 45
a2 (new differentiation of things previously rendered identically) 10
az (additional zoom levels) 9
r1 (fully removed features) 3
r2 (unification of features previously rendered identically) 2
rz (less zoom levels) 12
n (neutral regarding addition but visible) 87
invisible 73

Interesting is also the changing rate of modifications in the different categories - about 25 of the 45 fully new features (more than half) were added since the beginning of the year while only about a dozen of the 87 neutral visible changes (which is mostly bug fixes, styling adjustments etc.) is from 2018.

By the way the three PRs from the r1 category were #3174, #2573 and #2554.

@kocio-pl
Copy link
Collaborator

Thanks, that's interesting and quite detailed (241 cases analyzed). It would be interesting to know how the open issue tickets can be categorized, even if they're not as clear, because multiple solutions might be used. Simple search shows 48 (of 358) tickets with the word "add" in title.

@kocio-pl
Copy link
Collaborator

kocio-pl commented Apr 28, 2018

I see from #3201 (comment) that in your view this analysis proves something special and disturbing ("lacking responsibility", I guess), but I think it's quite simple: there are more than one "shift" during these 5-6 years and it's natural thing for a living project to evolve.

When Andy started style reimplementation, it was one man show focused on strict matching old features 1:1. Then it has grown to a small team of coding experts expanding it by mutual consensus (you call it a "bubble" when you don't like the output) - which was first shift from the original goal.

Then it became rather tight - second shift, where the team members were still very productive, but were both experienced and afraid of changes (described quite good in #1630 (comment)). Which was frustrating for some people like me.

Then some other shift has started slowly - it was harder and harder to make consensus and creator became less active. Next thing was making team bigger, but it didn't help with very different visions. Experienced team members become less active too, so it was harder to see any move, even if it could be quite easy to decide some things. I tried to gain some experience and deploy my visions to avoid project halt.

Then a new trend has begun and this is what you have observed - people from outside of the team started to design, discuss, code and decide things more and more. This is the onboarding success, but it's quite natural that they come with adding things, because fixing and groundwork are harder. This also explains acceleration - there are more people active than before, so they made more changes (some of the changes are just waiting for implementation).

I hope vector fork(s) will make the rendering fun again for both experienced and new developers. That will be next shift from the current state - and this might go in multiple directions in parallel.

@imagico
Copy link
Collaborator

imagico commented Apr 28, 2018

For better understanding: My analysis in #1630 (comment) was specifically meant to be based on objectively verifiable categories. I thought about also classifying PRs by other criteria, for example in how far they work towards or against the goals of this style, but such more specific analysis would require more work in defining and documenting criteria to be scientifically sound. Maybe some other time.

I am completely fine with and frankly i expect these observations to be used to support different views of the development of this style. Which of these views actually best represents reality is something you can ultimately only determine by critically evaluating its validity against this reality. This is hard and i therefore rarely observe it to happen and the lack of solid literature on OSM cartography and how it relates to cartography in general does not make it easier.

I explained this more elaborately in private conversation among the maintainers recently - which is where i mentioned the 'bubble' @kocio-pl is referring to. Since that is difficult to understand for those who have not been in this conversation here a quote of the part in question:

If you look over recent changes by maintainers merged or approved by
other maintainers you will also find in many cases that no substantial
discussion is taking place (i.e. discussions where pros and cons of a
change are analyzed, alternatives and context considered and it is
attempted to look at all relevant aspects of a problem). Like here:

#3107
#3144
#2972
#3065
#3103

To me as a relative outsider now since i no more actively participate in
development at the moment this more and more feels like a filter bubble
with developers re-affirming each others views rather that critically
reviewing things. Occasionally i make a comment trying to broaden the
perspective on the matter in question (like in the last example above)
but usually it feels like i am mostly inconveniencing people by
bringing in complications that originally did not seem to exist inside
the filter bubble. Only in rare situations where a topic is carried
outside the realm of osm-carto (like with the boundary relations/lines
recently or
#2816) we seem
to get a fairly exhaustive discussion of a matter that has the
potential to significantly widen the horizon of those participating.

@kocio-pl - you are speaking of your visions - a year ago in #2462 (comment) i called for you and others to write down the design principles that guide you. Nothing happened since then. Maybe that is something you want to look into. Note this is not something you should try to do in five minutes, i would recommend at least a few hours time for that. Trying to formulate your intentions in a consistent form that is comprehensible by others can be an eye opening experience.

@kocio-pl
Copy link
Collaborator

kocio-pl commented Apr 28, 2018

  1. I like the numbers - if they don't help me, they are at least inspiring and interesting, so I'm happy with your work. Sometimes I have no idea what they mean (like some tag history charts showing that in some cases lack of rendering or adding new features have no effect on tagging). In this case however nothing surprised me - not the number of "a" (adds), nor the number of them in this year. And I don't see anything wrong with it - more fixing (and probably removing) will be possible when we will have more regular contributors. As much as I feel adding things is perfectly right, I care also for other actions and try to do them too ([WIP] Rewrite roads layers #2869 is out of my reach, but would be great).

  2. As of the meaning of "bubble"/consensus, it's probably not good to go to the extremes. Too much consensus/single man work and you end up with biased choices, too few of it and any decision making is impossible. I like team work, when people have at least common goal, but can change details and share some work. Therefore I'm more happy with Adding different types of towers and masts #2671 then with some my own changes which had not a single comment, even if I was asking.

  3. My intentions are quite simple probably:

  • I want to have a rich map in the first place (hence adding features when they don't overlap too bad),
  • I want to make it clear and readable (hence moving some smaller elements later to not obfuscate the area or support for fading area colors on midzoom),
  • I like it to be diverse (hence merging the marketplace, as it can be quite popular outside Europe, but also adding outdoor symbols or advertising column)

As you can see, all these are the points from our goals. But I'm not willing to invest more time into writing them down, because we differ in how we understand them, so there's no use in trying to be more precise:

  • I see the goal of a rich map and feel safe with adding, since we're unable to add even most of all the keys, but team mates seem to not follow this goal and read the last sentence as warning, while for me it's more like a safe reminder for people unaware that it's impossible to have everything (impossible, not "forbidden"!).
  • I see the fading colors as a tool to make map more readable, while you for example see the reverse effect of exactly the same action and reading the same rule.
  • I see diversity not only for different countries (how this is always mentioned), but also urban/outdoor, big/small elements and so on
  • Encouraging correct mapping and adaptability don't create tension, IIRC, so we probably don't differ that much in that respect. Maintainability is not a direct source of conflicts too, but it's probably a part that comes to mind when opposing adding more shop/amenity code, while I see the road code as much worse problem.

How could we solve this by stating things more clearly? I think we just can't.

I think community building around osm-carto, but also around OSM map rendering, is very important, but that's not a style goal, rather my personal goal. This is important part of my vision anyway.

I doubt stating all that helps anything. Instead of trying to better share the box we're in (creating better rules), I prefer to make more space (I mean more styles and personalization).

@dieterdreist
Copy link

dieterdreist commented Apr 28, 2018 via email

@kocio-pl
Copy link
Collaborator

Yes, that won't solve all the rendering problems. It's just a real step toward overcoming them.

I think the sheer amount of OSM data is the biggest problem now, because the entry barrier for using them is so high. I think it's not a coincidence that even communities render only their own country with their style/fork - see the OSM Swiss for example. Only the strongest ones, as German or French are showing all the world.

It's even worse when it comes to the personal maps. uMap is just scratching the surface. Klokan pre-rendered vector tiles in the Docker containers are rather a technology preview. I think we need kind of community Mapbox Studio service:

  • full database - no need to manage the data, like importing the planet.osm for days or keeping them fresh
  • online presence - not just on a home machine
  • user friendly (visual) style editor

With v4.x we have an hstore and OSMF keeps database fresh, so we have it, the online presence is also there. But we have just one style rendered there, so making it possible to have few more, even limited by osm-carto queries, is just better.

@dieterdreist
Copy link

dieterdreist commented Apr 28, 2018 via email

@kocio-pl
Copy link
Collaborator

if you just want to create some maps you usually don’t need the whole world, or if you do you don’t need a lot of features (at world scale), so the amount of data is not really an issue if you know what you want to do.

I don't think it's so easy. Firstly because we have exactly this problem with rendering area:highway=* in Polish community - we would love to cover the whole world to spread this tagging scheme, but it needs better hardware than what we have:

http://osmapa.pl/w/area/

There's also an important question - why would you like to limit the map if you could show the names of all the places with your language and with your style (for example showing "Pekin" label instead of "北京")? And why the OSMF doesn't render multiple styles with many languages (Wikimedia plans to cover all the languages, but it's wealthy organization nowadays), just one style with default names?

"If you know what you're doing" you can use a big machine (24-32 GB rather then 4-8 GB of RAM for example) and import the planet in ~24h for a start (!), but if you'd like to just play with maps, there are too many things you lack. I will try to learn people using QGIS, but even importing only Poland will take ~1,5 h, so it's not casual activity.

If you still want the whole world at small scales you either have to have the resources or be patient, you still can import everything on cheap consumer hardware if you bring a week or two, SSDs have become significantly cheaper in the last years ;-)

If you're determined and patient you can do amazing things! But that's why I say about high entry barrier - it's perfectly possible in many cases, but it's just very hard.

@dieterdreist
Copy link

dieterdreist commented Apr 29, 2018 via email

@kocio-pl
Copy link
Collaborator

kocio-pl commented Apr 29, 2018

maybe they are only waiting for someone to propose it again who has a good plan how to implement it.

Yes, of course:

https://wiki.openstreetmap.org/wiki/Top_Ten_Tasks#Localized_map_rendering

But "good" means "we're not able to render raster tiles for each language with our hardware using current toolset":

However, this needs to be made possible without taking too many resources.

@jeisenbe
Copy link
Collaborator

Reopening: while there was not consensus about this back in 2015 and 2016, it would still be good to document our process for deciding when accept or reject a new feature request.

@jeisenbe jeisenbe reopened this Mar 23, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests