Update for lookups.dat

[afischerdev]

Initial with #414, we like to update the lookups.dat and start the collection

Add 'cycleway:both' (#414)

cycleway:both;0000000001 lane
cycleway:both;0000000001 shared_lane
cycleway:both;0000000001 no none
cycleway:both;0000000001 track
cycleway:both;0000000001 separate
cycleway:both;0000000001 share_busway
cycleway:both;0000000001 shared
cycleway:both;0000000001 yes

Add 'cycleway:lane' (#398)
see wiki https://wiki.openstreetmap.org/wiki/Key%3Acycleway%3Alane

cycleway:lane;0000000001 advisory
cycleway:lane;0000000001 exclusive
cycleway:lane;0000000001 pictogram

cycleway:both:lane;0000000001 advisory
cycleway:both:lane;0000000001 exclusive
cycleway:both:lane;0000000001 pictogram

cycleway:right:lane;0000000001 advisory
cycleway:right:lane;0000000001 exclusive
cycleway:right:lane;0000000001 pictogram

cycleway:left:lane;0000000001 advisory
cycleway:left:lane;0000000001 exclusive
cycleway:left:lane;0000000001 pictogram

Change 'highway=no' (#402)

highway;0000002942 abandoned disused razed demolished dismantled no

Change 'traffic_calming' removed * (1x)

traffic_calming;0000006808 yes *

[afischerdev]

Add 'unhewn_cobblestone' (#270)

surface;0000113800 cobblestone cobblestone:flattened unhewn_cobblestone
cycleway:surface;0000000011 cobblestone cobblestone:flattened unhewn_cobblestone

[EssBee59]

Great work!

As explained in #414, I think this update has for me first priority (number of occurences) and should be implemented.

On the other side, it is probably not possible to support/implement all the requests related to lookup/ new OSM tags:

I think many impacts should be considered: as example the workload, the impact on the size of the rd5 files, the procesing time of a script where the number of checked tags grows and grows

By tags with low/middle occurences (my old requests #241 and #276) I am today not sure, it was a good idea to open them
(of course they remain interesting for my own usage)

[polyscias]

Nice start indeed!

traffic_sign:direction;0000000001 forward
traffic_sign:direction;0000000001 backward
traffic_sign:direction;0000000001 both

traffic_sign:direction is the same as direction=forward/backward/both, both but more specific. Ideally it could be combined with that but AFAIK the current parser does not support that.

cyclestreet;0000000001 yes designated

cyclestreet is the "Belgium/Dutch version" of bicycle_road. Could again be combined if the parser would support that

agricultural;0000000001 no
agricultural;0000000001 yes designated official
agricultural;0000000001 private permissive
agricultural;0000000001 destination delivery

See #104 and agricultural

I think size is important but see #393, it seems like it is only about 20% of rd5-size.

Still I would think it would be good to remove low-used tags that are more-or-less an error, it is not only size but also processing overhead.

Examples:

• highway=mini_roundabout/turning_loop in way context
• highway=culvert/tertiary/proposed/footway/path/traffic_calming//door in node context.

Also dropping for example "living_street;0000000404 yes", "informal;0000002424 yes" etc. would make sense to me. And what about noexit, is that ever going to be used in routing context?

[EssBee59]

I think size is important but see #393, it seems like it is only about 20% of rd5-size.

Thank for this important information!
If so, adding new tags in the lookup is not very expensive, and only the scripts that check the new tags will get more processing time...
(but the processors become faster and faster every year)

[polyscias]

See #300, add support for just marking (like a boolean) if there is a *:conditional key with arbitrary value.

Following the use as documented in #300, what about:

access:conditional;000000001 *
motor_vehicle;conditional:000000001 *
hgv:conditional;000000001 *
foot:conditional;000000001 *
bicycle:conditional;000000001 *
motorcycle:conditional;000000001 *
motorcar:conditional;000000001 *

Both in way and node context.

Is support for '*' already present?

[afischerdev]

it would be good to remove low-used tags that are more-or-less an error

@polyscias
I agree, a rule e.g. 'no tags under 100 uses' could be helpful, but how to detect if this is an error?
For this I found
mini_roundabout = 11 - removed
turning_loop = 27 - removed

culvert = 0 - removed
tertiary = 146
proposed = 1107
footway = 1627
path = 617
traffic_calming = 0 - removed
door = 0 - removed

living_street, informal, noexit

I'm not sure about this, 'living_street' is grown up to 277.605 uses, but 'highway=living_street' should prefered. Also 'informal' has 103 522 entries. Seems to be pupular and 'living_street' is used in several profiles. 'noexit' - we already have the way end here, but could this be useful to determine a correct ending way and not an incorrect/unconnected way?

access:conditional and more, Is support for '*' already present?

That's is heavy stuff.
'*' is supported only for numerical values like 'maxheight' and it is not used in the lookups.dat at the moment. But it is used with current libs in a waterway context for bridge height, width and more - please see brouter.grade.de.
This has to be reworked, may be with the short variante: save only the way osm id to show the user there is a condition and you will find here.

[EssBee59]

no tags under 100 uses' could be helpful, but how to detect if this is an error?

Yes, it is difficult to decide which tags should be added or deleted:
As example #276 (access=service)
Occurence is in whole Germany only 46, but, each lost tag leads to a wrong route (for me access=service is to be handle as access=no)

Not only the occurences should be considered, the impact is also important.
BTW, can we afford to add this tag?
(please consider the remark of Poutnikl)

Some tags are only for "tuning" of the route and have a minor impact on the routing. (example #241 where I am not sure to suggest to add the tags I proposed 2 years ago)

[afischerdev]

can we afford to add this tag

Sure, with the current changes we have less data.

We always have the two choices:

# "Allowed if not forbidden"
if access=no|other_no|yet_other_no>    
     then    <value_for_no_access>
     else    <value_for_granted_access>

# and the other way round
if access=yes|permissive|designated|destination    
     then   <value_for_granted_access> 
     else    <value_for_no_access>

In second case 'access=service' will be negative

[EssBee59]

Thank for adding this tag (acces=service)!
-when the change is activated, I intend to use this logic in my fastbike profile:
assign defaultaccess switch access=private|no|service
0
1

About #241 now:
I suggested 2 years ago these new tags:
cycleway:smoothness
cycleway:right:surface
cycleway:left:surface
cycleway:right:smoothness
cycleway:left:smoothness

But between 5 and 10 values are for each line possible...
I think today, it is too much work and overhead for the intended fine tuning!
If you agree, please close #241

Regards
EssBee

[polyscias]

I'm not sure about this, 'living_street' is grown up to 277.605 uses, but 'highway=living_street' should prefered. Also 'informal' has 103 522 entries.

Yes, living_street did grow substantially, did not realize that, same for informal, so, at least these usage number should be updated ;-)

I made myself a python script that parses the lookups.dat file, get's the numbers in way and node context from taginfo and writes out lookups_new.dat that has the same key:tags as the original file but with a comment added on the actual use. It also checks if a key:tag in way context is also present in node context and vice-versa. If so, it calculates the usage ratio between these two and if it is smaller than 0.01 is add a note to the comment about that. It also spits out a lookups_update.log file that has all tag:keys covered with a usage >= 5.

Doing so I see there are quite some key:tags that are (almost) complete unused:

---context:way
highway;0000000679 unsurfaced                                                  # 0000000000
highway;0000000108 byway                                                       # 0000000000
motorcar;0000000100 psv                                                        # 0000000000
motorcar;0000000100 hov                                                        # 0000000000
motor_vehicle;0000000100 hov                                                   # 0000000000
motorcycle;0000000100 psv                                                      # 0000000000
motorcycle;0000000100 hov                                                      # 0000000000
vehicle;0000000100 hov                                                         # 0000000000
segregated;0000000000 unknown                                                  # 0000000003
sac_scale;0000000265 T1-hiking                                                 # 0000000000
noexit;0000000000 unknown                                                      # 0000000002
oneway;0000000481 true                                                         # 0000000000
waterway;0000000321 tidal_flat_slough                                          # 0000000000
construction;0000002093 crossing                                               # 0000000001
construction;0000001440 bus_stop                                               # 0000000002
construction;0000000679 unsurfaced                                             # 0000000001
construction;0000000108 byway                                                  # 0000000001
construction;0000000021 mini_roundabout                                        # 0000000000
construction;0000000020 turning_loop                                           # 0000000000

 ---context:node
highway;0000001324 street_light                                                # 0000000001
highway;0000001147 incline_steep                                               # 0000000000
highway;0000001101 stile                                                       # 0000000000
highway;0000000904 incline                                                     # 0000000000
highway;0000000817 traffic_calming                                             # 0000000000
highway;0000000436 no                                                          # 0000000001
highway;0000000353 door                                                        # 0000000000
highway;0000000283 level_crossing                                              # 0000000000
highway;0000000267 yes                                                         # 0000000001
highway;0000000197 junction                                                    # 0000000006
highway;0000000176 distance_marker                                             # 0000000000
highway;0000000158 noexit                                                      # 0000000000
highway;0000000113 priority_to_right                                           # 0000000000
highway;0000000113 culvert                                                     # 0000000000
highway;0000000046 toll_bridge                                                 # 0000000000
highway;0000000037 city_entry                                                  # 0000000000
barrier;0000000069 traffic_crossing_pole                                       # 0000000004
horse;0000000009 critical                                                      # 0000000007
wheelchair;0000000005 permissive                                               # 0000000007
entrance;0000000439 secondary_entrance                                         # 0000000000
entrance;0000000258 private                                                    # 0000000009
traffic_calming;0000000847 speed_bump                                          # 0000000001

I see also the route* keys having zero way usage, for example:

route_bicycle_icn;0000088753 yes                                                         # 0000000000
route_bicycle_icn;0000000001 proposed                                                    # 0000000000
route_bicycle_icn;0000000000 unknown                                                     # 0000000000
route_bicycle_ncn;0000268180 yes                                                         # 0000000000
route_bicycle_ncn;0000000001 proposed                                                    # 0000000000
route_bicycle_ncn;0000000000 unknown                                                     # 0000000000

But I assume that these key:tags are using for route relations...

I agree, a rule e.g. 'no tags under 100 uses' could be helpful, but how to detect if this is an error?

For key:tags in both node and way context I think it is better to look at the ratio.

highway;0000002093 crossing                # 0000000364 - 6223265 nodes (0.006%)
highway;0000001440 bus_stop                # 0000009601 - 3045903 nodes (0.315%)
highway;0000000021 mini_roundabout         # 0000000011 - 55676 nodes (0.020%)
highway;0000000020 turning_loop            # 0000000027 - 149562 nodes (0.018%)

---context:way
highway;0000002540 steps                   # 0000003207 - 1336795 ways (0.240%)
highway;0000001372 residential             # 0000000472 - 61624903 ways (0.001%)
highway;0000000819 service                 # 0000000219 - 47450022 ways (0.000%)
highway;0000000662 path                    # 0000000617 - 11538741 ways (0.005%)
highway;0000000603 footway                 # 0000001627 - 16692720 ways (0.010%)
highway;0000000438 track                   # 0000002139 - 22818351 ways (0.009%)
highway;0000000262 road                    # 0000000077 - 102055 ways (0.075%)
highway;0000000244 construction            # 0000001750 - 282115 ways (0.620%)
highway;0000000214 unclassified            # 0000000249 - 15830877 ways (0.002%)
highway;0000000213 proposed                # 0000001109 - 114622 ways (0.968%)
highway;0000000123 tertiary                # 0000000146 - 7089572 ways (0.002%)

I think everything in the order of 0.0xx% are tagging mistakes and can be removed for sure.

I also had a look on what is missing and I see:

---context:
highway=busway -- 5593
highway=elevator -- 3612
highway=via_ferrata -- 2476

surface=artificial_turf -- 21554
surface=tartan -- 19055
surface=rock -- 8307
surface=stone -- 8251
surface=soil -- 6969

maxspeed=30 mph -- 588434
maxspeed=25 mph -- 456321
maxspeed=20 mph -- 281531
maxspeed=45 mph -- 274791
maxspeed=35 mph -- 225205
maxspeed=40 mph -- 212141
maxspeed=60 mph -- 149112
maxspeed=55 mph -- 148245
maxspeed=50 mph -- 113565
maxspeed=65 mph -- 113513
maxspeed=70 mph -- 98454
maxspeed=15 mph -- 55622

service=yard -- 504673
service=spur -- 468875
service=siding -- 304639
service=crossover -- 75429
service=irrigation -- 15286
service=utility -- 14672
service=driveway2 -- 7558
service=slipway -- 6046

lit=sunset-sunrise -- 54802
lit=automatic -- 8411
lit=24/7 -- 5340

access=permit -- 41075
access=restricted -- 3890
access=military -- 3844
access=public -- 3262

foot=customers -- 5653
foot=permit -- 3113

bicycle=customers -- 8009
bicycle=permit -- 1610
bicycle=delivery -- 107

motorcar=delivery -- 637
motorcar=agricultural;forestry -- 332
motorcar=permit -- 231

motor_vehicle=permit -- 18801
motor_vehicle=customers -- 10007
motor_vehicle=delivery -- 4823
motor_vehicle=agricultural;destination -- 870

motorcycle=delivery -- 302
motorcycle=agricultural;forestry -- 264
motorcycle=permit -- 140

vehicle=delivery -- 4172
vehicle=military -- 855
vehicle=customers -- 823
vehicle=permit -- 483
vehicle=emergency -- 262

oneway=-1 -- 28882

bridge=boardwalk -- 22336
bridge=aqueduct -- 10411
bridge=movable -- 5427

incline=-10% -- 4650
incline=-15% -- 1843
incline=-5% -- 1652
incline=-20% -- 1551
incline=-8% -- 1173
incline=-6% -- 888
incline=-12% -- 874
incline=-4% -- 853
incline=-7% -- 763
incline=-25% -- 636
incline=-10° -- 619
incline=-30% -- 506

busway=lane -- 4550
busway=yes -- 805
busway=track -- 290
busway:left=lane -- 6174
busway:right=lane -- 8768

cycleway:left:oneway=-1 -- 3028
cycleway:right:oneway=-1 -- 1307

----- context:node
highway=toll_gantry -- 3969

barrier=height_restrictor -- 11114
barrier=sliding_gate -- 9214

access=permit -- 11932
access=delivery -- 5510

motor_vehicle=permit -- 2948
motor_vehicle=customers -- 347
motor_vehicle=delivery -- 299

crossing=marked -- 1319388
crossing=traffic_signals;marked -- 5742

railway=stop -- 96700

waterway=dam -- 48568

entrance=secondary -- 6363

The maxspeed tags have a space and I guess that is not supported.

Not all should be added but adding quite some as aliases is I think a good idea.

[afischerdev]

Great investigation.
If you don't mind, I'll bring that into a new lookups.dat and public it for review in my git space.

The maxspeed tags have a space and I guess that is not supported.

Could be, I thought it is, but I have an eye on it. (EncodeDecodeTest it comes back as unknown)

[EssBee59]

Yes great work!
Just a remark/question:
I do not see a real benefit of tags such as:
cycleway:both;0000000001 no none (with high occurences)
cycleway:both;0000000001 shared
(same by cycleway=no)

Is there a real benefit?
Is there a cost / overhead to implement ?

I found only 1 script with a check on such a tag, it is using this logic ==> or cycleway= cycleway=no|none
As expected, same use as "no "cycleway" tag!

[polyscias]

If you don't mind, I'll bring that into a new lookups.dat and public it for review in my git space.

Yes, that is a good idea. I might have done that myself, but I you can do it, prefect. I propose to keep the usage numbers for it to be the same as is so it is easier to review the changes and only after committing the change do update of the usage numbers, I can provide that using the script I have.

I will still review the list of tags once more for possible tags that an be removed with an usage between 10 and 100.

I would prefer combining things like surface=artificial_turf with surface=earth and surface=ground, that will not only make the list shorter but also the profiles more compact.

On access tags I think yes/designated/official, destination/delivery/customers and private/permit can be combined for routing purposes.

On cycleway*=no none: How the parser works is that all given values in looksups.dat are assign, but if there is a key:tag whereby the key is in looksups.dat but the tag not, it is automatically assigned to the unknown category.

So if you leave out no/none/private these will become part of unknown and the question is if you want that. Having no/none/private separate you can chose to handle them differently in a profile.

[afischerdev]

Here an update on my repository

It doesn't contain conditional tags.
All route_* are removed but I add network=...
Most other suggestions I added/removed. On some I'm not clear, e.g. slipway, dam, is it navigationable?
Or cycleway=no, this is still inside not to bring back 'unknown' value.
But this means profiles will not work, so I add small changes on trekking.brf and shortest.brf for easy testing.

maxspeed=30 mph or similar is escaped for ways. Please see

[polyscias]

Nice piece of work!

I would like to take a bit more time to review things, likely in the weekend.

[EssBee59]

Thank for the explanation about "cycleway=no" and the "unknown" value!

I checked my profile with lookup.dat V11, and had to change:
1-As explained above, "Cycleroute" are not longer supported
2- in the check "oneway=yes|true|1" value true is not longer supported
3-Of course cycleway:both has to be introduced, here my new code:
assign both_cycleway cycleway:both=track|lane|shared_lane|shared_lane|shared|separate|yes

My script is now error free, but to start some tests, I have to wait untill rd5´s for lookup version 11 become available

[afischerdev]

@polyscias
We have all the time we need.

@EssBee59
I've tested with the Hessen pdf. If you like I can send you a download link for the *.rd5 (around 22MB, without elevation).

[EssBee59]

I've tested with the Hessen pdf. If you like I can send you a download link for the *.rd5 (around 22MB, without elevation).

No time to test iIn the next 2 weeks, so I prefer to wait on the rd5´s when avalaible
Regards

[polyscias]

The header says 131 additions and 102 deletions, so more added then deleted ;-)

Let me start with three more requests:

I am running already a long time with a profile that has support for traffic signals, see #183 but analyzing some routes I saw that they were different then expected because of a highway=traffic_signals that is different and for that there is traffics_signals=. I am interested in the difference between a "normal" traffic light and things like blinker/blink_mode/continuous_green and tram_priority/bus_priority

Looking at taginfo and grouping things in 4 groups:

traffic_signals;0000000001 signal traffic_lights pedestrian_crossing yes crossing junction level_crossing crossing_only pedestrian normal signals on_demand crossing_on_demand
traffic_signals;0000000001 blinker emergency no blink_mode continuous_green blink emergency_priority
traffic_signals;0000000001 tram_priority bus_priority stop tram_priority;blinker
traffic_signals;0000000001 ramp_meter

One other key that can be used to detect how popular a cycle route is: monitoring:bicycle. A "boolean" (node context) is enough:

monitoring:bicycle;0000000001 yes induction_loop

For tracktype=4, can we added "grade3-5" as alias, that is mapped 917 times

On the deletion on (almost) unused tags:

I saw the commit also updates trekking.brf as that had else oneway=yes|true|1 in it, that "true" has been deleted because the usage was 0, so good catch!

I am updating my script to check what the impact of the deletion would be on other profiles I know.

I still would like to consider removing highway tags with the wrong context:

---context:way
highway;0000002093 crossing                # 0000000364 - 6223265 nodes (0.006%) <----
highway;0000001440 bus_stop                # 0000009601 - 3045903 nodes (0.315%) -- keep
highway;0000000021 mini_roundabout         # 0000000011 - 55676 nodes (0.020%)   -- done
highway;0000000020 turning_loop            # 0000000027 - 149562 nodes (0.018%)  -- done

---context:node
highway;0000002540 steps                   # 0000003207 - 1336795 ways (0.240%) -- keep
highway;0000001372 residential             # 0000000472 - 61624903 ways (0.001%) <----
highway;0000000819 service                 # 0000000219 - 47450022 ways (0.000%) <----
highway;0000000662 path                    # 0000000617 - 11538741 ways (0.005%) <----
highway;0000000603 footway                 # 0000001627 - 16692720 ways (0.010%) <----
highway;0000000438 track                   # 0000002139 - 22818351 ways (0.009%) <----
highway;0000000262 road                    # 0000000077 - 102055 ways (0.075%) <----
highway;0000000244 construction            # 0000001750 - 282115 ways (0.620%) -- keep
highway;0000000214 unclassified            # 0000000249 - 15830877 ways (0.002%) <----
highway;0000000213 proposed                # 0000001109 - 114622 ways (0.968%) -- keep
highway;0000000123 tertiary                # 0000000146 - 7089572 ways (0.002%) <----

Searching for examples I found josm already warns for them:

The blue way has the same problem as the problem node...

[afischerdev]

All all the other standard profiles.

[polyscias]

Thanks for incorporating what I wrote in my previous post in 17c9873!

I have updated the script to parse the profiles I know, should have done that earlier ;-)
The script now finds which removed tags are used in (other) profiles and the combination of taginfo count with "brf usage" gives better input on which tags can be safely removed.

Most "pain" is with:

Unsupported key route_bicycle_icn : brouter(7), poutnik(94), zossebart(2), Tapio(1), utack(2), dhanek(11), FFMbyBicycle(1), cxberlin(11), ThomasTraber(1)
Unsupported key route_bicycle_ncn : brouter(7), poutnik(94), zossebart(2), Tapio(1), utack(2), dhanek(11), FFMbyBicycle(1), cxberlin(11), ThomasTraber(1)
Unsupported key route_bicycle_rcn : brouter(7), poutnik(94), zossebart(2), Tapio(1), utack(2), dhanek(11), FFMbyBicycle(1), cxberlin(11), ThomasTraber(1)
Unsupported key route_bicycle_lcn : brouter(7), poutnik(94), zossebart(2), Tapio(1), utack(2), dhanek(11), FFMbyBicycle(1), cxberlin(11), ThomasTraber(1)
Unsupported tag oneway=true : brouter(14), poutnik(142), zossebart(2), utack(6), dhanek(13), FFMbyBicycle(2), afischer(1), cxberlin(13), ThomasTraber(1)

i.e. all profiles I know of.

less "pain" with:

Unsupported key route_hiking_iwn : brouter(1), poutnik(23), Tapio(1)
Unsupported key route_hiking_nwn : brouter(1), poutnik(23), Tapio(1)
Unsupported key route_hiking_rwn : brouter(1), poutnik(23), Tapio(1)
Unsupported key route_hiking_lwn : brouter(1), poutnik(23), Tapio(1)
Unsupported key route_hiking_ : brouter(1), poutnik(23), Tapio(1)
Unsupported key route_foot_ : brouter(1), poutnik(23), Tapio(1)
Unsupported key route_foot_nwn : brouter(1), poutnik(23), Tapio(1)
Unsupported key route_foot_rwn : brouter(1), poutnik(23), Tapio(1)
Unsupported key route_foot_lwn : brouter(1), poutnik(23), Tapio(1)
Unsupported tag sac_scale=T1-hiking : brouter(1), poutnik(23), Tapio(1)

So the the "network update", oneway=true (zero usage) and sac_scale=T1-hiking (zero usage)

On the "network update", with everything mapped to network=[inrl][cw]n:

• Is it (still) possible that a way is tagged as both part of a foot and cycle network?
• Should be mapcreator/RelationMerger.java not also be updated? (Or am I missing something?)
• Or is it better to revert this part of the update?

Most other warnings I see for water related tag for @afischerdev (understood ;-) but also:

Unsupported tag highway=toll_bridge : poutnik(68)
Unsupported key route_bicycle_ : utack(1)
Unsupported key route_bicycle_radweit : utack(1)
Unsupported tag motorcar=hov : utack(1)
Unsupported tag motorcar=psv : utack(1)
Unsupported tag highway=driveway : utack(6)
Unsupported tag highway=mini_roundabout : utack(7)
Unsupported tag highway=turning_loop : utack(6)
Unsupported tag highway=unsurfaced : utack(8)
Unsupported tag highway=crossing : utack(5)
Unsupported tag highway=byway : utack(5)
Unsupported tag highway=traffic_calming : utack(1)
Unsupported tag highway=city_entry : ThomasTraber(1)

So profiles of @poutnikl, @utack and @ThomasTraber

On further cleaning up: Instead of describing what I think can be done I took the last version of lookups.dat and made the changes in it:

• Make infrequent tags that are not used in profile an alias of another appropriate tag
• Add aliases for tags that were not part of lookups.dat yet but have quite some usage
• Remove some tags that have low usage and are not used in profiles (quite some surface=* tags, low usage and surface= is not used by any profile)
• Add a tag for many (>8) lanes

The updated looksup.dat: https://gist.github.com/polyscias/358055cdad801b7b62b248310e09fddd
The annotated version: https://gist.github.com/polyscias/b5118a56246a00bbd54ac3b9d67e7a48

[afischerdev]

Should be mapcreator/RelationMerger.java not also be updated?

Great, this was the needed pointer, my fail, I hadn't expected hardcoded tags here.
I will rewind the network logic. It is good as it is.

So we have only the smaller changes for 'oneway=true (zero usage) and sac_scale=T1-hiking (zero usage)' and more.
I'll go on then with the point 'On further cleaning up'.

[afischerdev]

Made the rewind and started with more cleaning:

Make infrequent tags that are not used in profile an alias of another appropriate tag
Add aliases for tags that were not part of lookups.dat yet but have quite some usage
Add a tag for many (>8) lanes

[polyscias]

I took an older version of netherlands.osm.pbf I had still laying around and did run mapcreation on it without and with the latest changes the documented in this issue

Then I took from the log of both runs the codec stats for E5_N50

stat	org count	org kbits	upd count	upd kbits	diff count	diff kbits	diff count	diff bits
externalNode	203006	6087	189222	5621	-13784	-465	-7,3%	-8,3%
extradata	1762640	1721	1719215	1679	-43425	-42	-2,5%	-2,5%
link-counts	1762640	5073	1719215	4952	-43425	-121	-2,5%	-2,4%
netdatasize	4467	67	4219	64	-248	-3	-5,9%	-4,7%
node-positions	4467	35429	4219	34612	-248	-817	-5,9%	-2,4%
nodeIdx	2298004	12863	2251595	12497	-46409	-366	-2,1%	-2,9%
nodeTagDictionary	4467	428	4219	392	-248	-36	-5,9%	-9,1%
nodeTagIdx	1762640	2073	1719215	1965	-43425	-108	-2,5%	-5,5%
nodecount	4467	47	4219	45	-248	-2	-5,9%	-3,7%
nodeele	1762640	7251	1719215	6970	-43425	-281	-2,5%	-4,0%
noisebits	4467	100	4219	94	-248	-6	-5,9%	-6,2%
transcount	2399507	5218	2346206	5096	-53301	-123	-2,3%	-2,4%
transele	3126548	10244	2999248	9753	-127300	-491	-4,2%	-5,0%
transpos	3126548	60215	2999248	57545	-127300	-2671	-4,2%	-4,6%
wayDescIdx	2501010	13759	2440817	13315	-60193	-444	-2,5%	-3,3%
wayTagDictionary	4467	11930	4219	11715	-248	-215	-5,9%	-1,8%
Total		172506		166314		-6192		-3.7%

Not sure what all these stats mean but overall a small decrees in size.

I saw that the mapcreation with a small patch:

@@ -104,18 +104,18 @@ public class OsmCutter extends MapCreatorBase
     }
     cyclewayDos.close();
     if ( restrictionsDos != null )
     {
       restrictionsDos.close();
     }
 
-//    System.out.println( "-------- way-statistics -------- " );
-//    _expctxWayStat.dumpStatistics();
-//    System.out.println( "-------- node-statistics -------- " );
-//    _expctxNodeStat.dumpStatistics();
+    System.out.println( "-------- way-statistics -------- " );
+    _expctxWay.dumpStatistics();
+    System.out.println( "-------- node-statistics -------- " );
+    _expctxNode.dumpStatistics();
 
     System.out.println( statsLine() );
   }

dumps the way and node stats for the data processed, filtered through looksup.dat.

After that still the NodeFilter is run, I want to see if I can run planet.osm.pbf and get stats on the nodes remaining this filtering maybe that gives more unused tags as only nodes connected the way network are filtered out.

It would be also good some feedback from profile developers especially @poutnikl I think.

[EssBee59]

It took a while but I got the traffic class into my hessen test

Hello,

I started a short retest with the last rd5 from adischerdev:
Yes, the tag "estimated_traffic_class" is now suported well!

But data for "bicycle_road" seems not available...
My test:
http://brouter.de/essbee/#map=16/50.0197/8.6778/osm-mapnik-german_style&lonlats=8.672504,50.020756;8.686731,50.017515

Remark: using the rd5 from Ployscia (see above), the tag bicycle_road was available

[EssBee59]

I found another strange situation using the rd5 afischerdev:
==> #map=16/49.8622/8.9165/osm-mapnik-german_style&lonlats=8.916135,49.863205;8.916778,49.861226
("cycleway:surface=sett" which is not in OSM)

As explained, due to errors or weakness in the tempoary rd5 file tests are much difficult.
The new lookups.dat seems for me ok, but we have time to activate it, and I would prefer to retest with the final lookups and rd5.

[afischerdev]

@EssBee59
Thanks for the remark.
I'll have a look on it when I'm back home.

[polyscias]

I see the segment4_hessen.zip has in it E5_N50.rd5, E5_N45.rd5 and E10_N50.rd5 but no lookups.dat so likely there is a disconnect between which lookups.dat was used to generated the .rd5's and the one @EssBee59 is using.

If that is not the case, can it be because we have the number for new keys/vals set to 00000001?

[afischerdev]

@polyscias
I agree, it seems a not synchrone lookups.dat. And it would have been a good idea to include the lookups.dat in the zip file.

@EssBee59
I used this one to generate the hessen file.

[EssBee59]

Yes, a retest with the right lookups.dat looks much better!
I only found differences within the tag "estimated_traffic_class", it is some times 1 class below the brouter.de instance, or it is missing.
example: #map=14/49.8668/8.9244/osm-mapnik-german_style&lonlats=8.863134,49.890374;8.929653,49.853065

(of course "elevation" is not available in this test version)

[afischerdev]

The discussion seems to be over. So it should become real.
I had to rebase repository, so the updates look very new but are old from discussion.

What do you think?

[afischerdev]

Add a check for man_made=pier in all.brf
Add hiking-beta.brf again. Is needed for current app. Has now the same content like hiking-mountain.brf and can be removed when new app is online.

[Mehrwegflasche]

Hi, I just wanted to ask:  Is this issue still being worked on?  I’d love to see the cycleway:both tags listed in the lookup so I can use them in my custom profile. 

Edit:  Nevermind, I just found issue #458.