Threshold scale - limits at five groups / six values?

[psychemedia]

folium 0.5.0

By observation of default choropleth maps, the legend seems to be split into six groups (so a list of seven boundary values).

But if a custom threshold scale is provided, then this seems to limit at five groups (the max length of the boundary value list is six items)?

if threshold_scale and len(threshold_scale) > 6

Why the difference?

More concerningly (by observation; I'm working with sensitive data and don't have a test example to hand at the moment) I notice that whilst the limits on the legend are updated to values provided via the threshold_scale parameter, the colours depicted on the map correspond to default value ranges, not the supplied threshold_scale values?

In the following:

mymap = folium.Map(zoom_start=10, max_zoom=15)

mymap.choropleth(geo_data='data/geodata.geojson',
                       data=df, 
                       columns=['code', 'data'], 
                       key_on='feature.properties.Name',
                       fill_color='RdYlBu', fill_opacity=0.4, line_opacity=0.9,
                       legend_name='*******', 
                       threshold_scale=[50, 60, 70, 80, 90, 100])

it doesn't matter what I set the threshold_scale to, the map coloring remains the same as used with the default scale, although the legend does change to use the colour boundaries I have set?

UPDATE: it seems that the red coloured group lower band coloring in the above map was being used to colour None value items in the data; in the default scale, no colouring is applied to those groups. It would be useful to be able to specify a 'novaluecolour, perhaps set toNone` by default (no colouring) or a specified colour if provided.

---

Another issue with threshold_scale appears to be a fencepost error in that if the upper bound of the threshold_scale list is less than the upper bound data value, an error occurs (as below), although no error is raised at the other end where the lower bound of the threshold_scale may be set above the lower bound value of the data.

IndexError                                Traceback (most recent call last)
/usr/local/lib/python3.5/dist-packages/IPython/core/formatters.py in __call__(self, obj)
    343             method = get_real_method(obj, self.print_method)
    344             if method is not None:
--> 345                 return method()
    346             return None
    347         else:

/usr/local/lib/python3.5/dist-packages/folium/map.py in _repr_html_(self, **kwargs)
    249             self._parent = None
    250         else:
--> 251             out = self._parent._repr_html_(**kwargs)
    252         return out
    253 

/usr/local/lib/python3.5/dist-packages/branca/element.py in _repr_html_(self, **kwargs)
    317 
    318         """
--> 319         html = self.render(**kwargs)
    320         html = "data:text/html;charset=utf-8;base64," + base64.b64encode(html.encode('utf8')).decode('utf8')  # noqa
    321 

/usr/local/lib/python3.5/dist-packages/branca/element.py in render(self, **kwargs)
    307         """Renders the HTML representation of the element."""
    308         for name, child in self._children.items():
--> 309             child.render(**kwargs)
    310         return self._template.render(this=self, kwargs=kwargs)
    311 

/usr/local/lib/python3.5/dist-packages/folium/map.py in render(self, **kwargs)
    336             '</style>'), name='map_style')
    337 
--> 338         super(LegacyMap, self).render(**kwargs)
    339 
    340 

/usr/local/lib/python3.5/dist-packages/branca/element.py in render(self, **kwargs)
    622 
    623         for name, element in self._children.items():
--> 624             element.render(**kwargs)

/usr/local/lib/python3.5/dist-packages/branca/element.py in render(self, **kwargs)
    618         script = self._template.module.__dict__.get('script', None)
    619         if script is not None:
--> 620             figure.script.add_child(Element(script(self, kwargs)),
    621                                     name=self.get_name())
    622 

/usr/local/lib/python3.5/dist-packages/jinja2/runtime.py in __call__(self, *args, **kwargs)
    573                             (self.name, len(self.arguments)))
    574 
--> 575         return self._invoke(arguments, autoescape)
    576 
    577     def _invoke(self, arguments, autoescape):

/usr/local/lib/python3.5/dist-packages/jinja2/runtime.py in _invoke(self, arguments, autoescape)
    577     def _invoke(self, arguments, autoescape):
    578         """This method is being swapped out by the async implementation."""
--> 579         rv = self._func(*arguments)
    580         if autoescape:
    581             rv = Markup(rv)

<template> in macro(l_1_this, l_1_kwargs)

/usr/local/lib/python3.5/dist-packages/jinja2/runtime.py in call(_Context__self, _Context__obj, *args, **kwargs)
    260                 args = (__self.environment,) + args
    261         try:
--> 262             return __obj(*args, **kwargs)
    263         except StopIteration:
    264             return __self.environment.undefined('value was undefined because '

/usr/local/lib/python3.5/dist-packages/folium/features.py in style_data(self)
    563 
    564         for feature in self.data['features']:
--> 565             feature.setdefault('properties', {}).setdefault('style', {}).update(self.style_function(feature))  # noqa
    566             feature.setdefault('properties', {}).setdefault('highlight', {}).update(self.highlight_function(feature))  # noqa
    567         return json.dumps(self.data, sort_keys=True)

/usr/local/lib/python3.5/dist-packages/folium/folium.py in style_function(x)
    303                 'color': line_color,
    304                 'fillOpacity': fill_opacity,
--> 305                 'fillColor': color_scale_fun(x)
    306             }
    307 

/usr/local/lib/python3.5/dist-packages/folium/folium.py in color_scale_fun(x)
    290             def color_scale_fun(x):
    291                 return color_range[len(
--> 292                     [u for u in color_domain if
    293                      get_by_key(x, key_on) in color_data and
    294                      u <= color_data[get_by_key(x, key_on)]])]

IndexError: list index out of range

[Conengmo]

You're right about the difference between the default number of legend items and the max on the custom threshold_scale. It seems it has been this way since a long time. What value do you think we should stick to, 6 or 7?

I'm not sure what you're saying about the None's. I suppose those shouldn't be allowed anyway.

I'm not seeing the error when the largest value in threshold_scale is lower than the largest value in the data. Can you first try updating to the most recent version of folium from git, and see if the error is still there? If so, please provide a code snippet that produces your error.

[Conengmo]

By the way this is my code snippet to test your problem:

m = folium.Map(location=(40, -90), tiles='cartodbpositron',  zoom_start=5)
path = os.path.join(os.path.dirname(folium.__file__), '..', 'tests')
with open(os.path.join(path, 'us-counties.json')) as f:
    geo_json_data = json.load(f)
column = 'Unemployment_rate_2011'
with open(os.path.join(path, 'us_county_data.csv')) as f:
    df = pd.read_csv(f, dtype={column: np.float64}, na_values=' ')
df['Area_name'] = df['Area_name'].apply(lambda x: x.strip('County').strip())
m.choropleth(
    geo_data=geo_json_data,
    data=df,
    columns=['Area_name', column],
    key_on='feature.properties.name',
    fill_color='RdYlBu',
    threshold_scale=[0, 2, 4, 8, 16, 32],
)
m.save("_map.html")