Create kwarg to normalize histogram such that sum of bin values equals 1

[louist87]

This StackOverflow question sums up the problem pretty well. The answer is straightforward but convoluted. This kind of functionality is widely used and requested, so it would be of great benefit for the hist function to handle it automatically.

Basically, setting the normed kwarg to a logically true value normalizes the histogram such that the integral is equal to 1. I suggest creating a new kwarg, named something like normedheight (or perhaps normedh), which implements the solution proposed in the StackOverflow post.

Here is the code, for convenience:

inp = normal(size=1000)
h = hist(inp)
bar(h[1][:-1], h[0]/float(len(inp)), diff(h[1]))  # normalized histogram

[piti118]

how about having normed accept multiple types
-bool bahave like it used to
-callable with two argument accept h and bins as argument and compute the normalization
-number normalize area to that amount
-string reserved for special case like what's described

[louist87]

@piti118 That's a good call. Might I suggest normed=height?

[WeatherGod]

Usually, I am against such ideas where a parameter can accept multiple types, but this might be a case where the use-case is compelling enough to over-ride such concerns. In this situation, the requested feature is not orthogonal to the existing features, and so adding a new kwarg would only complicate things.

I would suggest keeping it as simple as possible, though, and accept only booleans and (positive) numbers, to start. This approach makes sense because True == 1. Strings and callables I would avoid for now until a really compelling use-case is put forth.

[louist87]

@WeatherGod I humbly disagree with your suggestion to limit the kwarg values to booleans and positive integers. My concern is that it makes for very cryptic function calls because it's not immediately clear what the integers represent.

The original issue is that matplotlib normalizes the area under the curve to 1 instead of the height of the bars. In this case, it's not the numerical value that's of primary importance, but what that value represents. If we were to make it so that 2 represents the kind of height normalization I originally requested, the function call `hist(data, bins=20, normed=2) makes it seem as though we're normalizing to 2. Even if normalizing to 2 seems unlikely, the casual reader has to refer to the documentation to understand what's going on. This is not very pythonic.

Basically, I want to avoid the situation where I constantly have to refer to the documentation to remember which value to pass. A call such as hist(data, bins=20, normed='height') is much more explicit. In fact, if we decide to overload the normed kwarg, then I would suggest aliasing True with a string such as auc (area under the curve) or deriv (derivative). That way, passing True would still work as expected, but there's also a new way of writing more explicit code.

[piti118]

@WeatherGod I'm just toying the idea and alternative around. Your expert comment is more than welcome.

But in this case there are two types of normalization we need to distinguish.

1. Normalize by area like it is now
2. Normalize by sum like what's being suggested.

If we only allow normed=1., I'm not sure which one it will be using.

I hate adding another kwarg too but we might need one in this case. Lets call it norm_mode which takes string either area or sum and do the appropriate normalization.

normed keyword in numpy histogram is being deprecated anyway.
http://docs.scipy.org/doc/numpy/reference/generated/numpy.histogram.html

[WeatherGod]

On Thu, Dec 13, 2012 at 8:22 PM, Piti Ongmongkolkul <
notifications@github.com> wrote:

@WeatherGod https://github.com/WeatherGod I'm just toying the idea and
alternative around. Your expert comment is more than welcome.

But in this case there are two types of normalization we need to
distinguish.

1. Normalize by area like it is now
2. Normalize by sum like what's being suggested.

If we only allow normed=1., I'm not sure which one it will be using.

I hate adding another kwarg too but we might need one in this case. Lets
call it norm_mode which takes string either area or sum and do the
appropriate normalization.

Actually, I like that sort of distinction, and it does make the issues that
I was confusing in my head nicely orthogonal.

normed keyword in numpy histogram is being deprecated anyway.
http://docs.scipy.org/doc/numpy/reference/generated/numpy.histogram.html

Oy! Not again...

[louist87]

I'd be willing to take a stab at fixing this, but I've never contributed anything significant before. If someone feels like doing it instead, the community might be better off!

In the meantime, is this the place to ask for clarification regarding the code in axes.hist? If not, is there a more appropriate place? A developer's IRC, perhaps?

[piti118]

Contributing code is easy and fun. I have contributed to matplotlib couple times. I found the community to be super friendly.

Just make a pull request/issue and one of repo collab will help you.

[louist87]

@piti118 Alright, then! I'll take a crack at it sometime this week. If I get stuck, I'll be sure to let someone know.

[ivanov]

In the meantime, is this the place to ask for clarification regarding the code in axes.hist?

matplotlib-users list for help with using it, matplotlib-devel list for proposing changes. sign up here (note, the archives are broken on there, you can find the archives here)

[dmcdougall]

Contributing code is easy and fun. I have contributed to matplotlib couple times. I found the community to be super friendly.

Just make a pull request/issue and one of repo collab will help you.

It is feedback like this that makes me happy to contribute to this open source project. Thank you.

[neggert]

How's progress on this coming? I'm doing some work on the hist function anyway, so I could probably implement this if no one has started working on it yet.

What would make sense to me would be to let normed take either a string or a bool. So normed='area' gives the current behavior and normed='sum' gives the new behavior. We make normed=True equivalent to normed='area' both to preserve backwards compatibility and because IMO that's what most users want when asking for a normalized histogram.

[louist87]

@nic I haven't had a chance to get started -- have at it!

On Sat, Jan 26, 2013 at 4:27 PM, Nic Eggert notifications@github.comwrote:

How's progress on this coming? I'm doing some work on the hist function
anyway, so I could probably implement this if no one has started working on
it yet.

What would make sense to me would be to let normed take either a string
or a bool. So normed='area' gives the current behavior and normed='sum'gives the new behavior. We make
normed=True equivalent to normed='area' both to preserve backwards
compatibility and because IMO that's what most users want when asking for a
normalized histogram.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/1567#issuecomment-12736801.

Louis Thibault
+33 6 40 24 06 05

[phobson]

@neggert et al, think there's any value in a normed='max' option so that the tallest bar equals one? Just thinking out loud here

[tacaswell]

At some point does it make sense to be able to pass hist an arbitrary normalization function instead of trying to guess every possible normalization?

The function would take a single argument of a numpy array (the raw bin counts), and returns an array of the exact same shape scaled how ever the user wants.

Does it make sense to try and leverage the Normalize classes that already exist in color?