fill_between interpolation & nan issue

[Phlya]

The issue was discovered here
deeptools/pyGenomeTracks#37

Basically, when fill_between encounters a NaN in the y values, when interpolate=True it produces a strange output.

This is without interpolation:

import matplotlib.pyplot as plt
import numpy as np

x = np.arange(107).astype(float)
y = np.append(np.zeros((100))+0.01, [0.5, 0.75, 1, np.nan, 1, 0.75, 0.5])

plt.figure(figsize=(20, 5))
plt.fill_between(x, y, interpolate=False)

If I turn interpolation on (which I wouldn't expect to do anything in this case, since there are not zero crossings), this is what I get:

plt.figure(figsize=(20, 5))
plt.fill_between(x, y, interpolate=True)

Even if I am misunderstanding something about how interpolation is supposed to work, clearly this is not what it should do...

mpl 2.1.2, python 3.5.2

[Phlya]

Another example to clarify what's going on

x = np.arange(107).astype(float)
y = np.append(np.zeros((100))+0.01, [2, 0.75, 1, np.nan, 1, 0.75, 0.5])

plt.figure(figsize=(20, 5))
plt.fill_between(x, y, interpolate=True)

[Phlya]

Also possible to make an example where this fails when there actually are negative values and a where=y>0 condition:

x = np.arange(101).astype(float)
y = np.append(np.zeros((100))+np.random.random(100)-0.05, [np.nan])

plt.figure(figsize=(20, 5))
plt.fill_between(x, y, interpolate=True, where=y>0)

[jklymak]

While you are testing, have you tried masking instead of passing NaN?

[Phlya]

No, sorry - but can try tomorrow.

[timhoffm]

Part of the problem is that

_axes.py l.5039:

x, y1, y2 = np.broadcast_arrays(np.atleast_1d(x), y1, y2)

casts the masked arrays back to regular arrays.

_axes.py l.5063ff:

                    if len(diff_values) == 2:
                        if np.ma.is_masked(diff_values[1]):
                            return x[im1], y1[im1]
                        elif np.ma.is_masked(diff_values[0]):
                            return x[ind], y1[ind]

does not have an effect anymore (the invalid values are not masked anymore). It works if you replace np.ma.is_masked()by ~np.isfinite(). No sure if this is the right way to go, though.

[jklymak]

Good find; seems like thats definitely the issue.

Marking as "units", because how we normalize arrays after they come to us is tied up in the units discussion. It'd be nice if every x/y data array input to axes methods got the same consistent conversion, and that would certainly include how to handle NaN versus masked....

[Phlya]

Just in case, masking the NaN value doesn't help.

[jklymak]

This looks to have been introduced in #7562... ping @anntzer.

[anntzer]

I can't repro the issue, neither with mpl2.1.2 (original report) or master...

[jklymak]

You can't reproduce with interpolate=True?

[anntzer]

This is what I get when running the example copy-pasted from #11781 (comment) with master:

[jklymak]

#11781 (comment) ?

[anntzer]

[jklymak]

import matplotlib.pyplot as plt
import numpy as np

x = np.arange(107).astype(float)
y = np.append(np.zeros((100))+0.01, [0.5, 0.75, 1, np.nan, 1, 0.75, 0.5])

plt.figure()
plt.fill_between(x, y, interpolate=True)
plt.show()

[anntzer]

What version of numpy? I'm on 1.16.2.
I can't think of anything else...

[jklymak]

1.14.6... Upgrading to 1.16 fixes. So, I guess we can close this....