Fix regression for is_monotonic_increasing with nan in MultiIndex

[phofl]

• closes BUG: MultiIndex.is_montonic erronoes when nan in at least one level #37220
• closes Investigate whether re-encoding MultiIndex levels is beneficial #27498 (closes that one too probably, because we need this case)
• tests added / passed
• passes black pandas
• passes git diff upstream/master -u -- "*.py" | flake8 --diff
• whatsnew entry

Found this while debugging a different issue.

[dsaxton]

Maybe parametrize over different positions for NaN? I think it's probably correct to always return False here, although this is interesting:

[ins] In [32]: idx
Out[32]: Float64Index([nan, 1.0, 2.0], dtype='float64')

[ins] In [33]: idx.sort_values(ascending=True).is_monotonic_increasing
Out[33]: False

[phofl]

That really is interesting. With 0.25.3
pd.MultiIndex.from_tuples([(1,), (np.nan,)], names=["test"]).is_monotonic_increasing returned True. We could change that, but that would be more than a Regression fix then?

[dsaxton]

I think probably fine to call it a regression fix. I would say the previous behavior is not correct in this case (if monotone increasing means x <= y for every adjacent pair of elements, then no sequence with NaN values is monotone increasing, or decreasing for that matter), so we'd only want to fix the case with NaN at the beginning rather than restoring the behavior completely.

[phofl]

Thanks very much for the explanation. Adjusted the code accordingly.

[dsaxton]

Thanks @phofl

[simonjayhawkins]

@phofl can you resolve conflict.

[phofl]

@simonjayhawkins Done

[jreback]

this looks fine. can you run some asv's to see if any perf diffs here? (prob don't need to add, want to see the existing ones).

[phofl]

@jreback

 before           after         ratio
     [57c06d9b]       [31ef0b98]
     <36915^2>        <37220>   
-        63.5±5ms       56.5±0.4ms     0.89  multiindex_object.SetOperations.time_operation('non_monotonic', 'string', 'symmetric_difference')

SOME BENCHMARKS HAVE CHANGED SIGNIFICANTLY.
PERFORMANCE INCREASED.

Non monotonic indices increased (not surprising). No changes otherwise

[jreback]

thanks @phofl

[simonjayhawkins]

@meeseeksdev backport 1.1.x