Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BUG: Handle iterable of arrays in convert #16026

Merged
merged 3 commits into from
Apr 18, 2017

Conversation

TomAugspurger
Copy link
Contributor

DatetimeConverter.convert can take an array or iterable of arrays.
Fixed the converter to detect which case we're in and then re-use
the existing logic.

Posting here for feedback from @tacaswell. Specifically,

  1. Presumably our other two converters TimeConverter and PeriodConverter could suffer from the same problem?
  2. if is_list_like(values) and is_list_like(values[0]): feels hacky... Thoughts?

xref matplotlib/matplotlib#8459

@codecov
Copy link

codecov bot commented Apr 17, 2017

Codecov Report

Merging #16026 into master will increase coverage by <.01%.
The diff coverage is 100%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master   #16026      +/-   ##
==========================================
+ Coverage   90.99%      91%   +<.01%     
==========================================
  Files         153      153              
  Lines       50469    50481      +12     
==========================================
+ Hits        45924    45939      +15     
+ Misses       4545     4542       -3
Flag Coverage Δ
#multiple 88.77% <100%> (ø) ⬆️
#single 40.43% <25%> (-0.01%) ⬇️
Impacted Files Coverage Δ
pandas/core/dtypes/inference.py 98.38% <100%> (+0.05%) ⬆️
pandas/plotting/_converter.py 63.54% <100%> (+0.56%) ⬆️
pandas/util/testing.py 80.73% <0%> (+0.18%) ⬆️
pandas/core/common.py 91.03% <0%> (+0.34%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 90dd3f9...a786627. Read the comment docs.

@tacaswell
Copy link
Contributor

  1. I would also suspect.
  2. we have similar looking stuff to deal with ambiguous import from the user. That looks reasonable to me. Alternative is to try...except passing to the 1d version and then fall back to the list comprehension?

@ghost
Copy link

ghost commented Apr 17, 2017

Nice solution. One slight concern though: line 184 of _converter.py looks a bit buggy:
if is_list_like(values) and is_list_like(values[0]):
If values == [], it looks like is_list_like(values[0]) will cause an error while is_list_like(values) still returns True

@@ -178,6 +180,16 @@ class DatetimeConverter(dates.DateConverter):

@staticmethod
def convert(values, unit, axis):
# values might be a 1-d array, or a list-like of arrays.
if is_list_like(values) and is_list_like(values[0]):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

use

is_list_like(values) and all(is_list_like(v) for v in values).

you could make this a method in pandas.core.dtypes.common if you want (with tests!) maybe
is_list_like_nested (I don't know if there is a case where any of the sub-lists is True but you don't want all)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A (minor) problem with all(is_list_like(v) for v in values) is that this returns True for an empty list (the values=[] case).

It's only minor in that it still works but the final return result will become an empty list instead of an empty float array. Which might be a problem.

(I am not sure to what extent a Converter is supposed to return an array in matplotlib code)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ahh could add that to the condition as well (so even more reason to make this an instrospection function).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is_list_like(values) and len(values) and all(is_list_like(v) for v in values)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's roughly what I ended up doing. One issue is that I don't want this accidentally consuming a generator, so if the outer container doesn't define len we return False. Will push in a few minutes.

@jreback jreback added the Bug label Apr 17, 2017
Copy link
Contributor

@jreback jreback left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

whatsnew note, comment on impl. otherwise lgtm.

@jorisvandenbossche
Copy link
Member

@TomAugspurger Can you also add a small test for the Converter itself? https://github.com/pandas-dev/pandas/blob/master/pandas/tests/plotting/test_converter.py#L30 eg just an example with that string in a nested list

DatetimeConverter.convert can take an array or iterable of arrays.
Fixed the converter to detect which case we're in and then re-use
the existing logic.
@TomAugspurger
Copy link
Contributor Author

Can you also add a small test for the Converter itself?

Added a test for PeriodConverter and DateTimeConverter. I don't see tests for TimeConverter, so I haven't touched that code at all.


def test_convert_nested(self):
data = ['2012-1-1', '2012-1-2']
r1 = self.pc.convert(data, None, self.axis)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this one is not nested (needs a data = [data, data]?)


Examples
--------
>>> is_list_like([1, 2, 3])
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These examples are still copied from the is_list_like docstring I think

list, Series, np.array, tuple
])
def test_is_nested_list_like_passes(inner, outer):
result = outer([inner for _ in range(5)])
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice!

@jreback jreback added this to the 0.20.0 milestone Apr 18, 2017
@jreback jreback merged commit cd1031f into pandas-dev:master Apr 18, 2017
@TomAugspurger TomAugspurger deleted the mpl-hist-2d branch May 29, 2017 20:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants