BUG: Handle iterable of arrays in convert #16026

Merged
merged 3 commits into from Apr 18, 2017

Conversation

Projects
None yet
6 participants
Contributor

TomAugspurger commented Apr 17, 2017

DatetimeConverter.convert can take an array or iterable of arrays.
Fixed the converter to detect which case we're in and then re-use
the existing logic.

Posting here for feedback from @tacaswell. Specifically,

  1. Presumably our other two converters TimeConverter and PeriodConverter could suffer from the same problem?
  2. if is_list_like(values) and is_list_like(values[0]): feels hacky... Thoughts?

xref matplotlib/matplotlib#8459

codecov bot commented Apr 17, 2017 edited

Codecov Report

Merging #16026 into master will increase coverage by <.01%.
The diff coverage is 100%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master   #16026      +/-   ##
==========================================
+ Coverage   90.99%      91%   +<.01%     
==========================================
  Files         153      153              
  Lines       50469    50481      +12     
==========================================
+ Hits        45924    45939      +15     
+ Misses       4545     4542       -3
Flag Coverage Δ
#multiple 88.77% <100%> (ø) ⬆️
#single 40.43% <25%> (-0.01%) ⬇️
Impacted Files Coverage Δ
pandas/core/dtypes/inference.py 98.38% <100%> (+0.05%) ⬆️
pandas/plotting/_converter.py 63.54% <100%> (+0.56%) ⬆️
pandas/util/testing.py 80.73% <0%> (+0.18%) ⬆️
pandas/core/common.py 91.03% <0%> (+0.34%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 90dd3f9...a786627. Read the comment docs.

Contributor

tacaswell commented Apr 17, 2017

  1. I would also suspect.
  2. we have similar looking stuff to deal with ambiguous import from the user. That looks reasonable to me. Alternative is to try...except passing to the 1d version and then fall back to the list comprehension?
Contributor

Tuan333 commented Apr 17, 2017 edited

Nice solution. One slight concern though: line 184 of _converter.py looks a bit buggy:
if is_list_like(values) and is_list_like(values[0]):
If values == [], it looks like is_list_like(values[0]) will cause an error while is_list_like(values) still returns True

pandas/plotting/_converter.py
@@ -178,6 +180,16 @@ class DatetimeConverter(dates.DateConverter):
@staticmethod
def convert(values, unit, axis):
+ # values might be a 1-d array, or a list-like of arrays.
+ if is_list_like(values) and is_list_like(values[0]):
@jreback

jreback Apr 17, 2017

Contributor

use

is_list_like(values) and all(is_list_like(v) for v in values).

you could make this a method in pandas.core.dtypes.common if you want (with tests!) maybe
is_list_like_nested (I don't know if there is a case where any of the sub-lists is True but you don't want all)

@jorisvandenbossche

jorisvandenbossche Apr 17, 2017

Owner

A (minor) problem with all(is_list_like(v) for v in values) is that this returns True for an empty list (the values=[] case).

It's only minor in that it still works but the final return result will become an empty list instead of an empty float array. Which might be a problem.

(I am not sure to what extent a Converter is supposed to return an array in matplotlib code)

@jreback

jreback Apr 17, 2017

Contributor

ahh could add that to the condition as well (so even more reason to make this an instrospection function).

@jreback

jreback Apr 17, 2017

Contributor

is_list_like(values) and len(values) and all(is_list_like(v) for v in values)

@TomAugspurger

TomAugspurger Apr 17, 2017

Contributor

That's roughly what I ended up doing. One issue is that I don't want this accidentally consuming a generator, so if the outer container doesn't define len we return False. Will push in a few minutes.

jreback added the Bug label Apr 17, 2017

@jreback

whatsnew note, comment on impl. otherwise lgtm.

@TomAugspurger Can you also add a small test for the Converter itself? https://github.com/pandas-dev/pandas/blob/master/pandas/tests/plotting/test_converter.py#L30 eg just an example with that string in a nested list

@TomAugspurger TomAugspurger BUG: Handle iterable of arrays in convert
DatetimeConverter.convert can take an array or iterable of arrays.
Fixed the converter to detect which case we're in and then re-use
the existing logic.
e731cc0
Contributor

TomAugspurger commented Apr 17, 2017

Can you also add a small test for the Converter itself?

Added a test for PeriodConverter and DateTimeConverter. I don't see tests for TimeConverter, so I haven't touched that code at all.

@TomAugspurger TomAugspurger PEP8: remove unused is_list_like
d0b0453
pandas/tests/plotting/test_converter.py
+
+ def test_convert_nested(self):
+ data = ['2012-1-1', '2012-1-2']
+ r1 = self.pc.convert(data, None, self.axis)
@jorisvandenbossche

jorisvandenbossche Apr 17, 2017

Owner

this one is not nested (needs a data = [data, data]?)

pandas/core/dtypes/inference.py
+
+ Examples
+ --------
+ >>> is_list_like([1, 2, 3])
@jorisvandenbossche

jorisvandenbossche Apr 17, 2017

Owner

These examples are still copied from the is_list_like docstring I think

@TomAugspurger TomAugspurger Fix Joris comments
a786627
+ list, Series, np.array, tuple
+])
+def test_is_nested_list_like_passes(inner, outer):
+ result = outer([inner for _ in range(5)])
@jreback

jreback Apr 18, 2017

Contributor

nice!

jreback added this to the 0.20.0 milestone Apr 18, 2017

@jreback jreback merged commit cd1031f into pandas-dev:master Apr 18, 2017

5 checks passed

ci/circleci Your tests passed on CircleCI!
Details
codecov/patch 100% of diff hit (target 50%)
Details
codecov/project 91% (target 82%)
Details
continuous-integration/appveyor/pr AppVeyor build succeeded
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details

TomAugspurger deleted the TomAugspurger:mpl-hist-2d branch May 29, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment