CLN: Post Series subclass from NDFrame #4324

jreback · 2013-07-22T23:45:56Z

After #3482 is merged, some additional methods to fixup

Series.replace (use generic/replace) via f4fe5fd
Series align (use generic/align), PR CLN: move align out of series consolidating into core/generic.py #4800
Move Arithmetic/Comparison methods to generic, ENH: Use numexpr for all arithmetic operators + refactor arithmetic operations #4051, BUG/CLN: numpy compat with pandas numeric functions and cln of same (GH4435) #5034
Fillna column wise, column-wise fillna with Series/dict NotImplemented #4514
Verify Copy for Series (deep=True), Implement deep copy option in Series.copy #2721, PR API/CLN: make rename and copy consistent across NDFrame #4627
Verify that deep=True is implemented in core/internals/copy, related to ENH/BUG: Fix names, levels and labels handling in MultiIndex #4039, PR API/CLN: make rename and copy consistent across NDFrame #4627
consolidate tests as appropriate to tests/test_generic.py (ones that should have a conforming API, e.g. replace,filter) see ENH: unify the replace API #4118, (test_generic is created via API/CLN: make rename and copy consistent across NDFrame #4627)
add AXIS_LEN check in filter, PR CLN: _axis_len checking cleanup and better message #4855
rename consistency between series/frame, see Inconsistent behaviour of rename for MultiIndexed DataFrames and Series #4605, PR API/CLN: make rename and copy consistent across NDFrame #4627
clean up reindex / fix method= issues BUG: reindex strange behaviour #4604, PR is CLN: refactor Series.reindex to core/generic #4610
interpolation, consolidate Series/DataFrame interpolation to
internals.Block.interpolatte, ENH: Richer options for interpolate and resample #4434, cc @TomAugspurger
change setitem_with_indexer to use Block.setitem (core/indexing), PR API/CLN: setitem in Series now consistent with DataFrame #4624
move _get_numeric_data/_get_bool_data to generic.py PR CLN/API: Refactor of _get_numeric_data/_get_bool_data to core/generic.py, allowing Series/Panel functionaility #4675
fix tools/plotting for series sub of NDFrame PR CLN/API: Refactor of _get_numeric_data/_get_bool_data to core/generic.py, allowing Series/Panel functionaility #4675
~~remove use of common/_maybe_upcast_putmask and possibly_cast_item~~
clip ( see BUG: DataFrame.clip sets all values to the lower_bound #2747), need to move from frame.py to generic.py (series is ok), PR CLN: move clip to core/generic (adds to Panel as well), related to (GH2747) #4798
take ( PR BUG: Bug with reindexing on the index with a non-unique index will now raise a ValueError #4757 )
update, CLN: move update to BlockManager method (and just call from frame) #3025

The text was updated successfully, but these errors were encountered:

cpcloud · 2013-07-23T23:00:25Z

@jreback What do you think about a core.common.align_multiple function that essentially does what I do in pd.eval, i.e., align a collection of pandas objects, but it would be exposed as an API method? Would that be useful? I could open a PR for that and maybe a new label "internals" for things involving pandas internals...

This would happen after the eval PR is merged.

jreback · 2013-07-23T23:05:04Z

in #3482 I basically just moved align to generic, and all it really does is make calls out to _align_series and _align_frame (FYI, there is not _align_panel). This is also needed in core/indexing.py mainly by the setitem methods.

So would +1 be in favor of that. maybe create a new module though, pandas.core.align ? and has the 2 cases, get (the align method and set (the setitem in the indexing stuff)

jreback · 2013-07-23T23:05:23Z

is refactor label differen tthan internals?

cpcloud · 2013-07-23T23:10:09Z

internals could be for things/ideas like discussions around the BlockManager like the one that @wesm raised in your series ndframe PR. refactor i think of as like "clean this up because too much technical debt has been accumulated here"

cpcloud · 2013-07-23T23:12:09Z

While my alignment does "work" for nd objects it's not clear that there's a "right" way to do it. E.g., what axis should a Series/DataFrame align on with a Panel?

cpcloud · 2013-07-23T23:13:23Z

i'll open a new issue for this to prevent too much OT stuff here

jtratner · 2013-08-13T22:48:43Z

@jreback - I have most of the arithmetic refactor done - but I'd like to discuss a few decisions. Does it make sense for me to open it as a separate PR right now (based off your branch) so (hopefully) it can make it in soon after your big Series change?

jreback · 2013-08-13T22:50:43Z

that's totally fine
I keep rebasing to current master so as long as you want to rebase off of mine ok by me

jtratner · 2013-08-13T22:59:38Z

Yeah, I just tag your current master when I rebase, and then cherry-pick, which makes it pretty painless, basically:

git checkout jreback/series2
git cherry-pick series2_base..arithmetic-refactor
git branch -D arithmetic-refactor
git checkout -b arithmetic-refactor
git tag -d series2_base
git tag series2_base jreback/series2

jreback · 2013-08-13T23:02:03Z

git fu master!

cpcloud · 2013-08-13T23:28:48Z

@jtratner nice!

cpcloud · 2013-08-14T01:52:53Z

let me see if i follow that git....

cherry pick commits from arithmetic-refactor back to but not including series2_base on to the series2 branch
make ur branch again
replace your tag with the new merge base

so essentially u r creating a new tag with ur changes every time @jreback rebases

is that correct?

jtratner · 2013-08-14T02:05:18Z

@cpcloud well, you have to create the tag before @jreback rebases (so it's the base for your current work), but yes, each time I basically move the base commit tag to be the last commit of series2. i.e.

git checkout some-branch-that-might-rebase
git tag base_commit some-branch-that-might-rebase
git checkout -b  my-branch

and then after some-branch-that-might-rebase rebases, you just do:

git fetch # or merge or whatever
git checkout some-branch-that-might-rebase
git cherry-pick base_commit..my-branch
# now, update the tag to be the new base commit
git tag -d base_commit
git tag base_commit some-branch-that-might-rebase

and then you can do whatever else you want

jtratner · 2013-08-14T02:05:49Z

but yes, that's basically it. I like it because you don't have to remember any hashes.

cpcloud · 2013-08-14T02:08:41Z

just to play devils advocate how is this different from just git rebase jreback/td? or in your case

git fetch jreback
git checkout arithmethic-refactor
git rebase jreback/td

jtratner · 2013-08-14T02:25:40Z

@cpcloud what if upstream combines multiple previous commits? I only want to apply my commits on top of wherever it ended up.

cpcloud · 2013-08-14T02:28:52Z

ah i see, so yours takes into account possible rewritten history and you don't really care if that's happened....that is pretty useful....care to add to git tips when u get a chance?

jreback · 2013-08-14T02:33:42Z

what if the commit that series2_base is deleted, eg rebased away upstream?

jtratner · 2013-08-14T02:37:30Z

@jreback you'll always have it locally, because that's in the history of the local branch you're working on. the cherry-pick just runs off of the refs you say. and git cherry-pick a..b does not include a, but includes b. (you can use git cherry-pick a^..b to include a as well)

jtratner · 2013-08-14T02:42:26Z

@cpcloud I added it to the git tips with some additional explanation and a hopefully strongly worded note so newbies to git don't read it and think they need to do that off master. (which would work, but add extra complexity for them)

TomAugspurger · 2013-08-15T13:15:46Z

I'm not sure if this is the right place to ask, but is there a reason for the Panel constructor not taking a dict of Series?

In [26]: a = pd.Series(np.random.randn(4))

In [27]: b = pd.Series(np.random.randn(4), index=np.arange(1, 5))

In [28]: pd.Panel({'a': a, 'b': b})

will results in

---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
<ipython-input-28-bb125e2a1174> in <module>()
----> 1 pd.Panel({'a': a, 'b': b})

/Users/tom/python2.7/lib/python2.7/site-packages/pandas-0.12.0_169_gb2ad044-py2.7-macosx-10.8-x86_64.egg/pandas/core/panel.pyc in __init__(self, data, items, major_axis, minor_axis, copy, dtype)
    234         self._init_data(
    235             data=data, items=items, major_axis=major_axis, minor_axis=minor_axis,
--> 236             copy=copy, dtype=dtype)
    237 
    238     def _init_data(self, data, copy, dtype, **kwargs):

/Users/tom/python2.7/lib/python2.7/site-packages/pandas-0.12.0_169_gb2ad044-py2.7-macosx-10.8-x86_64.egg/pandas/core/panel.pyc in _init_data(self, data, copy, dtype, **kwargs)
    252             mgr = data
    253         elif isinstance(data, dict):
--> 254             mgr = self._init_dict(data, passed_axes, dtype=dtype)
    255             copy = False
    256             dtype = None

/Users/tom/python2.7/lib/python2.7/site-packages/pandas-0.12.0_169_gb2ad044-py2.7-macosx-10.8-x86_64.egg/pandas/core/panel.pyc in _init_dict(self, data, axes, dtype)
    293         # extract axis for remaining axes & create the slicemap
    294         raxes = [self._extract_axis(self, data, axis=i)
--> 295                  if a is None else a for i, a in enumerate(axes)]
    296         raxes_sm = self._extract_axes_for_slice(self, raxes)
    297 

/Users/tom/python2.7/lib/python2.7/site-packages/pandas-0.12.0_169_gb2ad044-py2.7-macosx-10.8-x86_64.egg/pandas/core/panel.pyc in _extract_axis(self, data, axis, intersect)
   1560             elif v is not None:
   1561                 have_raw_arrays = True
-> 1562                 raw_lengths.append(v.shape[axis])
   1563 
   1564         if have_frames:

IndexError: tuple index out of range

Obviously I can just recast a and b as DataFrames before passing them off to the Panel, so it's not a big deal; I was just curious.

jreback · 2013-08-15T13:19:21Z

@TomAugspurger its not obvious what the axes should be here (you only specified 2, but you need 3). Maybe need a better error message on that.

jreback · 2013-08-15T13:21:02Z

@TomAugspurger already had one, see #4185, would welcome a PR on this!

TomAugspurger · 2013-08-15T13:35:47Z

Thanks. I see what you're saying about the axes.

jreback · 2013-09-29T20:03:21Z

moving unchecked to #5044

ghost assigned jreback Jul 22, 2013

jreback mentioned this issue Jul 23, 2013

ENH: Use numexpr for all arithmetic operators + refactor arithmetic operations #4051

Closed

3 tasks

jreback mentioned this issue Jul 29, 2013

Implement deep copy option in Series.copy #2721

Closed

jreback mentioned this issue Aug 15, 2013

ER: better error messages on Panel construction with non-nested dict #4185

Closed

hayd mentioned this issue Aug 16, 2013

BUG: (iloc) Comparisons of Series have different indexing semantics from DataFrame comparisons #4581

Closed

jreback mentioned this issue Sep 10, 2013

API: DataFrame.interpolate will be deprecated in v0.13 #3676

Closed

cpcloud mentioned this issue Sep 27, 2013

ENH: unify the replace API #4118

Closed

jreback mentioned this issue Sep 29, 2013

CLN: 0.14 NDFrame cleaning #5044

Closed

7 tasks

jreback closed this as completed Sep 29, 2013

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

CLN: Post Series subclass from NDFrame #4324

CLN: Post Series subclass from NDFrame #4324

jreback commented Jul 22, 2013

cpcloud commented Jul 23, 2013

jreback commented Jul 23, 2013

jreback commented Jul 23, 2013

cpcloud commented Jul 23, 2013

cpcloud commented Jul 23, 2013

cpcloud commented Jul 23, 2013

jtratner commented Aug 13, 2013

jreback commented Aug 13, 2013

jtratner commented Aug 13, 2013

jreback commented Aug 13, 2013

cpcloud commented Aug 13, 2013

cpcloud commented Aug 14, 2013

jtratner commented Aug 14, 2013

jtratner commented Aug 14, 2013

cpcloud commented Aug 14, 2013

jtratner commented Aug 14, 2013

cpcloud commented Aug 14, 2013

jreback commented Aug 14, 2013

jtratner commented Aug 14, 2013

jtratner commented Aug 14, 2013

TomAugspurger commented Aug 15, 2013

jreback commented Aug 15, 2013

jreback commented Aug 15, 2013

TomAugspurger commented Aug 15, 2013

jreback commented Sep 29, 2013

CLN: Post Series subclass from NDFrame #4324

CLN: Post Series subclass from NDFrame #4324

Comments

jreback commented Jul 22, 2013

cpcloud commented Jul 23, 2013

jreback commented Jul 23, 2013

jreback commented Jul 23, 2013

cpcloud commented Jul 23, 2013

cpcloud commented Jul 23, 2013

cpcloud commented Jul 23, 2013

jtratner commented Aug 13, 2013

jreback commented Aug 13, 2013

jtratner commented Aug 13, 2013

jreback commented Aug 13, 2013

cpcloud commented Aug 13, 2013

cpcloud commented Aug 14, 2013

jtratner commented Aug 14, 2013

jtratner commented Aug 14, 2013

cpcloud commented Aug 14, 2013

jtratner commented Aug 14, 2013

cpcloud commented Aug 14, 2013

jreback commented Aug 14, 2013

jtratner commented Aug 14, 2013

jtratner commented Aug 14, 2013

TomAugspurger commented Aug 15, 2013

jreback commented Aug 15, 2013

jreback commented Aug 15, 2013

TomAugspurger commented Aug 15, 2013

jreback commented Sep 29, 2013