New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

"AssertionError: Gaps in blk ref_locs" when unstacking filtered DataFrame #18562

Closed
toobaz opened this Issue Nov 29, 2017 · 2 comments

Comments

Projects
None yet
3 participants
@toobaz
Member

toobaz commented Nov 29, 2017

Code Sample, a copy-pastable example if possible

In [2]: df = pd.DataFrame([[2009, 1, 15, 'City1', 'Housing'],
   ...:                    [2010, 2, np.nan, 'City2', 'Housing']],
   ...:                   columns=['year', 'month', 'day', 'city', 'sector'])
   ...:                   

In [3]: df['day'] = 15
   ...: df['date'] = pd.to_datetime(df[['year', 'month', 'day']])
   ...: 

In [4]: ind = df.set_index(['date', 'sector', 'city'], drop=False).sort_index()

In [5]: 

In [5]: cols = ['month', 'year', 'sector']
   ...: per_city = ind.loc[(slice(None), slice(None), 'City1'), cols].dropna(how='any')
   ...: 

In [6]: per_city.unstack()
---------------------------------------------------------------------------
AssertionError                            Traceback (most recent call last)
<ipython-input-6-97028d1092eb> in <module>()
----> 1 per_city.unstack()

/home/nobackup/repo/pandas/pandas/core/frame.py in unstack(self, level, fill_value)
   4575         """
   4576         from pandas.core.reshape.reshape import unstack
-> 4577         return unstack(self, level, fill_value)
   4578 
   4579     _shared_docs['melt'] = ("""

/home/nobackup/repo/pandas/pandas/core/reshape/reshape.py in unstack(obj, level, fill_value)
    463     if isinstance(obj, DataFrame):
    464         if isinstance(obj.index, MultiIndex):
--> 465             return _unstack_frame(obj, level, fill_value=fill_value)
    466         else:
    467             return obj.T.stack(dropna=False)

/home/nobackup/repo/pandas/pandas/core/reshape/reshape.py in _unstack_frame(obj, level, fill_value)
    476         unstacker = partial(_Unstacker, index=obj.index,
    477                             level=level, fill_value=fill_value)
--> 478         blocks = obj._data.unstack(unstacker)
    479         klass = type(obj)
    480         return klass(blocks)

/home/nobackup/repo/pandas/pandas/core/internals.py in unstack(self, unstacker_func)
   4374         new_columns = new_columns[columns_mask]
   4375 
-> 4376         bm = BlockManager(new_blocks, [new_columns, new_index])
   4377         return bm
   4378 

/home/nobackup/repo/pandas/pandas/core/internals.py in __init__(self, blocks, axes, do_integrity_check, fastpath)
   3042         self._consolidate_check()
   3043 
-> 3044         self._rebuild_blknos_and_blklocs()
   3045 
   3046     def make_empty(self, axes=None):

/home/nobackup/repo/pandas/pandas/core/internals.py in _rebuild_blknos_and_blklocs(self)
   3134 
   3135         if (new_blknos == -1).any():
-> 3136             raise AssertionError("Gaps in blk ref_locs")
   3137 
   3138         self._blknos = new_blknos

AssertionError: Gaps in blk ref_locs

@toobaz toobaz changed the title from "ValueError: could not broadcast input array from shape" to "AssertionError: Gaps in blk ref_locs" when unstacking filtered DataFrame Nov 29, 2017

@toobaz

This comment has been minimized.

Member

toobaz commented Nov 29, 2017

The error changes to ValueError: could not broadcast input array from shape (2) into shape (0) if I change cols = ['month', 'year', 'sector'] to cols=slice(None).

@toobaz

This comment has been minimized.

Member

toobaz commented Nov 29, 2017

Might be related to #17845 .

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment