Fix for supporting DF categoricals unboxing#923
Conversation
kozlov-alexey
requested review from
AlexanderKalistratov,
PokhodenkoSA and
densmirn
and removed request for
densmirn
| return lambda column: np.array(column) | ||
|
|
||
| if isinstance(column, (types.Array, StringArrayType)): | ||
| if isinstance(column, (types.Array, StringArrayType, Categorical)): |
There was a problem hiding this comment.
Are there other methods which need the same fix?
There was a problem hiding this comment.
I think half of all DF and Series methods should have some special handling of Categoricals (see e.g. df.values). If we talk only about boxing/unboxing/fix_df_array/fix_df_index function, then probably no, since we don't support CategoricalIndexes yet. At least it looks so.
| boxed = func(pd_value) | ||
| assert(boxed.equals(pd_value)) | ||
|
|
||
| @skip_numba_jit("capturing DFs (not only categoricals) as freevar not working") |
There was a problem hiding this comment.
OK, here's what's going on: this doesn't test lowering, but test possibility of capture as freevar/const and it currently works only for categorical series since impl for them was added via @lower_const in
sdc/sdc/datatypes/series/boxing.py
Line 33 in 8c0914a
Probably the same can be done for DFs, non-categorical Series and other types, but it's outside the scope of this PR.
3 participants
No description provided.