Skip to content

Conversation

@jbrockmendel
Copy link
Member

The assertions in here are intended to be temporary. Once reviewers are convinced this is accurate, then the assertions can be removed and a bunch of these blocks can be condensed down to share code.

The new helper func _get_data_and_dtype_name is also going to be used later to pre-empt the need for the state-altering get_data method.

@jbrockmendel jbrockmendel added Refactor Internal refactoring of code IO HDF5 read_hdf, HDFStore labels Dec 8, 2019
@jreback jreback added this to the 1.0 milestone Dec 8, 2019
Copy link
Contributor

@jreback jreback left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good about the assertions

k4 = kind[4:]
col_name = f"UInt{k4}Col"
elif kind.startswith("period"):
# we store as integer
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we may have very little coverage for PeriodIndex FYI

# For datetime64tz we need to drop the TZ in tests TODO: why?
dtype_name = data.dtype.name.split("[")[0]

if data.dtype.kind in ["m", "M"]:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

use needs_i8_conversion (followon ok)

# TODO: we used to reshape for the dt64tz case, but no longer
# doing that doesnt seem to break anything. why?

elif isinstance(data, PeriodIndex):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can likely use extract_array here

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that would give PeriodArray, we need to get i8 back here

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

right once you have the arrays, then you can simply .view('i8') and you are done (pretty generically)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok i see what you're saying now. will do on next pass (along with needs_i8 mentioned above)

@jreback jreback merged commit d1bc773 into pandas-dev:master Dec 10, 2019
@jreback
Copy link
Contributor

jreback commented Dec 10, 2019

looks good. if you can do follows as indicated above when you have a chance.

@jbrockmendel jbrockmendel deleted the ref-pytables-convert_index branch December 10, 2019 16:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

IO HDF5 read_hdf, HDFStore Refactor Internal refactoring of code

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants