Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature request] allow datetime index #386

Closed
ZhongSiming opened this issue Feb 19, 2020 · 3 comments
Closed

[Feature request] allow datetime index #386

ZhongSiming opened this issue Feb 19, 2020 · 3 comments

Comments

@ZhongSiming
Copy link

ZhongSiming commented Feb 19, 2020

I have hourly data for 3 years using a datetime as the index. Pandas allows me load/save with the following code (only one month with 2 variables shown):
`

Write data to .csv

jan90.to_csv('PEC fine course 1 grid 199001.csv', index=True)

Load data from .csv

jan90 = pd.read_csv('PEC fine course 1 grid 199001.csv', index_col=0, parse_dates=True)
`
Using .csv works, but is slow when I get to the full dataset of 26k+ rows and 21.6k+ columns (and more columns may be coming if I have to add lags to my data). So, a more efficient load/save routine is very desirable. I was excited when I found feather, but the lost index is a no-go for my use.

Thanks for your consideration.

@wesm
Copy link
Owner

wesm commented Feb 19, 2020

Could you open a JIRA issue in Apache Arrow (https://github.com/apache/arrow/blob/master/CONTRIBUTING.md)? That's where we're doing Feather development now.

@ZhongSiming
Copy link
Author

ZhongSiming commented Feb 21, 2020 via email

@wesm
Copy link
Owner

wesm commented Feb 24, 2020

Thanks

@wesm wesm closed this as completed Feb 24, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants