Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

issue creating parquet format file #14

Closed
mitch-bollinger opened this issue Mar 6, 2023 · 3 comments
Closed

issue creating parquet format file #14

mitch-bollinger opened this issue Mar 6, 2023 · 3 comments

Comments

@mitch-bollinger
Copy link

mitch-bollinger commented Mar 6, 2023

For some reason the dataframe in notebook "02 - Preprocessing London Smart Meter Dataset" does not seem to be able to convert to Parquet format per this screen shot. I have tried setting the encoder to both fastparquet and pyarrow but still no luck. This would not be a problem except for the fact that the next chapter relies on this file so I am sort of stuck until I figure it out. Any ideas?

image

@manujosephv
Copy link
Collaborator

Are you getting the same error for pyarrow?

And as a workaround, you can save the dataframe as a pickle or some other format and in future chapters, when reading, use the same format. Although Pickle did take up a lot of space and time when I tried it.

@mitch-bollinger
Copy link
Author

Thanks so much for the feedback. I got back on it today and started over and it worked this time. Best I can guess pyarrow and fastparquet has installed but perhaps but starting over with a new kernel they worked this time.

Also, this book is great as it is quite a challenge for someone like me who does not have an extensive Python programming background. But this is how you get good at it, learning from people who have a lot more experience.

@manujosephv
Copy link
Collaborator

Glad you figured it out. closing the issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

No branches or pull requests

2 participants