Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Can't pickle database #103

Closed
rwest opened this issue Nov 29, 2012 · 3 comments
Closed

Can't pickle database #103

rwest opened this issue Nov 29, 2012 · 3 comments

Comments

@rwest
Copy link
Member

rwest commented Nov 29, 2012

Most parallel processing frameworks require us to pickle things, like the database. Currently I get "TypeError: can't pickle instancemethod objects" when I try.

@jwallen
Copy link
Contributor

jwallen commented Nov 29, 2012

It looks like we may just need to add a few __reduce__() methods to some of the database classes to help pickle understand how to handle them. I have managed to pickle the thermo database after doing this (it didn't work before), and am working on the kinetics and frequencies databases.

@jwallen
Copy link
Contributor

jwallen commented Nov 29, 2012

Some timings from my test script (for the entire database):

Loaded database in 33.23 s
Pickled database in 11.54 s
Unpickled database in 24.81 s

@rwest
Copy link
Member Author

rwest commented Nov 29, 2012

Hmm - so storing it pickled wouldn't save us much.

@faribas this explains why our parallel implementation which requires pickling and unpickling the entire database for every edge species is kind of slow! (don't worry - we have a better idea...)

Thanks for fixing, Josh

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants