-
Notifications
You must be signed in to change notification settings - Fork 410
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Joblib silently ignores pickle exceptions and does not use cache #896
Comments
Indeed, the |
Btw, if you have a reproducing case, I would be very interested, to make sure we solve the problem at hand. |
Should be something as simple as import joblib as jb
import pandas as pd
memory = jb.memory.Memory()
@memory.cache
def f():
return list(pd.DataFrame({'a': [1]}).itertuples())
f() |
Note that if you do not provide an argument But indeed, this example reproduce the problem. |
Right, sorry - I was just reproducing it from my head. In the real code, I have |
Fixed by #1359 |
I'm trying to use
memory.cache
on a function that returns a bunch of Pandas named tuples made withlist(dataframe.itertuples())
.Apparently, such tuples cannot be pickled. I'm fine with that, however I'm not fine with the fact that Joblib doesn't even log the issue. Because of the empty
except
at https://github.com/joblib/joblib/blob/master/joblib/_store_backends.py#L193 the pickle exception is just ignored, and the hint about race condition doesn't really help to debug the issue.The text was updated successfully, but these errors were encountered: