Learner.export() fails with a pickle error.

[jakubLangr]

When running a trained model:

learn.export('models/unet-basic-v3')
---------------------------------------------------------------------------
PicklingError                             Traceback (most recent call last)
<ipython-input-16-7bf582ede4a1> in <module>
      1 learn.save('unet-basic-v3')
----> 2 learn.export('models/unet-basic-v3')

~/daisy-gan/venv/lib/python3.6/site-packages/fastai2/learner.py in export(self, fname)
    580         #To avoid the warning that come from PyTorch about model not being checked
    581         warnings.simplefilter("ignore")
--> 582         torch.save(self, self.path/fname)
    583     self.create_opt()
    584     self.opt.load_state_dict(state)

~/daisy-gan/venv/lib/python3.6/site-packages/torch/serialization.py in save(obj, f, pickle_module, pickle_protocol)
    258         >>> torch.save(x, buffer)
    259     """
--> 260     return _with_file_like(f, "wb", lambda f: _save(obj, f, pickle_module, pickle_protocol))
    261 
    262 

~/daisy-gan/venv/lib/python3.6/site-packages/torch/serialization.py in _with_file_like(f, mode, body)
    183         f = open(f, mode)
    184     try:
--> 185         return body(f)
    186     finally:
    187         if new_fd:

~/daisy-gan/venv/lib/python3.6/site-packages/torch/serialization.py in <lambda>(f)
    258         >>> torch.save(x, buffer)
    259     """
--> 260     return _with_file_like(f, "wb", lambda f: _save(obj, f, pickle_module, pickle_protocol))
    261 
    262 

~/daisy-gan/venv/lib/python3.6/site-packages/torch/serialization.py in _save(obj, f, pickle_module, pickle_protocol)
    330     pickler = pickle_module.Pickler(f, protocol=pickle_protocol)
    331     pickler.persistent_id = persistent_id
--> 332     pickler.dump(obj)
    333 
    334     serialized_storage_keys = sorted(serialized_storages.keys())

PicklingError: Can't pickle <function <lambda> at 0x7fba35a15f28>: attribute lookup <lambda> on __main__ failed

This is probably related to scikit-learn/scikit-learn#9467 as fastai2 seems to make heavy use of lambda functions. Will investigate where this comes from. This is probably higher priority for me than #60

This error can be triggered by simply including this line at the end of the camvid example.

[sgugger]

pickle can't pickle lambda functions and the data in the camvide example is defined with a lambda function, that's why it doesn't pickle. This isn't a bug, this is how pickle works. To fix your problem:

1. define a get_y_camvid function (that does the same thing as the lambda function in get_y DataBlock)
2. if you plan to use your model in a different setup for deployment, put that function in a utils module you import (do this before exporting) and put this utils module in your production environment too (otherwise pickle won't find that function)

[muellerzr]

To expand on 2:

I've reset the metrics to get rid of it (learn.metrics = []) and then manually brought it in in a utils module like @sgugger recommends. This way you can successfully pickle it

learn.metrics = []
learn.export()

[MadisLemsalu]

For someone that is not so familiar with fastai, how does the function look like in case of the Camvid notebook?

Would get_y=ColReader('label')' work?

Since I'm using DICOM as input, I cannot get anything else except lambda to work.

Also it remains elusive on how to put the function into a utils module. Is it a existing utils module or a new file?