Consider not using lambdas as default arguments to enable pickling

[benjamin-work]

Problem description

Some classes have lambdas as default arguments (e.g. here). This prevents these objects from being pickled.

Steps to Reproduce the Problem

import pickle
from torchnlp.text_encoders import StaticTokenizerEncoder

encoder = StaticTokenizerEncoder(['hi', 'you'])
pickle.dumps(encoder) 

# raises error
PicklingError: Can't pickle <function StaticTokenizerEncoder.<lambda> at 0x7fb77c042b70>: attribute lookup StaticTokenizerEncoder.<lambda> on torchnlp.text_encoders.static_tokenizer_encoder failed

Solution

Replace lambdas by actual functions.

[PetrochukM]

Until I get a freecycle to get to this, consider using dill a substitute for pickle that supports lambda functions.

[benjamin-work]

Yes, dill would work, but some libs resort to pickle (joblib). Maybe when I have time, I could make a PR for this but I wanted to make sure that this is something you would like to change.

[PetrochukM]

Ahh, Gotcha. Thanks for your patience. Feel free to make a PR this. This sounds like a super reasonable use case.

[PetrochukM]

Thanks for your PR! Merged it with a couple updates.