Add ability to save and load param store

[ndgAtUber]

We need the ability to easily (and reasonably quickly) save the param store to disk, and re-load it.

This is useful in a long training run both for saving state for future analyses and for checkpointing in case of a crash.

[eb8680]

We should also think about an interface and implementation to save and load models themselves. With cloudpickle we can serialize pretty much anything in Python except for running/suspended generators and coroutines.

[karalets]

For the parameter store I envision something to the effect of pyro.save(model_params,"model_params.pr") and pyro.load("model_params.pr") which will save and load the name-space and the affiliated parameters.

We may need to think about being able to define which names we want to load and mapping other names to these parameters, for instance when initializing another model partially with pre-trained parameters or some such hacky business.

[ngoodman]

this seems like the right first pass.

i think (from conversations) the main tricky thing is going to be inserting params into neural net modules when we re-load. presumably pytorch provides (or can be coerced to provide) some way to set the params of a module. we may need to pass into pyro.load a handle to modules or something like that....

the alternative is pickling the entire module and re-loading it. that seems like a less good idea to me.

[eb8680]

some way to set the params of a module

See here, should be pretty straightforward: serializing models in PyTorch

[ngoodman]

ah, cool. so as long as the param store keeps around a handle to the models from pyro.module calls we should be good to use the first method.

[eb8680]

Addressed by #47