Expose `Solver::Snapshot` to pycaffe

[gustavla]

This addresses #3077, making it possible to manually save a snapshot (caffemodel+solverstate) from Python.

On a related note, does it also make sense to expose Solver::TestAll, so that it's possible to manually instigate a test run through Python?

[longjon]

Looks good -- I wonder if we could have a basic test though?

Exposing TestAll in a different PR sounds good to me.

[gustavla]

Sorry for delay on this. I added a test that runs solver.snapshot() and checks if the expected files were generated.

[shelhamer]

Thanks for the pycaffe extension @gustavla!

[mpkuse]

Is there also a way to load a snapshot into the solver with pycaffe?

[rogertrullo]

Hi @gustavla , could you please provide an example on how to use this feature?
in #3077 you mentioned that you wrote a SIGTERM interrupt handler?
Thanks!

[rayryeng]

@mpkuse Just use the restore method that accompanies the solver. The input is the path to your .solverstate file that gets saved during snapshots --> solver.restore('your_solverstate_file.solverstate'). It's also mentioned in the Python section under Interfaces in the Caffe documentation: http://caffe.berkeleyvision.org/tutorial/interfaces.html

[VasLem]

Is there a way to set snapshot file name from inside a Python program, or is it necessary to have the name inside the solver prototxt from the beginning? Sorry if I ask something obvious, but there is no documentation, apart from this page and http://caffe.berkeleyvision.org/tutorial/interfaces.html.

[naibaf7]

@VasLem
This example demonstrates the Python interface to it's full extent. However do note some options may only be available on the opencl branch and not (yet) mainline. But it should give you a good idea of it:
https://github.com/naibaf7/opencl_caffe_examples/blob/master/mnist_lenet/mnist_lenet.ipynb