How to load params in C++ deploy?

[WolffyChen]

Hi all,

I use python fuction "nnvm.compiler.save_param_dict(params)" to generate "deploy.params".
Then I load it by "bytearray(open('deploy.params', "rb").read())" in python and run the deploy code successfully.
But when I try to deploy in c++ and use function "LoadParams" to load "deploy.params" that created in python, the program crashed with an error: "../src/runtime/graph/graph_runtime.cc:404: Check failed: strm->Read(&reserved) Invalid parameters file format".
The file's binary content in c++ is same with bytearray in python. Why this error happened and how I can fix it?

Thank you very much!
Hope Reply.

[tqchen]

LoadParams actually takes string from blob to load the parameters, so the string being passed in should not be file name, but should be the binary blob that is already loaded from the deploy.params

[WolffyChen]

I actually use the following code to load parameters. Does it not the right way to load binary blob?
tvm::runtime::PackedFunc load_params = mod.GetFunction("load_params");
std::ifstream pin("deploy.params", std::ios::binary);
std::stringstream ss;
ss << pin.rdbuf();
std::string params_data = ss.str();
pin.close();
load_params(params_data);

[tqchen]

The parameter would need to be TVMByteArray type to indicate the binary data

[WolffyChen]

Many thanks. I transfer the parameters to TVMByteArray type as follows and it works!
TVMByteArray arr;
arr.data = params_data.c_str();
arr.size = params_data.length();
load_params(arr);

[tqchen]

Your case study raises an interesting problems we did not write clearly in the nnvm how to deploy doc. Can you contribute a patch to current tutorial on what you learnt here ？thanks !

[WolffyChen]

I am glad to make a small contribution. What type of patch should I provide? Add a C++ example in "nnvm/docs/how_to/deploy.md" or something else?

[tqchen]

Yes that would be great

[WolffyChen]

dmlc/nnvm#219

[pallabdatta]

Hi @tqchen, @WolffyChen

I am trying to access the learned weights (params) with a graph pass on the C++ side. For..e.g. I see all the node params but I can't seem to have access to the learned weights..Note: within a pass, I haven't yet built the runtime module, i..e the (graph, lib, params = nnvm.compiler.build(...)) Can we save the params after the following:

onnx_graph = onnx.load('network.onnx')
sym, params = nnvm.frontend.from_onnx(onnx_graph) ? before we go into the compiler.build and then load it directly via some C++ calls? within a graph pass?
Can you please help me with this?
thanks a lot in advance,
Pallab