No docs for torch::from_blob

[Suhail]

📚 Documentation

Hi there, I was searching for documentation regarding torch::from_blob which is loosely demonstrated but I don't understand the API. I looked here: https://pytorch.org/cppdocs/api/namespace_torch.html#functions

but it's not listed.

I am trying to convert a vector that's {1, 3, 28, 28} to something I can pass to from_blob. I cannot tell if from_blob will take an array or if the array needs to flat somehow. If it's flat, how will it shape a list of floats into that shape?

Code:

    auto blob = j.get<vector<vector<vector<vector<float>>>>>();

    cout << blob << endl;

    at::TensorOptions options(at::ScalarType::Byte);
    at::Tensor t = torch::from_blob(blob, {1, 3, 28, 28});

I get an error:

error: no matching function for call to ‘from_blob(std::vector<std::vector<std::vector<std::vector<float, std::allocator<float> > > > >&, <brace-enclosed initializer list>)’
     at::Tensor t = torch::from_blob(blob, {1, 3, 28, 28});

Here's the actual json of the image: https://gist.github.com/Suhail/81ef7a763fbd0e5329b25fa41d31ad41

Documentation would help me :)

[soumith]

cc: @goldsborough

[goldsborough]

I will add documentation for it. In the meantime, the definition for it is here. The first argument is a void* and should point to a flat region of memory. You could first convert your nested vector into a flat vector by looping through each element and copying it into the new vector, and then use from_blob by passing it flat_vector.data() where flat_vector is your flattened C++ std::vector. Although if you're already going to copy, you're probably better off just creating an empty tensor and copying into it:

auto blob = j.get<vector<vector<vector<vector<float>>>>>();
auto tensor = torch::empty(1 * 3 * 28 * 28);
float* data = tensor.data<float>();
for (const auto& i : blob) {
    (for const auto& j : i) {
        for (const auto& k : j) {
            for (const auto& l : k) {
                *data++ = l;
            }
        }
    }
}

// do things with tensor

[Suhail]

Thank you for being so speedy! Super helpful.

Good to know it's just flat -- that means I don't know need json in the first place most likely.

Regardless, I am wondering if there's something a bit off about the code above since I am getting different inference results on successive results with the same data.

Sometimes I get a -nan, -nan result. Could be the model though:

Model loaded.
 1.9708e+26  1.1496e+26
[ Variable[CPUFloatType]{1,2} ]
paperspace@psokdwo8n:~/mnist-torch-script-test/build$ ./example-app ../mnist-traced.pt
Model loaded.
-nan -nan
[ Variable[CPUFloatType]{1,2} ]
paperspace@psokdwo8n:~/mnist-torch-script-test/build$ ./example-app ../mnist-traced.pt
Model loaded.
0.01 *
-3.6975 -41.3416
[ Variable[CPUFloatType]{1,2} ]

Here's my code:

#include <iostream>
#include <fstream>
#include <memory>
#include <string>
#include <exception>

#include <nlohmann/json.hpp>
#include <torch/script.h>

using namespace std;
using json = nlohmann::json;

const bool DEBUG_VERBOSE = false;

json get_img_convert_to_tensor(string);
string get_json_from_file(string);

int main(int argc, const char* argv[]) {
    if (argc != 2) {
        cerr << "usage: example-app <path-to-exported-script-module>\n";
        return -1;
    }

    // Deserialize the ScriptModule from a file using torch::jit::load().
    shared_ptr<torch::jit::script::Module> module = torch::jit::load(argv[1]);

    assert(module != nullptr);
    cout << "Model loaded.\n";

    vector<torch::jit::IValue> inputs;;
    //inputs.push_back(torch::ones({1, 3, 28, 28}));

    vector<int64_t> sizes = {1, 3, 28, 28};
    json j = get_img_convert_to_tensor(get_json_from_file("../29300.png.json"));

    auto blob = j.get<vector<vector<vector<vector<float>>>>>();
    auto tensor = torch::empty(1 * 3 * 28 * 28);
    float* data = tensor.data<float>();

    for (const auto& i : blob) {
        for (const auto& j : i) {
            for (const auto& k : j) {
                for (const auto& l : k) {
                    *data++ = l;
                }
            }
        }
    }

    at::TensorOptions options(at::ScalarType::Byte);

    at::Tensor t = torch::from_blob(data, {1, 3, 28, 28});
    t = t.toType(at::kFloat);

    inputs.emplace_back(t);

    // Run inference
    auto output = module->forward(inputs).toTensor();
    cout << output.slice(/*dim=*/1, /*start=*/0, /*end=*/5) << '\n';

}

Model: https://drive.google.com/file/d/14vzQV2UXigkrxAfcvqzQDsFg5AXBSgt7/view?usp=sharing

[goldsborough]

Sorry if I was unclear, but since tensor is already a tensor, there is no need to use from_blob anymore. I.e. replace

at::TensorOptions options(at::ScalarType::Byte);

at::Tensor t = torch::from_blob(data, {1, 3, 28, 28});
t = t.toType(at::kFloat);

inputs.emplace_back(t);

with

inputs.emplace_back(tensor);

I don't think that will solve the NaNs, but at leas the tensor operations look correct to me here. Do you get NaNs in Python?

[Suhail]

Hmm, I get this error now:

./example-app ../mnist-traced.pt
Model loaded.
terminate called after throwing an instance of 'c10::Error'
  what():  Expected 4-dimensional input for 4-dimensional weight [64, 3, 7, 7], but got 1-dimensional input of size [2352] instead (check_input_shape_forward at /pytorch/aten/src/ATen/native/Convolution.cpp:170)
frame #0: std::function<std::string ()>::operator()() const + 0x11 (0x7fa5ebdd50d1 in /home/paperspace/mnist-torch-script-test/libtorch/lib/libc10.so)

    auto blob = j.get<vector<vector<vector<vector<float>>>>>();
    auto tensor = torch::empty(1 * 3 * 28 * 28);
    float* data = tensor.data<float>();

    for (const auto& i : blob) {
        for (const auto& j : i) {
            for (const auto& k : j) {
                for (const auto& l : k) {
                    *data++ = l;
                }
            }
        }
    }

    //cout << data << endl;

    /*at::TensorOptions options(at::ScalarType::Byte);

    at::Tensor t = torch::from_blob(data, {1, 3, 28, 28});
    t = t.toType(at::kFloat);
    */

    inputs.emplace_back(tensor);

    // Run inference
    auto output = module->forward(inputs).toTensor();
    cout << output.slice(/*dim=*/1, /*start=*/0, /*end=*/5) << '\n';

[goldsborough]

try inputs.emplace_back(tensor.resize({1, 3, 28, 28}));

[Suhail]

Works! But, I looked & it looks like it's tensor.resize_({1, 3, 28, 28})); with a _

[mxgbs]

Hi,

I actually have the same problem when creates a tensor with an array data. I follow what you did and the major difference is that I directy assign *data with imgC_float which is a flat array. Though it works, I got warning when compiling "ATen/core/TensorBody.h:354:7: note: declared here T * data() const " Could you please help with this? The note seems indicate something is wrong.

for (int i = 0; i < 128 * 128 * 3; i++)
{
	*data++ = imgC_float[i];
}