Import ONNX with low_memory setting

[lauracanalini]

Hi, I was trying to import the resnet101 ONNX file with the low_memory setting from import_net_from_onnx_file.
However, when I try to use a batch_size greater than 1, the training explode in the resize of the network, in particular of the first Conv layer, in the ConvolDescriptor::resize when it tries to free the ptrI pointer (line 265 of descriptor_conv2D.cpp). I am using EDDL 1.0.4b.
A simple program that reproduces the error:

#include <iostream>

#include "eddl/apis/eddl.h"
#include "eddl/serialization/onnx/eddl_onnx.h"

using namespace eddl;

int main(int argc, char **argv) {
    auto resnet101 = import_net_from_onnx_file("resnet101.onnx", { 3, 224, 224 }, 0);
    auto resnet101_low_mem = import_net_from_onnx_file("resnet101.onnx", { 3, 224, 224 }, 2);

    auto conv = dynamic_cast<LConv*>(resnet101->layers[1]);
    auto conv_low_mem = dynamic_cast<LConv*>(resnet101_low_mem->layers[1]);

    eddl_free(conv->cd->ptrI);
    cout << "Conv ptrI freed" << endl;
    eddl_free(conv_low_mem->cd->ptrI);  // here the program breaks
    cout << "Conv_low_mem ptrI freed" << endl;
    
    return EXIT_SUCCESS;
}

Regardless of the error, does it make sense to import the ONNX with the low_memory setting or passing "low_memory" to the computing service when the network is built does the same thing?

[chavicoski]

Hi,

If you want to use the model with a "low_mem" configuration, you should say it in the constructor of the CompServ. The argument "mem" of the ONNX import function is not valid for doing that, and we will probably remove this argument in the future.

You can see the example using the CompServ here:

#include <iostream>

#include "eddl/apis/eddl.h"
#include "eddl/serialization/onnx/eddl_onnx.h"

using namespace eddl;

int main(int argc, char **argv) {
    auto resnet101 = import_net_from_onnx_file("resnet101.onnx", { 3, 224, 224 }, 0);
    auto resnet101_low_mem = import_net_from_onnx_file("resnet101.onnx", { 3, 224, 224 }, 0); // <- Use 0

    auto conv = dynamic_cast<LConv*>(resnet101->layers[1]);
    auto conv_low_mem = dynamic_cast<LConv*>(resnet101_low_mem->layers[1]);

    build(resnet101,
          adam(),
          {"softmax_cross_entropy"},
          {"categorical_accuracy"},
          CS_CPU(-1, "full_mem")); // <- "full_mem"

    build(resnet101_low_mem,
          adam(),
          {"softmax_cross_entropy"},
          {"categorical_accuracy"},
          CS_CPU(-1, "low_mem")); // <- "low_mem"

    eddl_free(conv->cd->ptrI);
    cout << "Conv ptrI freed" << endl;
    eddl_free(conv_low_mem->cd->ptrI);  // Now the program doesn't break
    cout << "Conv_low_mem ptrI freed" << endl;

    return EXIT_SUCCESS;
}

And regarding your last question, you can import the model and then use any configuration that you want, it just changes the amount of memory reserved for the operations, but the results should be the same.