Loading onnx file with external data via CreateSessionFromArray or equivalent

[prsmendonca]

I have a model that was converted to onnx, and because its size is larger than 2 GB it was split into .onnx and .onnx.data files. I'd like to load the data from the .onnx and the .onnx.data files into buffers, and the reconstitute the original model in memory. Is that possible? A somewhat in-between solution where I'd load the .onnx file into a buffer and read the contents of the .onnx.data file from a path would be acceptable, although much less desirable. I'm mentioning this option mainly because it seems there is a Python equivalent to it via onnx.external_data_helper, but I could not find a C/C++ equivalent.

Note that for a simple .onnx file, without external data, OrtApi::CreateSessionFromArray does something similar to what I want, and I have that working for smaller models. For a different use case, also with a large model with external data, I can use OrtApi::CreateSession with the path of the .onnx file alone, and that works too. But in the current use case I specifically need to load the contents of the files into memory, a la OrtApi::CreateSessionFromArray or equivalent.

[pranavsharma]

See

onnxruntime/include/onnxruntime/core/session/onnxruntime_c_api.h

Lines 3488 to 3490 in 8e6019a

	ORT_API2_STATUS(AddExternalInitializers, _In_ OrtSessionOptions* options,
	_In_reads_(input_len) const char* const* initializer_names,
	_In_reads_(input_len) const OrtValue* const* initializers, size_t initializers_num);

[prsmendonca]

Thank you for the reply and sorry for my delay. This does look promising, but I was looking into test_model_loading.cc, and it seems that this method loads (indeed as external data) each tensor in the model by name/key. Specifically, in that test "Pads_not_on_disk" is a name/key known a priori, for a tensor with known size matching a key in the example file "testdata/model_with_external_initializer_come_from_user.onnx". This is somewhat different from CreateSessionFromArray, which loads all tensors at once from a buffer populated with data read from a model file. The latter has the advantage of not requiring a priori knowledge of the tensor names and sizes. Moreover, I don't how the model tensors are laid out in a .onnx.data file, if I were to try to individually reconstitute each tensor of the model.

If all I have are the .onnx and .onnx.data files for a model, is AddExternalInitializers a viable option to load the model from buffers into which the data from .onnx and .onnx.data files have been loaded?

[jf99]

For those still interested: There is now a function AddExternalInitializersFromFilesInMemory, which does exactly what is needed here. The current test_model_loading.cc file shows how to use it.