GetDimensionsCount return wrong value for dynamic shape model

[gedoensmax]

Describe the issue

GetDimensionsCount() returns uninitialized number for a dynamic shape model.

terminate called after throwing an instance of 'std::length_error'
  what():  cannot create std::vector larger than max_size()
dimensions: 4303820303055852962

To reproduce

Compiling this against onnxruntime gives me the above error:

#include <onnxruntime_cxx_api.h>
#include <filesystem>
#include <iostream>

int main() {
  try {
    std::filesystem::path model_name = "conv.onnx";
//    auto allocator = Ort::AllocatorWithDefaultOptions();
    Ort::SessionOptions so;
    auto env = Ort::Env();
    Ort::Session session(env, model_name.c_str(), so);
    for (int input_idx = 0; input_idx < int(session.GetInputCount()); ++input_idx) {
      auto input_name = session.GetInputNameAllocated(input_idx, Ort::AllocatorWithDefaultOptions());
      auto tensor_info = session.GetInputTypeInfo(input_idx).GetTensorTypeAndShapeInfo();
      auto dim = tensor_info.GetDimensionsCount();
      std::cout << "dimensions: " << dim << std::endl;
      auto shape = tensor_info.GetShape();
    }
    return 0;
}

Urgency

No response

Platform

Linux

OS Version

22.04

ONNX Runtime Installation

Built from Source

ONNX Runtime Version or Commit ID

1.21+custom commits

ONNX Runtime API

C++

Architecture

X64

Execution Provider

TensorRT

Execution Provider Library Version

No response

[gedoensmax]

conv.onnx.zip

[chilo-ms]

i can repro from my side. Will investigate.

[chilo-ms]

Interestingly, if you rewrite your app code, it can run as expected without error.

      - auto tensor_info = session.GetInputTypeInfo(input_idx).GetTensorTypeAndShapeInfo();
      + auto type_info = session.GetInputTypeInfo(input_idx);
      + auto tensor_info = type_info.GetTensorTypeAndShapeInfo();
      auto dim = tensor_info.GetDimensionsCount();

[chilo-ms]

I investigated using gdb and it's value_ in TensorShape which is gsl::span that causing the issue ...

void TensorShape::Allocate(size_t size) {
  if (values_.size() == size)
    return;

  allocated_buffer_.reset();

  if (size > std::size(small_buffer_)) {
    allocated_buffer_ = std::make_unique<int64_t[]>(size);
    values_ = gsl::span<int64_t>(allocated_buffer_.get(), size);
  } else
    values_ = gsl::span<int64_t>(small_buffer_, size);
}

gdb log:

....
Thread 1 "TestOutTreeEp" hit Breakpoint 1, Ort::detail::TypeInfoImpl<OrtTypeInfo>::GetTensorTypeAndShapeInfo (this=0x7ffd25a87ee0) at /home/lochi/repos/ort8/include/onnxruntime/core/session/onnxruntime_cxx_inline.h:1255
1255    inline ConstTensorTypeAndShapeInfo TypeInfoImpl<T>::GetTensorTypeAndShapeInfo() const {
(gdb) l
1250      ThrowOnError(GetApi().GetDimensions(this->p_, out.data(), out.size()));
1251      return out;
1252    }
1253
1254    template <typename T>
1255    inline ConstTensorTypeAndShapeInfo TypeInfoImpl<T>::GetTensorTypeAndShapeInfo() const {
1256      const OrtTensorTypeAndShapeInfo* out;
1257      ThrowOnError(GetApi().CastTypeInfoToTensorInfo(this->p_, &out));
1258      return ConstTensorTypeAndShapeInfo{out};
1259    }
(gdb) p this->p_
$1 = (Ort::detail::Base<OrtTypeInfo>::contained_type *) 0x62595b4a7720
(gdb) p this->p_->shape
There is no member or method named shape.
(gdb) p this->p_->data
$2 = std::unique_ptr<OrtTensorTypeAndShapeInfo> = {
  get() = 0x62595b363290
}
(gdb) n
1257      ThrowOnError(GetApi().CastTypeInfoToTensorInfo(this->p_, &out));
(gdb) n
1258      return ConstTensorTypeAndShapeInfo{out};
(gdb) p out
$3 = (const OrtTensorTypeAndShapeInfo *) 0x62595b363290
(gdb) p out->shape
$4 = {
  values_ = {
    storage_ = {
      <gsl::details::extent_type<18446744073709551615>> = {
        size_ = 4
      },
      members of gsl::span<long, 18446744073709551615>::storage_type<gsl::details::extent_type<18446744073709551615> >:
      data_ = 0x62595b3632a8
    }
  },
  small_buffer_ = {-1, 48, 256, 256, 0},
  allocated_buffer_ = std::unique_ptr<long []> = {
    get() = 0x0
  }
}

At this point, calling GetTensorTypeAndShapeInfo(), you can see TensorShape.values_ has size of 4 and its underlying memory is TensorShape.values_.small_buffer

(gdb) c
Continuing.

Thread 1 "TestOutTreeEp" hit Breakpoint 2.1, Ort::detail::TensorTypeAndShapeInfoImpl<Ort::detail::Unowned<OrtTensorTypeAndShapeInfo const> >::GetDimensionsCount (this=0x7ffd25a87ec0) at /home/lochi/repos/ort8/include/onnxruntime/core/session/onnxruntime_cxx_inline.h:1231
1231    inline size_t TensorTypeAndShapeInfoImpl<T>::GetDimensionsCount() const {
(gdb) l
1226      ThrowOnError(GetApi().GetTensorShapeElementCount(this->p_, &out));
1227      return static_cast<size_t>(out);
1228    }
1229
1230    template <typename T>
1231    inline size_t TensorTypeAndShapeInfoImpl<T>::GetDimensionsCount() const {
1232      size_t out;
1233      ThrowOnError(GetApi().GetDimensionsCount(this->p_, &out));
1234      return out;
1235    }
(gdb) p this->p_
$5 = (Ort::detail::Base<Ort::detail::Unowned<OrtTensorTypeAndShapeInfo const> >::contained_type *) 0x62595b363290
(gdb) p this->p_->shape
$6 = {
  values_ = {
    storage_ = {
      <gsl::details::extent_type<18446744073709551615>> = {
        size_ = 14215538567980114220
      },
      members of gsl::span<long, 18446744073709551615>::storage_type<gsl::details::extent_type<18446744073709551615> >:
      data_ = 0x62595b3632a8
    }
  },
  small_buffer_ = {-1, 48, 256, 256, 0},
  allocated_buffer_ = std::unique_ptr<long []> = {
    get() = 0x0
  }
}

After calling GetDimensionsCount(), you can see that TensorShape.values_ has size of 14215538567980114220, which is wrong.....

[chilo-ms]

I tried both Debug and Release mode, they all have the same issue.
It seems to me with your original code that makes gsl::span and compiler generate wrong code/behavior?

[gedoensmax]

So you are saying that the reference counting or something in the vector is broken when using rvalue instead of lvalue ?

[github-actions]

This issue has been automatically marked as stale due to inactivity and will be closed in 30 days if no further activity occurs. If further support is needed, please provide an update and/or more details.

[snnn]

This issue has been automatically closed as 'not planned' because it has been marked as 'stale' for more than 30 days without activity. If you believe this is still an issue, please feel free to reopen it.

[hughsando]

session.GetOutputTypeInfo(output_idx) does indeed fix the problem. But I still think there is an issue when referencing the output[output_idx].GetTypeInfo() in the variable output size case. A static cast when there should be an api call?

[microsoft-github-policy-service]

Applying stale label due to no activity in 30 days