Importing Bidirectional LSTM model via ONNX shows an error

[sjeongab]

System information (version)

• OpenCV => 4.4
• Operating System / Platform => Mac OS 10.15
• Compiler => PyCharm

Detailed description

Importing model from easyocr(https://github.com/JaidedAI/EasyOCR/blob/master/easyocr/model.py) via ONNX shows an error. The model itself works well with torch and ONNX, but the error occurs when the ONNX version file is imported through openCV, and especially BiLSTM is the part where the error occurs.

Also reported to openCV forum: https://answers.opencv.org/question/233370/importing-bidirectional-lstm-model-via-onnx-shows-an-error/

Below is the error code:

error msg is cv2.error: OpenCV(4.4.0) ../modules/dnn/src/layers/permute_layer.cpp:134: error: (-215:Assertion failed) (int)_numAxes == inputs[0].size() in function 'getMemoryShapes'

Below is the simplified model I made which reproduces error:

Steps to reproduce

import torch.nn as nn
import torch


class BidirectionalLSTM(nn.Module):
    def __init__(self, input_size, hidden_size, output_size):
        super(BidirectionalLSTM, self).__init__()
        self.rnn = nn.LSTM(input_size, hidden_size, bidirectional=True, batch_first=True)
        self.linear = nn.Linear(hidden_size * 2, output_size)

    def forward(self, input):
        self.rnn.flatten_parameters()
        recurrent, _ = self.rnn(input)
        output = self.linear(recurrent)
        return output


class Model(nn.Module):
    def __init__(self):
        super(Model, self).__init__()
        self.SequenceModeling = nn.Sequential(
            BidirectionalLSTM(512, 512, 512),
            BidirectionalLSTM(512, 512, 512))

    def forward(self, input):
        contextual_feature = self.SequenceModeling(input)
        return contextual_feature


x = torch.rand(1, 65, 512).float().cpu()
model = Model()
model.eval()
torch.onnx.export(model, x, "ocr0811_1.onnx")


import cv2
net = cv2.dnn.readNetFromONNX('ocr0811_1.onnx')

Issue submission checklist

• I report the issue, it's not a question
• I checked the problem with documentation, FAQ, open issues,
  answers.opencv.org, Stack Overflow, etc and have not found solution
• I updated to latest OpenCV version and the issue is still there
• There is reproducer code and related data files: videos, images, onnx, etc

[zihaomu]

@sjeongab I encountered the same problem as you.
And in my case, the following BiLSTM layer definition can work in OpenCV:

import torch.nn as nn

class BidirectionalLSTM(nn.Module):

    def __init__(self, input_size, hidden_size, output_size):
        super(BidirectionalLSTM, self).__init__()
        self.rnn = nn.LSTM(input_size, hidden_size, bidirectional=True, batch_first=True)
        self.linear = nn.Linear(hidden_size * 2, output_size)

    def forward(self, input):
        """
        input : visual feature [batch_size x T x input_size]
        output : contextual feature [batch_size x T x output_size]
        """
        # self.rnn.flatten_parameters()
        recurrent, _ = self.rnn(input)  # batch_size x T x input_size -> batch_size x T x (2*hidden_size)
        b, T, h = recurrent.size()
        recurrent = recurrent.view(b*T, h)
        output = self.linear(recurrent)  # batch_size x T x output_size
        output = output.view(b, T, -1)
        return output

For more details, please refer: https://github.com/zihaomu/deep-text-recognition-benchmark/blob/master/modules/sequence_modeling.py

[dkurt]

please try #18078