Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix ONNX parser for single-layer LSTM hidden and cell states #23475

Merged
merged 6 commits into from
Apr 24, 2023

Conversation

Abdurrahheem
Copy link
Contributor

@Abdurrahheem Abdurrahheem commented Apr 9, 2023

Fix ONNX parser for single-layer LSTM hidden and cell states

Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

  • I agree to contribute to the project under Apache 2 License.
  • To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
  • The PR is proposed to the proper branch
  • There is a reference to the original bug report and related work
  • There is accuracy test, performance test and test data in opencv_extra repository, if applicable
    Patch to opencv_extra has the same branch name.
  • The feature is well documented and sample code can be built with the project CMake

This PR addresses #21118 issue. The problem is that the ONNX parser is unable to read the hidden state and cell state for single-layer LSTMs. This PR fixes the issue by updating the parser to correctly read hidden and cell states.

modules/dnn/src/onnx/onnx_importer.cpp Outdated Show resolved Hide resolved
modules/dnn/src/onnx/onnx_importer.cpp Outdated Show resolved Hide resolved
@asmorkalov asmorkalov added bug category: dnn (onnx) ONNX suport issues in DNN module labels Apr 10, 2023
@asmorkalov asmorkalov added this to the 4.8.0 milestone Apr 10, 2023
@fengyuentau
Copy link
Member

Not related to this PR but I see in some cases our ONNX parser inserts several layers to properly execute LSTM layer. Could it be fixed so that we do not need those extra layers?

@asmorkalov asmorkalov marked this pull request as ready for review April 13, 2023 14:24
Copy link
Member

@rogday rogday left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, with minor comments.

modules/dnn/src/layers/recurrent_layers.cpp Outdated Show resolved Hide resolved
modules/dnn/src/onnx/onnx_importer.cpp Outdated Show resolved Hide resolved
@asmorkalov asmorkalov merged commit e4e774d into opencv:4.x Apr 24, 2023
18 checks passed
@rogday
Copy link
Member

rogday commented Apr 25, 2023

We had a branch with test for this pr, didn't we?

@opencv-alalek
Copy link
Contributor

There is even PR with testdata: opencv/opencv_extra#1052

@rogday
Copy link
Member

rogday commented Apr 25, 2023

@Abdurrahheem, can you please add tests for new functionality?

@rogday
Copy link
Member

rogday commented Apr 25, 2023

@asmorkalov

@asmorkalov asmorkalov mentioned this pull request May 31, 2023
thewoz pushed a commit to thewoz/opencv that referenced this pull request Jan 4, 2024
…tion

Fix ONNX parser for single-layer LSTM hidden and cell states opencv#23475

### Fix ONNX parser for single-layer LSTM hidden and cell states

### Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake


This PR addresses opencv#21118 [issue](opencv#21118). The problem is that the ONNX parser is unable to read the hidden state and cell state for single-layer LSTMs. This PR fixes the issue by updating the parser to correctly read hidden and cell states.
thewoz pushed a commit to thewoz/opencv that referenced this pull request May 29, 2024
…tion

Fix ONNX parser for single-layer LSTM hidden and cell states opencv#23475

### Fix ONNX parser for single-layer LSTM hidden and cell states

### Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake


This PR addresses opencv#21118 [issue](opencv#21118). The problem is that the ONNX parser is unable to read the hidden state and cell state for single-layer LSTMs. This PR fixes the issue by updating the parser to correctly read hidden and cell states.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug category: dnn (onnx) ONNX suport issues in DNN module
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants