Skip to content

Enable arch tests for Qwen3VL and Cohere2 in OpenVINO backend#180

Merged
zhaixuejun1993 merged 2 commits into
ravi9:dev_backend_openvinofrom
zhaixuejun1993:xuejun/arch-test-qw3vl-cohere2
May 25, 2026
Merged

Enable arch tests for Qwen3VL and Cohere2 in OpenVINO backend#180
zhaixuejun1993 merged 2 commits into
ravi9:dev_backend_openvinofrom
zhaixuejun1993:xuejun/arch-test-qw3vl-cohere2

Conversation

@zhaixuejun1993
Copy link
Copy Markdown
Collaborator

This pull request introduces improvements to the handling of dynamic sequence lengths in the ROPE (Rotary Positional Embedding) implementation within the OpenVINO integration. The changes ensure that sequence slicing is handled correctly per ROPE node, prevent unsafe sharing of positional encodings, and allow passing dynamic sequence lengths through the computation graph.

Dynamic sequence handling and ROPE improvements:

  • The make_sin_cos function now accepts an optional token_len_per_seq parameter, allowing dynamic slicing of input positions based on the active sequence length. The function applies slicing when token_len_per_seq is provided and the mode is not imrope. [1] [2] [3]
  • The translate_rope function is updated to detect and pass the token_len_per_seq input to make_sin_cos if available, enabling dynamic sequence length support in ROPE computations.
  • The computation of token_len_per_seq in GgmlOvDecoder::compute_llm_params is corrected to derive the value from the input sequence shape, ensuring accurate sequence length propagation.

Safety and correctness enhancements:

  • The add_rope_sin_cos function now avoids reusing shared ROPE sine/cosine values across the graph when dynamic active-sequence slicing is in use, preventing potential mismatches in positional encoding.## Overview

Additional information

Requirements

@zhaixuejun1993
Copy link
Copy Markdown
Collaborator Author

This ops test failed will disappear with OV release 2026.2
image

@zhaixuejun1993 zhaixuejun1993 merged commit da48690 into ravi9:dev_backend_openvino May 25, 2026
4 of 14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant