Adding multi_specializations_frames#909
Merged
Merged
Conversation
Contributor
|
@mohiso22 can you fix the quickcheck pls? |
vbaddi
reviewed
Apr 7, 2026
quic-rishinr
requested changes
Apr 8, 2026
|
|
||
| def _generate_multi_frame_specialization( | ||
| self, | ||
| inputs: List[str] = None, |
Contributor
There was a problem hiding this comment.
nit: update the type annotation for inputs as inputs: Dict[str, torch.Tensor]
4 tasks
quic-rishinr
requested changes
Apr 10, 2026
| } | ||
| ] | ||
| else: | ||
| assert vision_size * f < user_vision_size, ( |
Contributor
There was a problem hiding this comment.
nit: better to use exception instead of assert.
| grid_height = grid_height * time * batch_size | ||
| if not user_vision_size: | ||
| max_vision_size = max(max_vision_size, vision_size * f) | ||
| assert max_vision_size < ctx_len, ( |
Contributor
There was a problem hiding this comment.
nit: better to use exception instead of assert.
| return self._generate_regular_batching(vision_prompts, generation_len, stream, **kwargs) | ||
|
|
||
| def run_prefill_multi_frame_specialization( | ||
| self, inputs: Optional[torch.Tensor], num_frames: Optional[int] = 1, generation_len: int = None |
Contributor
There was a problem hiding this comment.
please add a doc string for the method
| self._session.deactivate() | ||
| self._vision_session.activate() | ||
|
|
||
| if not num_frames: |
Contributor
There was a problem hiding this comment.
nit: better to specific it as if num_frames ==0: num_frames = 1 and add a warning if that was the check. Since default value is set to 1 ideally we need not need this condition unless somewhere we are passing the value as none.
added 10 commits
April 10, 2026 16:44
Signed-off-by: Mohit Soni <mohisoni@qti.qualcomm.com>
Signed-off-by: Mohit Soni <mohisoni@qti.qualcomm.com>
Signed-off-by: Mohit Soni <mohisoni@qti.qualcomm.com>
Signed-off-by: Mohit Soni <mohisoni@qti.qualcomm.com>
Signed-off-by: Mohit Soni <mohisoni@qti.qualcomm.com>
Signed-off-by: Mohit Soni <mohisoni@qti.qualcomm.com>
Signed-off-by: Mohit Soni <mohisoni@qti.qualcomm.com>
Signed-off-by: Mohit Soni <mohisoni@qti.qualcomm.com>
Signed-off-by: Mohit Soni <mohisoni@qti.qualcomm.com>
quic-rishinr
force-pushed
the
qwen_multi
branch
from
ccaac2c to
d50cf6d
Compare
Contributor
|
Merging the PR on release/v1.21.6. @mohiso22 please raise a new PR on mainline with the couple of changes requested. |
quic-dhirajku
added a commit
to quic-dhirajku/efficient-transformers
that referenced
this pull request
Jun 2, 2026
…models as per reference from quic#909. Signed-off-by: Dhiraj Kumar Sah <dhirajku@qti.qualcomm.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds multi-specialization support for Qwen2.5-VL and Qwen3-VL, enabling the vision encoder to be compiled with multiple resolution configurations (height/width/num_frames as lists) in one shot.
Introduces a dedicated _generate_multi_frame_specialization inference path that selects the right specialization at runtime, along with example scripts for both model families.