Skip to content

sampler: pad dry sequence breakers tensor#875

Merged
AlpinDale merged 1 commit into
mainfrom
dry-seq-breakers-pad
Dec 8, 2024
Merged

sampler: pad dry sequence breakers tensor#875
AlpinDale merged 1 commit into
mainfrom
dry-seq-breakers-pad

Conversation

@AlpinDale

Copy link
Copy Markdown
Member

A potential fix for:

  File "/home/ubuntu/aphrodite-engine/aphrodite/modeling/layers/sampler.py", line 148, in forward
    self._init_sampling_tensors(logits, sampling_metadata)
  File "/home/ubuntu/aphrodite-engine/aphrodite/modeling/layers/sampler.py", line 112, in _init_sampling_tensors
    ) = SamplingTensors.from_sampling_metadata(
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/aphrodite-engine/aphrodite/modeling/sampling_metadata.py", line 600, in from_sampling_metadata
    sampling_tensors = SamplingTensors.from_lists(
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/aphrodite-engine/aphrodite/modeling/sampling_metadata.py", line 793, in from_lists
    dry_sequence_breakers_t = torch.tensor(
                              ^^^^^^^^^^^^^
ValueError: expected sequence of length 4 at dim 1 (got 5)

@AlpinDale

Copy link
Copy Markdown
Member Author

Seems to hold up in production.

@AlpinDale AlpinDale merged commit f0cc35b into main Dec 8, 2024
@AlpinDale AlpinDale deleted the dry-seq-breakers-pad branch March 12, 2025 14:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant