This repository was archived by the owner on Jun 3, 2025. It is now read-only.
[Fix] Inferring a default sequence length during transformers export #1856
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR: #1826 has added new functionality so that when no
sequence_lengthis being specified, we are defaulting to a max sequence length from the config. Unfortunately, as discussed in the PR comments of1826, indeed - not only an HF config is not guaranteed to havemax_position_embeddingsattribute, but this information can be present under a different key value.E.g
For the model
TinyLlama-1.1B-Chat-v0.3we are looking atconfig.max_position_embeddingsFor the model
zoo:mpt-7b-mpt_chat_mpt_pretrain-pruned80_quantizedwe are looking atconfig.max_seq_lenThis PR:
max_seq_lento the set of config attributes that may be potentially used to infer the default sequence_length