Duplicate tokenizer entry in config.json
auto_map
section
#544
Labels
type/question
An issue that's a question
config.json
auto_map
section
#544
❓ The question
The Hugging Face config.json files for the olmo models (and for the tokenizers) have the following repeated line under the
auto_map
AutoTokenizer
key:(e.g. link for olmo 7b config.json).
Couple of questions:
This currently causes an
AttributeError: 'list' object has no attribute 'split'
error when trying to load the (peft) model with MLflow, which does not expect lists withinauto_map
. If it is expected that theAutoTokenizer
entry is an array, I can pursue this from the MLflow side.Thanks for looking!
The text was updated successfully, but these errors were encountered: