Clash in requirements for finetuning Starcoder2 #12
Description
#Facing the following error while trying to finetune Starcoder2 with the given script.
Description:
For transformers.AutoModelForCausalLM to recognize Starcoder2 transformers>4.39.0 is required.
But trl is still using transformers==4.38.2. Even if I compile from source & use trl=0.7.12.dev0 I still get an issue.
Here is the error with using transformers==4.38.2
KeyError Traceback (most recent call last)
[/usr/local/lib/python3.10/dist-packages/transformers/models/auto/configuration_auto.py](https://localhost:8080/#) in from_pretrained(cls, pretrained_model_name_or_path, **kwargs)
1127 try:
-> 1128 config_class = CONFIG_MAPPING[config_dict["model_type"]]
1129 except KeyError:
4 frames
KeyError: 'starcoder2'
During handling of the above exception, another exception occurred:
ValueError Traceback (most recent call last)
[/usr/local/lib/python3.10/dist-packages/transformers/models/auto/configuration_auto.py](https://localhost:8080/#) in from_pretrained(cls, pretrained_model_name_or_path, **kwargs)
1128 config_class = CONFIG_MAPPING[config_dict["model_type"]]
1129 except KeyError:
-> 1130 raise ValueError(
1131 f"The checkpoint you are trying to load has model type `{config_dict['model_type']}` "
1132 "but Transformers does not recognize this architecture. This could be because of an "
ValueError: The checkpoint you are trying to load has model type `starcoder2` but Transformers does not recognize this architecture. This could be because of an issue with the checkpoint, or because your version of Transformers is out of date.
Here is the error when using transformers==4.39.0
ImportError Traceback (most recent call last)
[<ipython-input-2-3ef713ffd06d>](https://localhost:8080/#) in <cell line: 1>()
----> 1 from trl import SFTTrainer
2 print("trl version:", trl.__version__)
1 frames
[/usr/local/lib/python3.10/dist-packages/trl/__init__.py](https://localhost:8080/#) in <module>
3 __version__ = "0.7.12.dev0"
4
----> 5 from .core import set_seed
6 from .environment import TextEnvironment, TextHistory
7 from .extras import BestOfNSampler
[/usr/local/lib/python3.10/dist-packages/trl/core.py](https://localhost:8080/#) in <module>
23 import torch.nn.functional as F
24 from torch.nn.utils.rnn import pad_sequence
---> 25 from transformers import top_k_top_p_filtering
26
27 from .import_utils import is_npu_available, is_xpu_available
ImportError: cannot import name 'top_k_top_p_filtering' from 'transformers' (/usr/local/lib/python3.10/dist-packages/transformers/__init__.py)
---------------------------------------------------------------------------
NOTE: If your import is failing due to a missing package, you can
manually install dependencies using either !pip or !apt.
To view examples of installing some common dependencies, click the
"Open Examples" button below.
---------------------------------------------------------------------------