How to use language models such as LLaMA / Alpaca / Vicuna with BertViz?

[watashiwa-toki]

I'm try to do this in CoLab:

! pip install transformers
!pip install bertviz

from transformers import AutoModelWithLMHead, AutoTokenizer
model = AutoModelWithLMHead.from_pretrained("decapoda-research/llama-13b-hf")
tokenizer = AutoTokenizer.from_pretrained("decapoda-research/llama-13b-hf")

and get errors:

Downloading (…)lve/main/config.json: 100%
427/427 [00:00<00:00, 9.54kB/s]
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-7-71e58b40f790> in <cell line: 2>()
      1 from transformers import AutoModelWithLMHead, AutoTokenizer
----> 2 model = AutoModelWithLMHead.from_pretrained("decapoda-research/llama-13b-hf")
      3 tokenizer = AutoTokenizer.from_pretrained("decapoda-research/llama-13b-hf")

1 frames
/usr/local/lib/python3.10/dist-packages/transformers/models/auto/auto_factory.py in from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
    472                 pretrained_model_name_or_path, *model_args, config=config, **hub_kwargs, **kwargs
    473             )
--> 474         raise ValueError(
    475             f"Unrecognized configuration class {config.__class__} for this kind of AutoModel: {cls.__name__}.\n"
    476             f"Model type should be one of {', '.join(c.__name__ for c in cls._model_mapping.keys())}."

ValueError: Unrecognized configuration class <class 'transformers.models.llama.configuration_llama.LlamaConfig'> for this kind of AutoModel: AutoModelWithLMHead.
Model type should be one of AlbertConfig, BartConfig, BertConfig, BigBirdConfig, BigBirdPegasusConfig, BlenderbotSmallConfig, BloomConfig, CamembertConfig, CodeGenConfig, ConvBertConfig, CpmAntConfig, CTRLConfig, Data2VecTextConfig, DebertaConfig, DebertaV2Config, DistilBertConfig, ElectraConfig, EncoderDecoderConfig, ErnieConfig, EsmConfig, FlaubertConfig, FNetConfig, FSMTConfig, FunnelConfig, GitConfig, GPT2Config, GPT2Config, GPTBigCodeConfig, GPTNeoConfig, GPTNeoXConfig, GPTNeoXJapaneseConfig, GPTJConfig, GPTSanJapaneseConfig, IBertConfig, LayoutLMConfig, LEDConfig, LongformerConfig, LongT5Config, LukeConfig, M2M100Config, MarianConfig, MegaConfig, MegatronBertConfig, MobileBertConfig, MPNetConfig, MvpConfig, NezhaConfig, NllbMoeConfig, NystromformerConfig, OpenAIGPTConfig, PegasusXConfig, PLBartConfig, QDQBertConfig, ReformerConfig, RemBertConfig, RobertaConfig, RobertaPreLayerNormConfig, RoCBertConfig, RoFormerConfig, Speech2TextConfig, SqueezeBertConfig, SwitchTransformersConfig, T5Config, TapasConfig, TransfoXLConfig, Wav2Vec2Config, WhisperConfig, XLMConfig, XLMRobertaConfig, XLMRobertaXLConfig, XLNetConfig, XmodConfig, YosoConfig.

Is it possible at all? Or i simple do it wrong?

[SoyGema]

Hello there @watashiwa-toki !
Found your issue tangentially while digging into this project. :)
What I think is happening here is that that AutoModelWithLMHead doesn´t support the config

You can load this model using this code

from transformers import AutoTokenizer, AutoModelForCausalLM
tokenizer = AutoTokenizer.from_pretrained("decapoda-research/llama-13b-hf")
model = AutoModelForCausalLM.from_pretrained("decapoda-research/llama-13b-hf")

You can check it here , in the UI </> Use in transformers .
I quick check it and there seems to be an error in the LLamaTokenizer, as it should be LlamaTokenizer . See issue

Please note that all these things come more from the HugginFace Hub model owner management, and not from this project...However HF seem quite supportive, you might want to take it from here!

Interesting project this one, right ?
Have a nice day!
👍