transformers 4.21.3 doesn't support `load_in_8bit`

[SadeghRahnamoon]

Using transformers version 4.21.3, I see this error:

Using `load_in_8bit=True` to use quanitized model
Traceback (most recent call last):
 File "bloom-accelerate-inference.py", line 115, in <module>
   model = AutoModelForCausalLM.from_pretrained(model_name, **kwargs)
 File "/disk1/srahnamoon/llm/.lvenv/lib/python3.8/site-packages/transformers/models/auto/auto_factory.py", line 446, in from_pretrained
   return model_class.from_pretrained(pretrained_model_name_or_path, *model_args, config=config, **kwargs)
 File "/disk1/srahnamoon/llm/.lvenv/lib/python3.8/site-packages/transformers/modeling_utils.py", line 2113, in from_pretrained
   model = cls(config, *model_args, **model_kwargs)
TypeError: __init__() got an unexpected keyword argument 'load_in_8bit'

upgrading to 4.22.0 resolved the issue. Maybe the documentation for HF Accelerate solution needs to be corrected?

[mayank31398]

Yes, I am aware of this. I think we can leave it as it is.
The current release doesn't provide LLM.int8() support.
Its better to build transformers from source.
The next release should provide it :)