Auth token is not passable to load_textual_inversion due to named argument conflicting with accepted kwargs

[Teriks]

Describe the bug

You cannot provide an auth token value to the **kwargs of load_textual_inversion in src/diffusers/loaders/textual_inversion.py due to it being a named argument used for another purpose.

It seems this is due to a refactor at some point.

Since the named argument token is already used, you cannot pass an authentication key to this function.

The doc string has two entries for this argument:

token (`str` or *bool*, *optional*):
    The token to use as HTTP bearer authorization for remote files. If `True`, the token generated from
    `diffusers-cli login` (stored in `~/.huggingface`) is used.

token (`str` or `List[str]`, *optional*):
    Override the token to use for the textual inversion weights. If `pretrained_model_name_or_path` is a
    list, then `token` must also be a list of equal length.

Reproduction

N/A

Logs

No response

System Info

diffusers 0.27.2

Who can help?

No response

[yiyixuxu]

Can you provide a reproducible script?

[Teriks]

See also this line

diffusers/src/diffusers/loaders/textual_inversion.py

Line 44 in cf6e040

token = kwargs.pop("token", None)

Here is code which tries to describe the problem

from diffusers import StableDiffusionPipeline
import torch

model_id = "runwayml/stable-diffusion-v1-5"
pipe = StableDiffusionPipeline.from_pretrained(model_id, torch_dtype=torch.float16).to("cuda")

# How can you pass a User Auth Token to load_textual_inversion for private repositories?

# once upon a time, **kwargs accepted the argument "use_auth_token"
# which was refactored to "token" within load_textual_inversion_state_dicts
# and elsewhere within the library

# see: diffusers\loaders\textual_inversion.py, line: 43

# now an auth token cannot be passed, only the "token" that
# is used to specify a textual inversion from the prompt can be passed

# it is not possible to manually specify credentials, however the environmental
# variable HF_TOKEN is *probably* still respected

# The documentation of this function erroneously contains two references 
# to the parameter "token" one being for the purpose of specifying the textual
# inversion token, and the other being for specifying an auth token

# which is not possible, as both arguments have the same name, and the
# prompt token takes priority due to being a named argument

pipe.load_textual_inversion("sd-concepts-library/cat-toy",
                            token="not-an-auth-token")

prompt = "A <not-an-auth-token> backpack"

image = pipe(prompt, num_inference_steps=50).images[0]
image.save("cat-backpack.png")

Potential workaround:

import os

from diffusers import StableDiffusionPipeline
import torch

model_id = "runwayml/stable-diffusion-v1-5"
pipe = StableDiffusionPipeline.from_pretrained(model_id, torch_dtype=torch.float16).to("cuda")

# set an HF_TOKEN value here that may differ from what is in the environment initially
use_auth_token = None

old_token = os.environ.get('HF_TOKEN', None)
if use_auth_token is not None:
    os.environ['HF_TOKEN'] = use_auth_token
try:
    pipe.load_textual_inversion("sd-concepts-library/cat-toy",
                                token="not-an-auth-token")
finally:
    if old_token is not None:
        os.environ['HF_TOKEN'] = old_token


prompt = "A <not-an-auth-token> backpack"

image = pipe(prompt, num_inference_steps=50).images[0]
image.save("cat-backpack.png")

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

[a-r-r-o-w]

Hi. Really sorry for the delay here! Is this still an issue? If yes, please gently ping DN6 and sayakpaul to help resolve this asap

[Teriks]

I have compared the revision on which I reported this issue cf6e040 to the current main branch and this issue still exists @DN6 @sayakpaul

[sayakpaul]

Yes, you're right here. But I am not sure if it was ever correct i.e., the existence of two token arguments. Gently pinging @yiyixuxu for advice here.

Some ideas that come to mind:

1. Detect if both token and kwargs["token"] are present and error out with a sensible message.
2. If 1. is detected, instruct the users to pass something like hf_token when loading private checkpoints and we make the changes necessary in the code.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.