h2oGPT Client 401-Status caused by raising KeyError: 'dependencies' in client.py gradio_client v.0.6.1

[abuyusif01]

The error am getting.

  File "/home/abuyusif01/python3/lib/python3.11/site-packages/gradio_client/client.py", line 134, in __init__
    for fn_index, dependency in enumerate(self.config["dependencies"])
                                          ~~~~~~~~~~~^^^^^^^^^^^^^^^^
KeyError: 'dependencies'

I try debugging it on my own, here's the actual cause of the error

file name: ..../lib/python3.11/site-packages/gradio_client/client.py", line 134,

        self.upload_url = urllib.parse.urljoin(self.src, utils.UPLOAD_URL)
        self.reset_url = urllib.parse.urljoin(self.src, utils.RESET_URL)
        self.config = self._get_config()
        self.session_hash = str(uuid.uuid4())

        self.endpoints = [
            Endpoint(self, fn_index, dependency)
            for fn_index, dependency in enumerate(self.config["dependencies"])
        ]

the var self.config is set by calling this function: self._get_config()

    def _get_config(self) -> dict:
        r = requests.get(
            urllib.parse.urljoin(self.src, utils.CONFIG_URL), headers=self.headers
        )

        # 0 debugging skils
        print (r) # this will print 401
        if r.ok:
            return r.json()
        else:  # to support older versions of Gradio
            r = requests.get(self.src, headers=self.headers)
            # some basic regex to extract the config
            result = re.search(r"window.gradio_config = (.*?);[\s]*</script>", r.text)
            try:
                config = json.loads(result.group(1))  # type: ignore
            except AttributeError as ae:
                raise ValueError(
                    f"Could not get Gradio config from: {self.src}"
                ) from ae
            if "allow_flagging" in config:
                raise ValueError(
                    "Gradio 2.x is not supported by this client. Please upgrade your Gradio app to Gradio 3.x or higher."
                )
            return config

I know 401 is unauthorized. but am certain the access token is valid. and i set the --allow_api and --api_open to True in my docker run command. below is the command i use to start the container.

docker command

gcr.io/vorvan/h2oai/h2ogpt-runtime:latest /workspace/generate.py \
          --height=700 \
          --visible_h2ogpt_logo=False \
          --visible_h2ogpt_links=False \
          --visible_h2ogpt_qrcode=False \
          --visible_chatbot_label=False \
          --visible_system_tab=False \
          --visible_models_tab=False \
          --visible_expert_tab=False \
          --visible_login_tab=False \
          --actions_in_sidebar=True \
          --auth_access='closed' \
          --h2ogpt_api_keys="['xxxx-xxxxx-xxxxxx-xxxxx-xxxxx', 'xxx-xxx-xxxxx-xxxx-xxxx-xxxxx']" \
          --auth="[('test','xxx-xxx-xxxx-xxxxx-xxxxxx-xxxx')]" \
          --save_dir='/workspace/save/' \
          --use_gpu_id=False \
          --score_model=None \
          --prompt_type=mistral \
          --base_model=llama \
          --model_path_llama=https://huggingface.co/TheBloke/Mistral-7B-Instruct-v0.1-GGUF/resolve/main/mistral-7b-instruct-v0.1.Q4_K_M.gguf \
          --max_seq_len=4096 \
          --max_new_tokens=1024 \
          --max_max_new_tokens=2048 \
          --enable_ocr=True \
         --allow_api=True \
         --api_open=True \

Libs and system specs

Ubuntu 22.04.03
GPU: Nvidia Tesla T4

gradio-client related

v0.6.1 as required in the docs

$ pip show gradio-client
Name: gradio_client
Version: 0.6.1
Summary: Python library for easily interacting with trained machine learning models
Home-page: 
Author: 
Author-email: Abubakar Abid <team@gradio.app>, Ali Abid <team@gradio.app>, Ali Abdalla <team@gradio.app>, Dawood Khan <team@gradio.app>, Ahsen Khaliq <team@gradio.app>, Pete Allen <team@gradio.app>, Freddy Boulton <team@gradio.app>
License: 
Location: ......................................
Requires: fsspec, httpx, huggingface-hub, packaging, requests, typing-extensions, websockets
Required-by: h2ogpt-client

https://github.com/h2oai/h2ogpt/releases/download/client-nightly-2023-11-16/h2ogpt_client-0.1.0-py3-none-any.whl
nightly release h2ogpt-client:latest

$ pip how h20gpt-client
Name: h2ogpt-client
Version: 0.1.0
Summary: 
Home-page: 
Author: 
Author-email: 
License: 
Location: ........................................................
Requires: gradio-client
Required-by: 

misc

I found the following issues interesting.

gradio-app/gradio#3805
#946

[pseudotensor]

What are you running exactly as client, I cannot tell. E.g. are you are using plain gradio client to talk to h2oGPT? Or some other way? What are your commands?

[abuyusif01]

I cant bellieve i forgot to paste the code am using for the client.
below is what am using, as mention in the OpenAI client compliant section of client readme file.
https://github.com/h2oai/h2ogpt/blob/main/client/README.md#usage

def test_readme_example(local_server):
    import os
    import asyncio
    from h2ogpt_client import Client

    if local_server:
        client = Client("http://0.0.0.0:7860")
    else:
        h2ogpt_key = os.getenv("H2OGPT_KEY") or os.getenv("H2OGPT_H2OGPT_KEY")
        # this actually prints the h20gpt_key that i set with --h20gpt_keys=[....] :param
        print (h20gpt_key) 
        if h2ogpt_key is None:
            return
        # if you have API key for public instance:
        client = Client("http://xx.xxxx.xxxxx.xx:7860", h2ogpt_key=h2ogpt_key)

    # Text completion
    text_completion = client.text_completion.create()
    response = asyncio.run(text_completion.complete("Hello world"))
    print("asyncio text completion response: %s" % response)
    # Text completion: synchronous
    response = text_completion.complete_sync("Hello world")
    print("sync text completion response: %s" % response)

    # Chat completion
    chat_completion = client.chat_completion.create()
    reply = asyncio.run(chat_completion.chat("Hey!"))
    print("asyncio text completion user: %s gpt: %s" % (reply["user"], reply["gpt"]))
    chat_history = chat_completion.chat_history()
    print("chat_history: %s" % chat_history)
    # Chat completion: synchronous
    reply = chat_completion.chat_sync("Hey!")
    print("sync chat completion gpt: %s" % reply["gpt"])

test_readme_example(local_server=False)

[abuyusif01]

@pseudotensor

Can u please look at the above code i pasted, and lmk if u know the causes of this issue. Thanks

[pseudotensor]

Sorry for not responding, I don't have any good ideas. @this maintains the OpenAI compliant API you are trying. Maybe he has ideas.

[sffranke]

Its the --auth=... what causes the error.

[pseudotensor]

gradio 3 doesn't support auth via API.