Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

QUESTION: xinference registrations command error: requests.exceptions.JSONDecodeError: Expecting value: line 1 column 1 (char 0) #1147

Closed
BaiMeiyingxue opened this issue Mar 14, 2024 · 12 comments
Closed

QUESTION: xinference registrations command error: requests.exceptions.JSONDecodeError: Expecting value: line 1 column 1 (char 0) #1147

BaiMeiyingxue opened this issue Mar 14, 2024 · 12 comments
Labels
question Further information is requested stale
Milestone
v0.14

Comments

@BaiMeiyingxue
Copy link

the version is xinference 0.9.2 ,when inspect the registered LLM using 'xinference registrations --model-type LLM', an error occurred

  File "/home/ubuntu/anaconda3/envs/xinference092/bin/xinference", line 8, in <module>
    sys.exit(cli())
  File "/home/ubuntu/anaconda3/envs/xinference092/lib/python3.10/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
  File "/home/ubuntu/anaconda3/envs/xinference092/lib/python3.10/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
  File "/home/ubuntu/anaconda3/envs/xinference092/lib/python3.10/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/ubuntu/anaconda3/envs/xinference092/lib/python3.10/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/ubuntu/anaconda3/envs/xinference092/lib/python3.10/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
  File "/home/ubuntu/anaconda3/envs/xinference092/lib/python3.10/site-packages/xinference/deploy/cmdline.py", line 447, in list_model_registrations
    client = RESTfulClient(base_url=endpoint)
  File "/home/ubuntu/anaconda3/envs/xinference092/lib/python3.10/site-packages/xinference/client/restful/restful_client.py", line 658, in __init__
    self._check_cluster_authenticated()
  File "/home/ubuntu/anaconda3/envs/xinference092/lib/python3.10/site-packages/xinference/client/restful/restful_client.py", line 681, in _check_cluster_authenticated
    f"Failed to get cluster information, detail: {response.json()['detail']}"
  File "/home/ubuntu/anaconda3/envs/xinference092/lib/python3.10/site-packages/requests/models.py", line 975, in json
    raise RequestsJSONDecodeError(e.msg, e.doc, e.pos)
requests.exceptions.JSONDecodeError: Expecting value: line 1 column 1 (char 0)```
@BaiMeiyingxue BaiMeiyingxue added the question Further information is requested label Mar 14, 2024
@XprobeBot XprobeBot added this to the v0.9.3 milestone Mar 14, 2024
@BaiMeiyingxue
Copy link
Author

the version of package is

----------------------------- ------------
accelerate                    0.28.0
addict                        2.4.0
aiobotocore                   2.7.0
aiofiles                      23.2.1
aiohttp                       3.9.3
aioitertools                  0.11.0
aioprometheus                 23.12.0
aiosignal                     1.3.1
aliyun-python-sdk-core        2.15.0
aliyun-python-sdk-kms         2.16.2
altair                        5.2.0
annotated-types               0.6.0
anyio                         4.3.0
async-timeout                 4.0.3
attrs                         23.2.0
auto_gptq                     0.7.1
bcrypt                        4.1.2
bitsandbytes                  0.43.0
botocore                      1.31.64
certifi                       2024.2.2
cffi                          1.16.0
charset-normalizer            3.3.2
chatglm-cpp                   0.3.1
click                         8.1.7
cloudpickle                   3.0.0
colorama                      0.4.6
coloredlogs                   15.0.1
contourpy                     1.2.0
controlnet-aux                0.0.7
crcmod                        1.7
cryptography                  42.0.5
cycler                        0.12.1
datasets                      2.18.0
diffusers                     0.26.3
dill                          0.3.8
diskcache                     5.6.3
distro                        1.9.0
ecdsa                         0.18.0
einops                        0.7.0
exceptiongroup                1.2.0
fastapi                       0.110.0
ffmpy                         0.3.2
filelock                      3.13.1
fonttools                     4.49.0
frozenlist                    1.4.1
fsspec                        2023.10.0
gast                          0.5.4
gekko                         1.0.7
gradio                        4.21.0
gradio_client                 0.12.0
h11                           0.14.0
httpcore                      1.0.4
httptools                     0.6.1
httpx                         0.27.0
huggingface-hub               0.21.4
humanfriendly                 10.0
idna                          3.6
imageio                       2.34.0
importlib_metadata            7.0.2
importlib_resources           6.3.0
Jinja2                        3.1.3
jmespath                      0.10.0
joblib                        1.3.2
jsonschema                    4.21.1
jsonschema-specifications     2023.12.1
kiwisolver                    1.4.5
lazy_loader                   0.3
llama_cpp_python              0.2.56
markdown-it-py                3.0.0
MarkupSafe                    2.1.5
matplotlib                    3.8.3
mdurl                         0.1.2
modelscope                    1.13.1
mpmath                        1.3.0
msgpack                       1.0.8
multidict                     6.0.5
multiprocess                  0.70.16
networkx                      3.2.1
ninja                         1.11.1.1
numpy                         1.26.4
nvidia-cublas-cu12            12.1.3.1
nvidia-cuda-cupti-cu12        12.1.105
nvidia-cuda-nvrtc-cu12        12.1.105
nvidia-cuda-runtime-cu12      12.1.105
nvidia-cudnn-cu12             8.9.2.26
nvidia-cufft-cu12             11.0.2.54
nvidia-curand-cu12            10.3.2.106
nvidia-cusolver-cu12          11.4.5.107
nvidia-cusparse-cu12          12.1.0.106
nvidia-nccl-cu12              2.18.1
nvidia-nvjitlink-cu12         12.4.99
nvidia-nvtx-cu12              12.1.105
openai                        1.13.3
opencv-python                 4.9.0.80
optimum                       1.17.1
orjson                        3.9.15
oss2                          2.18.4
packaging                     24.0
pandas                        2.2.1
passlib                       1.7.4
peft                          0.9.0
pillow                        10.2.0
pip                           23.3.1
platformdirs                  4.2.0
protobuf                      4.25.3
psutil                        5.9.8
pyarrow                       15.0.1
pyarrow-hotfix                0.6
pyasn1                        0.5.1
pycparser                     2.21
pycryptodome                  3.20.0
pydantic                      2.6.4
pydantic_core                 2.16.3
pydub                         0.25.1
Pygments                      2.17.2
pynvml                        11.5.0
pyparsing                     3.1.2
python-dateutil               2.9.0.post0
python-dotenv                 1.0.1
python-jose                   3.3.0
python-multipart              0.0.9
pytz                          2024.1
PyYAML                        6.0.1
quantile-python               1.1
ray                           2.9.3
referencing                   0.33.0
regex                         2023.12.25
requests                      2.31.0
rich                          13.7.1
rouge                         1.0.1
rpds-py                       0.18.0
rsa                           4.9
ruff                          0.3.2
s3fs                          2023.10.0
safetensors                   0.4.2
scikit-image                  0.22.0
scikit-learn                  1.4.1.post1
scipy                         1.12.0
semantic-version              2.10.0
sentence-transformers         2.5.1
sentencepiece                 0.2.0
setuptools                    68.2.2
shellingham                   1.5.4
simplejson                    3.19.2
six                           1.16.0
sniffio                       1.3.1
sortedcontainers              2.4.0
sse-starlette                 2.0.0
starlette                     0.36.3
sympy                         1.12
tabulate                      0.9.0
tblib                         3.0.0
threadpoolctl                 3.3.0
tifffile                      2024.2.12
tiktoken                      0.6.0
timm                          0.9.16
tokenizers                    0.15.2
tomli                         2.0.1
tomlkit                       0.12.0
toolz                         0.12.1
torch                         2.1.2
torchvision                   0.16.2
tqdm                          4.66.2
transformers                  4.38.2
transformers-stream-generator 0.0.5
triton                        2.1.0
typer                         0.9.0
typing_extensions             4.10.0
tzdata                        2024.1
urllib3                       2.0.7
uvicorn                       0.28.0
uvloop                        0.19.0
vllm                          0.3.0
watchfiles                    0.21.0
websockets                    11.0.3
wheel                         0.41.2
wrapt                         1.16.0
xformers                      0.0.23.post1
xinference                    0.9.2
xoscar                        0.2.1
xxhash                        3.4.1
yapf                          0.40.2
yarl                          1.9.4
zipp                          3.18.0```

@BaiMeiyingxue BaiMeiyingxue changed the title QUESTION requests.exceptions.JSONDecodeError: Expecting value: line 1 column 1 (char 0) Mar 14, 2024
@ChengjieLi28
Copy link
Contributor

the version is xinference 0.9.2 ,when inspect the registered LLM using 'xinference registrations --model-type LLM', an error occurred

  File "/home/ubuntu/anaconda3/envs/xinference092/bin/xinference", line 8, in <module>
    sys.exit(cli())
  File "/home/ubuntu/anaconda3/envs/xinference092/lib/python3.10/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
  File "/home/ubuntu/anaconda3/envs/xinference092/lib/python3.10/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
  File "/home/ubuntu/anaconda3/envs/xinference092/lib/python3.10/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/ubuntu/anaconda3/envs/xinference092/lib/python3.10/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/ubuntu/anaconda3/envs/xinference092/lib/python3.10/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
  File "/home/ubuntu/anaconda3/envs/xinference092/lib/python3.10/site-packages/xinference/deploy/cmdline.py", line 447, in list_model_registrations
    client = RESTfulClient(base_url=endpoint)
  File "/home/ubuntu/anaconda3/envs/xinference092/lib/python3.10/site-packages/xinference/client/restful/restful_client.py", line 658, in __init__
    self._check_cluster_authenticated()
  File "/home/ubuntu/anaconda3/envs/xinference092/lib/python3.10/site-packages/xinference/client/restful/restful_client.py", line 681, in _check_cluster_authenticated
    f"Failed to get cluster information, detail: {response.json()['detail']}"
  File "/home/ubuntu/anaconda3/envs/xinference092/lib/python3.10/site-packages/requests/models.py", line 975, in json
    raise RequestsJSONDecodeError(e.msg, e.doc, e.pos)
requests.exceptions.JSONDecodeError: Expecting value: line 1 column 1 (char 0)```

Does the xinference start normally on port 9997?

@ChengjieLi28 ChengjieLi28 changed the title requests.exceptions.JSONDecodeError: Expecting value: line 1 column 1 (char 0) QUESTION: requests.exceptions.JSONDecodeError: Expecting value: line 1 column 1 (char 0) Mar 14, 2024
@BaiMeiyingxue
Copy link
Author

the version is xinference 0.9.2 ,when inspect the registered LLM using 'xinference registrations --model-type LLM', an error occurred

  File "/home/ubuntu/anaconda3/envs/xinference092/bin/xinference", line 8, in <module>
    sys.exit(cli())
  File "/home/ubuntu/anaconda3/envs/xinference092/lib/python3.10/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
  File "/home/ubuntu/anaconda3/envs/xinference092/lib/python3.10/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
  File "/home/ubuntu/anaconda3/envs/xinference092/lib/python3.10/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/ubuntu/anaconda3/envs/xinference092/lib/python3.10/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/ubuntu/anaconda3/envs/xinference092/lib/python3.10/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
  File "/home/ubuntu/anaconda3/envs/xinference092/lib/python3.10/site-packages/xinference/deploy/cmdline.py", line 447, in list_model_registrations
    client = RESTfulClient(base_url=endpoint)
  File "/home/ubuntu/anaconda3/envs/xinference092/lib/python3.10/site-packages/xinference/client/restful/restful_client.py", line 658, in __init__
    self._check_cluster_authenticated()
  File "/home/ubuntu/anaconda3/envs/xinference092/lib/python3.10/site-packages/xinference/client/restful/restful_client.py", line 681, in _check_cluster_authenticated
    f"Failed to get cluster information, detail: {response.json()['detail']}"
  File "/home/ubuntu/anaconda3/envs/xinference092/lib/python3.10/site-packages/requests/models.py", line 975, in json
    raise RequestsJSONDecodeError(e.msg, e.doc, e.pos)
requests.exceptions.JSONDecodeError: Expecting value: line 1 column 1 (char 0)```

Does the xinference start normally on port 9997?

2024-03-14 14:19:56,218 xinference.core.supervisor 2424703 INFO     Xinference supervisor 0.0.0.0:55416 started
2024-03-14 14:19:57,414 xinference.core.worker 2424703 INFO     Starting metrics export server at 0.0.0.0:None
2024-03-14 14:19:57,418 xinference.core.worker 2424703 INFO     Checking metrics export server...
2024-03-14 14:19:59,950 xinference.core.worker 2424703 INFO     Metrics server is started at: http://0.0.0.0:34957
2024-03-14 14:19:59,951 xinference.core.worker 2424703 INFO     Xinference worker 0.0.0.0:55416 started
2024-03-14 14:19:59,952 xinference.core.worker 2424703 INFO     Purge cache directory: /home/ubuntu/.xinference/cache
2024-03-14 14:20:02,775 xinference.api.restful_api 2424453 INFO     Starting Xinference at endpoint: http://0.0.0.0:9997```
everything seems ok

@ChengjieLi28 ChengjieLi28 changed the title QUESTION: requests.exceptions.JSONDecodeError: Expecting value: line 1 column 1 (char 0) QUESTION: xinference registrations command error: requests.exceptions.JSONDecodeError: Expecting value: line 1 column 1 (char 0) Mar 14, 2024
@ChengjieLi28
Copy link
Contributor

@BaiMeiyingxue Cannot reproduce this situation on my ubuntu machine.
Could you please paste your command that is used for starting xinference?

@BaiMeiyingxue
Copy link
Author

@BaiMeiyingxue Cannot reproduce this situation on my ubuntu machine. Could you please paste your command that is used for starting xinference?

@BaiMeiyingxue Cannot reproduce this situation on my ubuntu machine. Could you please paste your command that is used for starting xinference?

(xinference) (base) ubuntu@ubuntu-EG540-G30:~$ xinference-local --host 0.0.0.0 --port 9997
2024-03-14 14:19:56,218 xinference.core.supervisor 2424703 INFO     Xinference supervisor 0.0.0.0:55416 started
2024-03-14 14:19:57,414 xinference.core.worker 2424703 INFO     Starting metrics export server at 0.0.0.0:None
2024-03-14 14:19:57,418 xinference.core.worker 2424703 INFO     Checking metrics export server...
2024-03-14 14:19:59,950 xinference.core.worker 2424703 INFO     Metrics server is started at: http://0.0.0.0:34957
2024-03-14 14:19:59,951 xinference.core.worker 2424703 INFO     Xinference worker 0.0.0.0:55416 started
2024-03-14 14:19:59,952 xinference.core.worker 2424703 INFO     Purge cache directory: /home/ubuntu/.xinference/cache
2024-03-14 14:20:02,775 xinference.api.restful_api 2424453 INFO     Starting Xinference at endpoint: http://0.0.0.0:9997

@ChengjieLi28
Copy link
Contributor

@BaiMeiyingxue Cannot reproduce this situation on my ubuntu machine. Could you please paste your command that is used for starting xinference?

@BaiMeiyingxue Cannot reproduce this situation on my ubuntu machine. Could you please paste your command that is used for starting xinference?

(xinference) (base) ubuntu@ubuntu-EG540-G30:~$ xinference-local --host 0.0.0.0 --port 9997
2024-03-14 14:19:56,218 xinference.core.supervisor 2424703 INFO     Xinference supervisor 0.0.0.0:55416 started
2024-03-14 14:19:57,414 xinference.core.worker 2424703 INFO     Starting metrics export server at 0.0.0.0:None
2024-03-14 14:19:57,418 xinference.core.worker 2424703 INFO     Checking metrics export server...
2024-03-14 14:19:59,950 xinference.core.worker 2424703 INFO     Metrics server is started at: http://0.0.0.0:34957
2024-03-14 14:19:59,951 xinference.core.worker 2424703 INFO     Xinference worker 0.0.0.0:55416 started
2024-03-14 14:19:59,952 xinference.core.worker 2424703 INFO     Purge cache directory: /home/ubuntu/.xinference/cache
2024-03-14 14:20:02,775 xinference.api.restful_api 2424453 INFO     Starting Xinference at endpoint: http://0.0.0.0:9997

Have you tried other commands? Do all commands run with this problem?

@BaiMeiyingxue
Copy link
Author

@BaiMeiyingxue Cannot reproduce this situation on my ubuntu machine. Could you please paste your command that is used for starting xinference?

@BaiMeiyingxue Cannot reproduce this situation on my ubuntu machine. Could you please paste your command that is used for starting xinference?

(xinference) (base) ubuntu@ubuntu-EG540-G30:~$ xinference-local --host 0.0.0.0 --port 9997
2024-03-14 14:19:56,218 xinference.core.supervisor 2424703 INFO     Xinference supervisor 0.0.0.0:55416 started
2024-03-14 14:19:57,414 xinference.core.worker 2424703 INFO     Starting metrics export server at 0.0.0.0:None
2024-03-14 14:19:57,418 xinference.core.worker 2424703 INFO     Checking metrics export server...
2024-03-14 14:19:59,950 xinference.core.worker 2424703 INFO     Metrics server is started at: http://0.0.0.0:34957
2024-03-14 14:19:59,951 xinference.core.worker 2424703 INFO     Xinference worker 0.0.0.0:55416 started
2024-03-14 14:19:59,952 xinference.core.worker 2424703 INFO     Purge cache directory: /home/ubuntu/.xinference/cache
2024-03-14 14:20:02,775 xinference.api.restful_api 2424453 INFO     Starting Xinference at endpoint: http://0.0.0.0:9997

Have you tried other commands? Do all commands run with this pr

@BaiMeiyingxue Cannot reproduce this situation on my ubuntu machine. Could you please paste your command that is used for starting xinference?

@BaiMeiyingxue Cannot reproduce this situation on my ubuntu machine. Could you please paste your command that is used for starting xinference?

(xinference) (base) ubuntu@ubuntu-EG540-G30:~$ xinference-local --host 0.0.0.0 --port 9997
2024-03-14 14:19:56,218 xinference.core.supervisor 2424703 INFO     Xinference supervisor 0.0.0.0:55416 started
2024-03-14 14:19:57,414 xinference.core.worker 2424703 INFO     Starting metrics export server at 0.0.0.0:None
2024-03-14 14:19:57,418 xinference.core.worker 2424703 INFO     Checking metrics export server...
2024-03-14 14:19:59,950 xinference.core.worker 2424703 INFO     Metrics server is started at: http://0.0.0.0:34957
2024-03-14 14:19:59,951 xinference.core.worker 2424703 INFO     Xinference worker 0.0.0.0:55416 started
2024-03-14 14:19:59,952 xinference.core.worker 2424703 INFO     Purge cache directory: /home/ubuntu/.xinference/cache
2024-03-14 14:20:02,775 xinference.api.restful_api 2424453 INFO     Starting Xinference at endpoint: http://0.0.0.0:9997

Have you tried other commands? Do all commands run with this problem?

yes, I tried to register a model with commands, and the seem error occured

@XprobeBot XprobeBot modified the milestones: v0.9.3, v0.9.4, v0.9.5 Mar 15, 2024
@YoungerXu
Copy link

If you have enabled a proxy, please try disabling it and then attempt the operation again. This maybe resolve the issue you encountered.

@XprobeBot XprobeBot modified the milestones: v0.10.0, v0.10.1 Mar 29, 2024
@XprobeBot XprobeBot modified the milestones: v0.10.1, v0.10.2 Apr 12, 2024
@XprobeBot XprobeBot modified the milestones: v0.10.2, v0.10.3, v0.11.0 Apr 19, 2024
@dongrixinyu
Copy link

have encountered the same bug.

@XprobeBot XprobeBot modified the milestones: v0.11.0, v0.11.1, v0.11.2 May 11, 2024
@XprobeBot XprobeBot modified the milestones: v0.11.2, v0.11.3 May 24, 2024
@XprobeBot XprobeBot modified the milestones: v0.11.3, v0.11.4, v0.12.0, v0.12.1 May 31, 2024
@XprobeBot XprobeBot modified the milestones: v0.12.1, v0.12.2 Jun 14, 2024
@zmh2000829
Copy link

zmh2000829 commented Jun 21, 2024 •
edited
Loading

have encountered the same bug. -- chatglm全系列

@XprobeBot XprobeBot modified the milestones: v0.12.2, v0.12.4, v0.13.0, v0.13.1 Jun 28, 2024
@XprobeBot XprobeBot modified the milestones: v0.13.1, v0.13.2 Jul 12, 2024
@XprobeBot XprobeBot modified the milestones: v0.13.2, v0.13.4 Jul 26, 2024
@github-actions GitHub Actions
Copy link

github-actions bot commented Aug 6, 2024

This issue is stale because it has been open for 7 days with no activity.

@github-actions github-actions bot added the stale label Aug 6, 2024
@github-actions GitHub Actions
Copy link

This issue was closed because it has been inactive for 5 days since being marked as stale.

@github-actions github-actions bot closed this as not planned Won't fix, can't repro, duplicate, stale Aug 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
question Further information is requested stale
Projects
None yet
Milestone
v0.14
Development

No branches or pull requests
6 participants
@dongrixinyu @YoungerXu @zmh2000829 @BaiMeiyingxue @ChengjieLi28 @XprobeBot