The space between words disapear

[yiyexy]

xtuner chat internlm/internlm2-chat-7b \ --visual-encoder openai/clip-vit-large-patch14-336 \ --llava xtuner/llava-internlm2-7b \ --prompt-template internlm2_chat \ --image $IMAGE_PATH

[LZHgrla]

Hi @yiyexy
Could you please paste your env info here, and I will try to fix it.

pip list

[yiyexy]

Hi @yiyexy Could you please paste your env info here, and I will try to fix it.

pip list

Sure, here is my env info.

Package                       Version
----------------------------- ------------
accelerate                    0.26.1
addict                        2.4.0
aiohttp                       3.9.1
aiosignal                     1.3.1
annotated-types               0.6.0
async-timeout                 4.0.3
attrs                         23.2.0
bitsandbytes                  0.42.0
certifi                       2023.11.17
charset-normalizer            3.3.2
contourpy                     1.2.0
cycler                        0.12.1
datasets                      2.16.1
deepspeed                     0.12.6
dill                          0.3.7
distro                        1.9.0
einops                        0.7.0
et-xmlfile                    1.1.0
filelock                      3.13.1
fonttools                     4.47.2
frozenlist                    1.4.1
fsspec                        2023.10.0
func-timeout                  4.3.5
hjson                         3.1.0
huggingface-hub               0.20.2
idna                          3.6
importlib-metadata            7.0.1
Jinja2                        3.1.3
jsonschema                    4.21.0
jsonschema-specifications     2023.12.1
kiwisolver                    1.4.5
lagent                        0.1.2
markdown-it-py                3.0.0
MarkupSafe                    2.1.3
matplotlib                    3.8.2
mdurl                         0.1.2
mmengine                      0.10.2
mpi4py-mpich                  3.1.2
mpmath                        1.3.0
multidict                     6.0.4
multiprocess                  0.70.15
MultiScaleDeformableAttention 1.0
networkx                      3.2.1
ninja                         1.11.1.1
numpy                         1.26.3
nvidia-cublas-cu12            12.1.3.1
nvidia-cuda-cupti-cu12        12.1.105
nvidia-cuda-nvrtc-cu12        12.1.105
nvidia-cuda-runtime-cu12      12.1.105
nvidia-cudnn-cu12             8.9.2.26
nvidia-cufft-cu12             11.0.2.54
nvidia-curand-cu12            10.3.2.106
nvidia-cusolver-cu12          11.4.5.107
nvidia-cusparse-cu12          12.1.0.106
nvidia-nccl-cu12              2.18.1
nvidia-nvjitlink-cu12         12.3.101
nvidia-nvtx-cu12              12.1.105
opencv-python                 4.9.0.80
openpyxl                      3.1.2
packaging                     23.2
pandas                        2.1.4
peft                          0.7.1
pillow                        10.2.0
pip                           23.3.1
platformdirs                  4.1.0
protobuf                      4.25.2
psutil                        5.9.7
py-cpuinfo                    9.0.0
pyarrow                       14.0.2
pyarrow-hotfix                0.6
pydantic                      2.5.3
pydantic_core                 2.14.6
Pygments                      2.17.2
pynvml                        11.5.0
pyparsing                     3.1.1
python-dateutil               2.8.2
pytz                          2023.3.post1
PyYAML                        6.0.1
referencing                   0.32.1
regex                         2023.12.25
requests                      2.31.0
rich                          13.7.0
rpds-py                       0.17.1
safetensors                   0.4.1
scipy                         1.11.4
sentencepiece                 0.1.99
setuptools                    68.2.2
six                           1.16.0
sympy                         1.12
termcolor                     2.4.0
tiktoken                      0.5.2
tokenizers                    0.15.0
tomli                         2.0.1
torch                         2.1.2
tqdm                          4.66.1
transformers                  4.36.2
transformers-stream-generator 0.0.4
triton                        2.1.0
typing_extensions             4.9.0
tzdata                        2023.4
urllib3                       2.1.0
wheel                         0.41.2
xtuner                        0.1.12
xxhash                        3.4.1
yapf                          0.40.2
yarl                          1.9.4
zipp                          3.17.0

[LZHgrla]

Still cannot reproduce this issue in my env, even after aligning the versions of your env.

Some tips your can try:

1. Update xtuner version to 0.1.13
2. Update your LLM files to those of upstream huggingface hub

Another solution is to disable the streamer, by xtuner chat XXX --no-streamer, and I think this can solve this problem.

[yiyexy]

Still cannot reproduce this issue in my env, even after aligning the versions of your env.

Some tips your can try:

1. Update xtuner version to 0.1.13
2. Update your LLM files to those of upstream huggingface hub

Another solution is to disable the streamer, by xtuner chat XXX --no-streamer, and I think this can solve this problem.

OK, I will try to do that and will get back to you with the results as soon as possible today. Please keep an eye on this issue.

[yiyexy]

Still cannot reproduce this issue in my env, even after aligning the versions of your env.

Some tips your can try:

1. Update xtuner version to 0.1.13
2. Update your LLM files to those of upstream huggingface hub

Another solution is to disable the streamer, by xtuner chat XXX --no-streamer, and I think this can solve this problem.

By the way, the result is normal when I use llava-internlm2-20b.

[LZHgrla]

@yiyexy
How do you download these LLMs?
Automatically downloaded to .cache or downloaded manually to local?

[yiyexy]

@yiyexy How do you download these LLMs? Automatically downloaded to .cache or downloaded manually to local?

I downloaded internlm/internlm2-chat-7b manually.
And my steps in 7b are same as 20b.

[LZHgrla]

@yiyexy I think you can update the files of 7B LLM.

Updating .json, .py, ... files is enough, and keeping the .bin, .model files as they are

[yiyexy]

@yiyexy I think you can update the files of 7B LLM.

Updating .json, .py, ... files is enough, and keeping the .bin, .model files as they are

But I downloaded this LLM today. Where are new files to update?

[LZHgrla]

@yiyexy I think you can update the files of 7B LLM.
Updating .json, .py, ... files is enough, and keeping the .bin, .model files as they are

But I downloaded this LLM today. Where is the new files to update?

Oh, nothing needs.

[xmy0916]

把streamer关了吧，chat.py的代码应该写的有点bug，streamer默认是打开的，他是一个个单词往外蹦，没仔细看怎么写的，应该手动加个空格就好了，简单点就直接--no-streamer把它关了就行了

[LZHgrla]

Hi! @yiyexy @xmy0916

Sorry for the late reply.

There is indeed a bug of our streamer.
#404 will fix it!