diff --git a/vllm/KNOWN_ISSUES.md b/vllm/KNOWN_ISSUES.md index f9567d7..9c147e1 100644 --- a/vllm/KNOWN_ISSUES.md +++ b/vllm/KNOWN_ISSUES.md @@ -12,9 +12,9 @@ Workaround: Change the PCIe slot configuration in BIOS from Auto/x16 to x8/x8. With this change, over 40 GB/s bi-directional P2P bandwidth can be achieved. Root cause analysis is still in progress. -# 03. Container OOM killed by using `--enable-auto-tool-choice` and starting container not by /bin/bash and not run `source /opt/intel/oneapi/setvars.sh` +# 03. Container OOM killed (and vllm performance drop) when starting container not by /bin/bash and not run `source /opt/intel/oneapi/setvars.sh` -When using `--enable-auto-tool-choice` and deploy container by docker-compose without `source /opt/intel/oneapi/setvars.sh`, the LD_LIBRARY_PATH will be different and cause the container OOM. It can be reproduced by this two command: +When using `--enable-auto-tool-choice` and deploy container by docker-compose without `source /opt/intel/oneapi/setvars.sh`, the LD_LIBRARY_PATH will be different and cause the container OOM (or performance drop). It can be reproduced by this two command: ```bash docker run --rm --entrypoint "/bin/bash" --name=test intel/llm-scaler-vllm:latest -c env | grep LD_LIBRARY_PATH diff --git a/vllm/README.md b/vllm/README.md index 549dcd7..f81ab0c 100644 --- a/vllm/README.md +++ b/vllm/README.md @@ -2177,6 +2177,8 @@ curl http://localhost:8000/v1/chat/completions \ "max_tokens": 128 }' ``` + +if want to process image in server local, you can `"url": "file:/llm/models/test/1.jpg"` to test. --- ### 2.4.1 Audio Model Support [Deprecated] @@ -2319,12 +2321,11 @@ python3 -m vllm.entrypoints.openai.api_server \ #### Run the demo -To verify your setup, clone the official MinerU repository and run the demo script: +To verify mineru ```bash -git clone https://github.com/opendatalab/MinerU.git -cd MinerU/demo -python3 demo.py +#mineru -p -o -b vlm-http-client -u http://127.0.0.1:8000 +mineru -p /llm/MinerU/demo/pdfs/small_ocr.pdf -o ./ -b vlm-http-client -u http://127.0.0.1:8000 ``` ---