[CI/Build] fix pip cache with vllm_nccl & refactor dockerfile to build wheels

[youkaichao]

the vllm_nccl package must be installed from source distribution
pip is too smart to store a wheel in the cache, and other CI jobs
will directly use the wheel from the cache, which is not what we want.
we need to remove it manually

[simon-mo]

Actually this won't work because we don't want to ship devel image for production. We should still use the runtime base image. The right fix should be change

FROM nvidia/cuda:12.1.0-runtime-ubuntu22.04 AS vllm-base

to

FROM nvidia/cuda:12.1.0-base-ubuntu22.04 AS vllm-base

[youkaichao]

@simon-mo please take a look and see if the modification of dockerfile is good. The test seems to be ok with the modification.

[youkaichao]

@simon-mo so we want to build wheel using dev image, and use that wheel for the rest images, right?

[simon-mo]

Yes. Also also manually test the openai image locally to ensure it has all the necessary dependencies.

We want to avoid using the devel image for testing and production in general

[youkaichao]

Ideal case:

nvidia/cuda:12.1.0-devel-ubuntu22.04 --> dev (install requirements) --> build (for building vllm wheels) and flash-attn-builder (for building flash-attn wheels)

nvidia/cuda:12.1.0-base-ubuntu22.04 --> vllm-base (install vllm from wheel) --> test ( for testing, only need to copy tests folder), and vllm-openai (for openai server, only need to install accelerate hf_transfer modelscope and start the server)

[simon-mo]

plz confirm locally that the openai server still work as expected.

[youkaichao]

plz confirm locally that the openai server still work as expected.

don't get it. I think we have tests for api server 👀

[simon-mo]

Similar to release process, can you build the final container locally (DOCKER_BUILDKIT=1 docker build . --target vllm-openai --tag vllm/vllm-openai --build-arg max_jobs=1) and confirm the server works (docker run --runtime nvidia --gpus all -p 8000:8000 vllm/vllm-openai). Our CI uses the test container but not the openai server container.

[youkaichao]

Okay, confirmed it works.