lvllm-v2.3.4

• Merge vllm tag 'v0.25.0'

• feat: update lk_moe==2.2.1 add activation_type config for swiglu/oai/relu2/silu, for example: nvidia/MiniMax-M3-NVFP4

• Example launch command for MiniMax-M3-NVFP4 (EPYC 7642 *2, RTX 3090 *2, 1TB DDR4 3200, decode ~18.5 t/s). Note: --block-size 128 is required for this model.:

PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True \
LVLLM_MOE_NUMA_ENABLED=1 \
LK_THREADS=48 \
OMP_NUM_THREADS=1 \
LK_THREAD_BINDING=CPU_CORE \
LVLLM_GPU_PREFETCH_WINDOW=1 \
LVLLM_GPU_PREFILL_MIN_BATCH_SIZE=128 \
LK_POWER_SAVING=1 \
VLLM_TEST_FORCE_FP8_MARLIN=1 \
vllm serve \
  --model ${HOME}/Downloads/MiniMax-M3-NVFP4 \
  --host 0.0.0.0 \
  --port 8070 \
  --tensor-parallel-size 2 \
  --max-model-len 36000 \
  --gpu-memory-utilization 0.95 \
  --trust-remote-code \
  --tokenizer-mode auto \
  --served-model-name MiniMax-M3-NVFP4 \
  --compilation_config.cudagraph_mode FULL_DECODE_ONLY \
  --enable_prefix_caching \
  --enable-chunked-prefill \
  --max_num_batched_tokens 17000 \
  --dtype bfloat16 \
  --max_num_seqs 2 \
  --compilation_config.mode VLLM_COMPILE \
  --enable-auto-tool-choice \
  --tool-call-parser minimax_m3 \
  --reasoning-parser minimax_m3 \
  --block-size 128