Skip to content

feat(vllm): add optional --tokenizer argument for k8s deployments#841

Merged
starpit merged 1 commit intoIBM:mainfrom
starpit:feat/vllm-tokenizer-support
Feb 15, 2026
Merged

feat(vllm): add optional --tokenizer argument for k8s deployments#841
starpit merged 1 commit intoIBM:mainfrom
starpit:feat/vllm-tokenizer-support

Conversation

@starpit
Copy link
Copy Markdown
Member

@starpit starpit commented Feb 14, 2026

Add support for specifying a custom tokenizer when deploying vLLM to Kubernetes.
The tokenizer argument is optional and defaults to the model's built-in tokenizer.

This implementation is for --target=k8s only. Support for --target=gce is TODO.

Changes

  • Add --tokenizer CLI argument to vllm up command
  • Pass tokenizer through to k8s deployment manifest
  • Update deployment.yml to conditionally add --tokenizer flag to vllm serve
  • Box VllmCommands and ImageCommands to fix clippy large_enum_variant warning
  • Add comprehensive test coverage for tokenizer functionality

Usage

# With custom tokenizer
spnl vllm up --target k8s my-deployment \
  --model meta-llama/Llama-2-7b-hf \
  --tokenizer custom/tokenizer \
  --hf-token <token>

# Without tokenizer (uses model's default)
spnl vllm up --target k8s my-deployment \
  --model meta-llama/Llama-2-7b-hf \
  --hf-token <token>

Testing

✅ All 15 k8s tests pass
✅ Clippy checks pass with -D warnings
✅ Rustfmt formatting is correct

Made with Bob

Add support for specifying a custom tokenizer when deploying vLLM to Kubernetes.
The tokenizer argument is optional and defaults to the model's built-in tokenizer.

This implementation is for --target=k8s only. Support for --target=gce is TODO.

Changes:
- Add --tokenizer CLI argument to vllm up command
- Pass tokenizer through to k8s deployment manifest
- Update deployment.yml to conditionally add --tokenizer flag to vllm serve
- Box VllmCommands and ImageCommands to fix clippy large_enum_variant warning
- Add comprehensive test coverage for tokenizer functionality

Made with Bob

Signed-off-by: Nick Mitchell <nickm@us.ibm.com>
@starpit starpit added the made with bob PR created with assistance from Bob AI label Feb 14, 2026
@starpit starpit merged commit d02d4a3 into IBM:main Feb 15, 2026
43 of 45 checks passed
@starpit starpit deleted the feat/vllm-tokenizer-support branch February 15, 2026 14:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

made with bob PR created with assistance from Bob AI

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant