Release v0.13.0-rc0 · kserve/kserve

🌈 What's New?

add support for async streaming in predict by @alexagriffith in #3475
Fix: Support model parallelism in HF transformer by @gavrishp in #3459
Support model revision and tokenizer revision in huggingface server by @lizzzcai in #3558
OpenAI schema by @tessapham in #3477
Support OpenAIModel in ModelRepository by @grandbora in #3590
updated xgboost to support json and ubj models by @andyi2it in #3551
Add OpenAI API support to Huggingfaceserver by @cmaddalozzo in #3582
VLLM support for OpenAI Completions in HF server by @gavrishp in #3589
Add a user friendly error message for http exceptions by @grandbora in #3581
feat: Provide minimal distribution of CRDs by @terrytangyuan in #3492
set default SAFETENSORS_FAST_GPU and HF_HUB_DISABLE_TELEMETRY in HF Server by @lizzzcai in #3594
Enabled the multiple domains support on an inference service by @houshengbo in #3615
Add base model for proxying request to an OpenAI API enabled model server by @cmaddalozzo in #3621
Add headers to predictor exception logging by @grandbora in #3658
Enhance controller setup based on available CRDs by @israel-hdez in #3472

Remove conversion webhook from manifests by @Jooho in #3476
Remove cluster level list/watch for configmaps, serviceaccounts, secrets by @sivanantha321 in #3469
chore: Remove Seldon Alibi dependencies. Fixes #3380 by @terrytangyuan in #3443
docs: Move Alibi explainer to docs by @terrytangyuan in #3579
Remove generate endpoints by @cmaddalozzo in #3654

Fix:Support Parallelism in vllm runtime by @gavrishp in #3464
fix: Instantiate HuggingfaceModelRepository only when model cannot be loaded. Fixes #3423 by @terrytangyuan in #3424
Fix isADirectoryError in Azure blob download by @tjandy98 in #3502
Fix bug: Remove redundant helm chart affinity on predictor CRD by @trojaond in #3481
Make the modelcar injection idempotent by @rhuss in #3517
Only pad left for decode-only architecture models. by @sivanantha321 in #3534
fix lint typo on Makefile by @spolti in #3569
fix: Set writable cache folder to avoid permission issue. Fixes #3562 by @terrytangyuan in #3576
Fix model unload in server stop method by @sivanantha321 in #3587
Fix golint errors by @andyi2it in #3552
Fix make deploy-dev-storage-initializer not working by @sivanantha321 in #3617
Fix Pydantic 2 warnings by @cmaddalozzo in #3622
build: Fix CRD copying in generate-install.sh by @terrytangyuan in #3620
Only load from model repository if model binary is not found under model_dir by @sivanantha321 in #3559
build: Remove misleading logs from minimal-crdgen.sh by @terrytangyuan in #3641
Assign device to input tensors in huggingface server with huggingface backend by @saileshd1402 in #3657
Fix Huggingface server stopping criteria by @cmaddalozzo in #3659
Explicitly specify pad token id when generating tokens by @sivanantha321 in #3565
Fix quick install does not cleans up Istio installer by @sivanantha321 in #3660

Upgrade orjson to version 3.9.15 by @spolti in #3488
feat: upgrade to new fastapi, update models to handle both pydantic v… by @timothyjlaurent in #3374
Update cert manager version in quick install script by @shauryagoel in #3496
ci: Bump minikube version to work with newer K8s version by @terrytangyuan in #3498
upgrade knative to 1.13 by @andyi2it in #3457
Upgrade istio to 1.20 works for the Github Actions by @houshengbo in #3529
chore: Bump ModelMesh version to v0.12.0-rc0 in Helm chart by @terrytangyuan in #3642

Enhance CI environment by @sivanantha321 in #3440
Fixed go lint error using golangci-lint tool. by @andyi2it in #3378
chore: Update list of reviewers by @ckadner in #3484
build: Add helm docs update to make generate command by @terrytangyuan in #3437
Added v2 infer test for supported model frameworks. by @andyi2it in #3349
fix the quote format same with others and docstrings by @leyao-daily in #3490
remove unnecessary Istio settings from quick_install.sh by @peterj in #3493
Remove GOARCH by @mkumatag in #3523
GH Alert: Potential file inclusion via variable by @spolti in #3520
Update codeQL to v3 by @spolti in #3548
switch e2e test inference graph to raw mode by @andyi2it in #3511
Black lint by @cmaddalozzo in #3568
Fix python linter by @sivanantha321 in #3571
build: Add flake8 and black to pre-commit hooks by @terrytangyuan in #3578
build: Allow pre-commit to keep changes in reformatted code by @terrytangyuan in #3604
Allow rerunning failed workflows by comment by @andyi2it in #3550
add re-run info in the PR templates by @spolti in #3633
Add e2e tests for huggingface by @sivanantha321 in #3600
Test image builds for ARM64 arch in CI by @sivanantha321 in #3629
workflow file for cherry-pick on comment by @andyi2it in #3653

CVE-2024-24762 - update fastapi to 0.109.1 by @spolti in #3556
golang.org/x/net Allocation of Resources Without Limits or Throttling by @spolti in #3596
Fix CVE-2023-45288 for qpext by @sivanantha321 in #3618
Security fix - CVE 2024 24786 by @andyi2it in #3585

Full Changelog: v0.12.1...v0.13.0-rc0