v0.10.0

🌈 What's New?

Core Inference

• Capture exit code in model status by @Suresh-Nakkeran in #2290
• Added support for loading models with custom name by @Suresh-Nakkeran in #2252
• Support Knative rollout duration annotation by @andyi2it in #2300
• Allows to make Istio and VirtualServices optional for serverless mode by @Suresh-Nakkeran in #2380
• Add prometheus metrics for kserve model server by @alexagriffith in #2425
• Add prometheus port configs for aggregating metrics in queue proxy by @alexagriffith in #2459
• Add queue proxy ext for metrics aggregation by @alexagriffith in #2478
• FastAPI: Separate model server and data plane by @sukumargaonkar @xfu83 in #2444
• Make webhook port number configurable by @sel in #2498
• Support V2 GRPC for KServe Model Server by @Suresh-Nakkeran in #2415
• Add Unified Inference Type and refactor REST/gRPC server code by @yuzisun in #2629
• add model_ready v2 endpoint by @alexagriffith in #2617
• Start uvicorn server in multiple process as per worker count by @Suresh-Nakkeran in #2573

Serving Runtimes

• Add labels and annotations to ServingRuntimePodSpec by @lizzzcai in #2440
• Add ImagePullSecrets to ServingRuntimePodSpec by @lizzzcai in #2443
• Bump torchserve version to 0.7.0 by @jagadeeshi2i in #2530

Advanced Inference

• Propagating IG headers to it's nodes. by @rachitchauhan43 in #2396

Storage Provider

• AWS IRSA S3 Support by @matty-rose in #2373
• Retrieve SAS token for Azure storage by @tjandy98 in #2418
• Supports more authentication approaches on Azure in Storage Initializer by @laozc in #2014
• Supports more authentication approaches on Azure in Storage Initializer - Build fix by @Suresh-Nakkeran in #2481

Multi-Arch Image

• Adding support for linux/ppc64le in github action for kserve-controller by @adilhusain-s in #2550
• Adding support for linux/ppc64le arch in github action for kserve-agent by @adilhusain-s in #2549
• Adding multi-arch support for linux-ppc64le for router by @adilhusain-s in #2605
• Adding multi-arch support for linux-ppc64le for qpext by @adilhusain-s in #2604
• Fix multi-arch docker publish by @ddelange in #2619

⚠️ What's Changed

• Cleanup InferenceService configmap for ML framework related fields by @Suresh-Nakkeran in #2225
  Warning: If you still have inference service predictors which are not mutated by the webook to convert to the new model spec introduced since 0.8, you need to make a simple spec update to have them go through the mutating webhook.

• Pass request headers to predict method (#2284) by @andyi2it in #2360
  Warning: the preprocess, predict and postprocess now requires passing the additional argument with headers.

🐛 What's Fixed

• Fix cluster local label InferenceService by @TimKleinloog in #2101
• Restoring init container support. by @rachitchauhan43 in #2475
• Make aix use default image only if no image is provided by @andyi2it in #2503
• Change cluster-local label to networking.knative.dev/visibility by @tenzen-y in #2518
• Update with dry-run before diffing deployments by @cmaddalozzo in #2490
• storage-initializer: Let boto3 decide the endpoint for S3 by @dimara in #2377
• Fix failure to create gRPC isvc when specifying multiple ContainerPorts by @andyi2it in #2464
• fix light gbm model format by @alexagriffith in #2640

⬆️ Version Upgrade

• Update quick install to use 0.9.0 by @yuzisun in #2362
• update ray to 2.0.0 by @park12sj in #2410
• Go 1.18 upgrade by @yuzisun in #2420
• Reduce numpy version constraints by @alembiewski in #2424
• Update quick install knative version to 1.7.0 by @alexagriffith in #2431
• Update quick install to use Istio 1.15 by @yuzisun in #2446
• Updated yq version (v4.28.1) by @Suresh-Nakkeran in #2480
• Loosen boto3 requirement by @ddelange in #2644
• Bump kube-rbac-proxy version for kserve-controller by @ddelange in #2628

🔨 Project SDLC

• Regenerate the Go client by code-generator by @tenzen-y in #2458
• Code coverage by @andyi2it in #2351
• remove pytorchserver dead code by @alexagriffith in #2453
• Update KServe generated client code by @yuzisun in #2492
• Added a pytest ini to register the markers by @andyi2it in #2505
• Add security vulnerabilities reporting process by @yuzisun in #2523
• Add openssf best practice badge by @yuzisun in #2525
• Updated docker build to use a cpu only version of torchvision by @andyi2it in #2515
• Adding new manifests for 0.10.0 by @rachitchauhan43 in #2544
• Update configmap for helm chart v0.10.0-rc0 by @yuzisun in #2545
• Fix kserve annotation constants by @ddelange in #2491
• removed arch dependency for multiarc support by @pranavpandit1 in #2476
• Fixing indentation bug in image patch config by @rachitchauhan43 in #2539

Security Patches

• Optimize docker image builds by @andyi2it in #2319
• Update versions for art-explainer to resolve several critical CVEs by @MessKon in #2272
• Vulnerability fixes for kserve and model images by @andyi2it in #2320
• Patch fixed critical CVEs in kserve/alibi-explainer by @MessKon in #2270
• aix-explainer: update versions to fix CVEs by @MessKon in #2364

📝 Documentation Update

• fix: torchserve-grpc client by @jagadeeshi2i in #2353
• Update KServe main page README by @yuzisun in #2375
• Fix typos in main README by @rafvasq in #2384
• Update python/kserve dependency and art example by @yuzisun in #2391
• fix: update torchserve readme links by @jagadeeshi2i in #2394
• Add script to identify broken links by @ckadner in #2423
• Update CIFAR-10 Outlier Detector sample by @rafvasq in #2472
• Add an example of Lime text explainer from AIX360 toolkit by @C1berwiz in #2355
• Update sample YAMLs for s3 secrets to use right annotation by @dilverse in #2528
• Copy-edit torchserve readme by @rafvasq in #2531
• fix: set default torchserve model store uri to v1 by @jagadeeshi2i in #2635
• Point transformer doc to website repo by @yuzisun in #2623
• fix: Incorrect URL when making predictions via curl by @terrytangyuan in #2626

Full Changelog

• Optimize docker image builds by @andyi2it in #2319
• Update versions for art-explainer to resolve several critical CVEs by @MessKon in #2272
• Vulnerability fixes for kserve and model images by @andyi2it in #2320
• Capture exit code in model status by @Suresh-Nakkeran in #2290
• Update quick install to use 0.9.0 by @yuzisun in #2362
• Update quick install to use 0.9.0 by @yuzisun in #2363
• Patch fixed critical CVEs in kserve/alibi-explainer by @MessKon in #2270
• aix-explainer: update versions to fix CVEs by @MessKon in #2364
• fix: torchserve-grpc client by @jagadeeshi2i in #2353
• Update KServe main page README by @yuzisun in #2375
• Added support for loading models with custom name by @Suresh-Nakkeran in #2252
• Fix typos in main README by @rafvasq in #2384
• Add rollout duration feature by @andyi2it in #2300
• Update python/kserve dependency and art example by @yuzisun in #2391
• fix: update torchserve readme links by @jagadeeshi2i in #2394
• AWS IRSA S3 Support by @matty-rose in #2373
• storage-initializer: Let boto3 decide the endpoint for S3 by @dimara in #2377
• Allows to make Istio and VirtualServices optional by @Suresh-Nakkeran in #2380
• update ray to 2.0.0 by @park12sj in #2410
• Cleanup InferenceService configmap for ML framework related fields by @Suresh-Nakkeran in #2225
• Go 1.18 upgrade by @yuzisun in #2420
• Reduce numpy version constraints by @alembiewski in #2424
• Add script to identify broken links by @ckadner in #2423
• Pass request headers to predict method (#2284) by @andyi2it in #2360
• update quick install knative version to 1.7.0 by @alexagriffith in #2431
• add prometheus metrics for kserve model server by @alexagriffith in #2425
• add labels and annotations to ServingRuntimePodSpec by @lizzzcai in #2440
• Update quick install to use Istio 1.15 by @yuzisun in #2446
• remove pytorchserver dead code by @alexagriffith in #2453
• adding prometheus port configs for aggregating metrics in queue proxy by @alexagriffith in #2459
• Regenerate the Go client by code-generator by @tenzen-y in #2458
• Retrieve SAS token by @tjandy98 in #2418
• Code coverage by @andyi2it in #2351
• Add ImagePullSecrets to ServingRuntimePodSpec by @lizzzcai in #2443
• Update CIFAR-10 Outlier Detector sample by @rafvasq in #2472
• Fix cluster local label InferenceService by @TimKleinloog in #2101
• Add an example of Lime text explainer from AIX360 toolkit by @C1berwiz in #2355
• FastAPI: Separate model server and data plane by @sukumargaonkar in #2444
• Supports more authentication approaches on Azure in Storage Initializer by @laozc in #2014
• Updated yq version (v4.28.1) by @Suresh-Nakkeran in #2480
• Supports more authentication approaches on Azure in Storage Initializer - Build fix by @Suresh-Nakkeran in #2481
• Propagating IG headers to it's nodes. by @rachitchauhan43 in #2396
• Agriffith96/add queue proxy ext by @alexagriffith in #2478
• Restoring init container support. by @rachitchauhan43 in #2475
• Update KServe generated client code by @yuzisun in #2492
• Make webhook port number configurable by @sel in #2498
• Added a pytest ini to register the markers by @andyi2it in #2505
• Make aix use default image only if no image is provided by @andyi2it in #2503
• Support V2 GRPC for KServe Model Server by @Suresh-Nakkeran in #2415
• Updated docker build to use a cpu only version of torchvision by @andyi2it in #2515
• Change cluster-local label to networking.knative.dev/visibility by @tenzen-y in #2518
• Add security vulnerabilities reporting process by @yuzisun in #2523
• Add openssf best practice badge by @yuzisun in #2525
• Fix kserve annotation constants by @ddelange in #2491
• removed arch dependency for multiarc support by @pranavpandit1 in #2476
• Update sample YAMLs for s3 secrets to use right annotation by @dilverse in #2528
• Bump torchserve version by @jagadeeshi2i in #2530
• Fixing indentation bug in image patch config by @rachitchauhan43 in #2539
• Copy-edit torchserve readme by @rafvasq in #2531
• Update with dry-run before diffing deployments by @cmaddalozzo in #2490
• Adding new manifests for 0.10.0-rc0 by @rachitchauhan43 in #2544
• Update configmap for helm chart v0.10.0-rc0 by @yuzisun in #2545
• Update OWNERS by @alexagriffith in #2540
• Update quick install to use v0.10.0-rc0 by @sukumargaonkar in #2547
• Update OWNERS by @sukumargaonkar in #2546
• move kserve crd into separate chart by @yuzisun in #2552
• remove redundant kserve-config configmap by @Suresh-Nakkeran in #2555
• Made changes to fix issues with quick-install version parsing by @andyi2it in #2563
• Fix grpc is not working with logger by @andyi2it in #2463
• user defined model name overwrite issue fix by @Suresh-Nakkeran in #2342
• make prom metric names consistent by @alexagriffith in #2577
• Snyk scan by @andyi2it in #2378
• Replace ioutil package with os and io package by @tenzen-y in #2581
• Handle scenario where annotations are not given in raw deployment mode by @andyi2it in #2569
• sync cluster runtimes by @alexagriffith in #2593
• Removing duplicate servingruntimes CRD by @sukumargaonkar in #2595
• Change to fix sdk replace does not wait for new isvc to be ready by @andyi2it in #1925
• Update KServe 2023 Roadmap by @yuzisun in #2526
• Upgrade the K8s version to 1.24 for tests by @tenzen-y in #2584
• Update knative serving go dependency by @yuzisun in #2603
• Add constrains for numpy<1.24.0 by @yuzisun in #2606
• TR-16622 Ingress Kubernetes manifest upgrade to v1 by @stephanschielke in #2586
• Fix serverless installation mode link by @panli889 in #2610
• Bump Torchserve version to 0.7.0 by @jagadeeshi2i in #2611
• adding prom tags and tests by @alexagriffith in #2589
• Bumping up version to 0.10.0 for all the manifests. by @rachitchauhan43 in #2615
• Adding support for linux/ppc64le in github action for kserve-controller by @adilhusain-s in #2550
• Adding support for linux/ppc64le arch in github action for kserve-agent by @adilhusain-s in #2549
• Adding multi-arch support for linux-ppc64le for router by @adilhusain-s in #2605
• Update to v0.10.0-rc1 release manifests by @yuzisun in #2616
• Adding multi-arch support for linux-ppc64le for qpext by @adilhusain-s in #2604
• Fix multi-arch docker publish by @ddelange in #2619
• Start uvicorn server in multiple process as per worker count by @Suresh-Nakkeran in #2573
• Point transformer doc to website repo by @yuzisun in #2623
• fix: Incorrect URL when making predictions via curl by @terrytangyuan in #2626
• Added github action to test kserve helm by @Suresh-Nakkeran in #2588
• add model_ready v2 endpoint by @alexagriffith in #2617
• Update PRESENTATIONS.md by @yuzisun in #2630
• Use kubectl -k instead of kustomize by @yuzisun in #2634
• Bump kube-rbac-proxy version for kserve-controller by @ddelange in #2628
• fix: set default torchserve model store uri to v1 by @jagadeeshi2i in #2635
• fix light gbm model format by @alexagriffith in #2640
• Loosen boto3 requirement by @ddelange in #2644
• Update slack link by @yuzisun in #2648
• [bugfix]: support kubectl server version correctly by @anencore94 in #2652
• Update ModelMesh version to v0.10.0 by @ckadner in #2645
• Add Unified Inference Type and refactor REST/gRPC server code by @yuzisun in #2629
• Bumping version to 0.10.0 by @rachitchauhan43 in #2656
• Update to 2023 license by @yuzisun in #2657
• Fix failure to create gRPC isvc when specifying multiple ContainerPorts by @andyi2it in #2464

New Contributors

• @MessKon made their first contribution in #2272
• @rafvasq made their first contribution in #2384
• @dimara made their first contribution in #2377
• @park12sj made their first contribution in #2410
• @alembiewski made their first contribution in #2424
• @ckadner made their first contribution in #2423
• @alexagriffith made their first contribution in #2431
• @lizzzcai made their first contribution in #2440
• @tenzen-y made their first contribution in #2458
• @tjandy98 made their first contribution in #2418
• @TimKleinloog made their first contribution in #2101
• @C1berwiz made their first contribution in #2355
• @rachitchauhan43 made their first contribution in #2396
• @sel made their first contribution in #2498
• @pranavpandit1 made their first contribution in #2476
• @dilverse made their first contribution in #2528
• @cmaddalozzo made their first contribution in #2490
• @stephanschielke made their first contribution in #2586
• @panli889 made their first contribution in #2610
• @adilhusain-s made their first contribution in #2550
• @anencore94 made their first contribution in #2652

Full Changelog: v0.9.0...v0.10.0