Version 2.14.3

What's Changed

• fix(sdk): Set spec.description when compiling to Kubernetes manifests (#12132)
• fix(tools): Fix a typo in convert_to_k8s_format (#12161)
• fix(backend): Fix mutating webhook when pipeline name label value is too long (#12162)
• fix(backend): Return from pipeline deletions after the cache is updated (#12153)
• fix(backend/sdk): Fix REST API outputs for pipeline versions with invalid platform specs (#12183)
• fix(backend): Add support for additional filters on the Kubernetes native API (#12189)

Full Changelog: 2.14.0...2.14.3

Version 2.14.0

This release is a version alignment release. This release succeeds the KFP 2.5.0 release. In this 2.14.0 release, KFP backend is aligned in major and minor versions (X.Y) with all the KFP python packages. Please read more about this and the KFP versioning policy here.

What's Changed

• chore(sdk): kfp and kfp-kubernetes 2.13 & 1.5 by @HumairAK in #11870
• update release doc paths & make script executable by @HumairAK in #11871
• chore(components): GCPC 2.20.0 Release by @copybara-service[bot] in #11869
• chore(components): Relax KFP SDK dependency for GCPC by @copybara-service[bot] in #11879
• chore: bump master to release 2.5 by @HumairAK in #11872
• chore(deps): bump sagemaker from 2.1.0 to 2.237.3 in /components/aws/sagemaker by @dependabot[bot] in #11876
• Internal cleanup by @copybara-service[bot] in #11877
• feat(backend): Add the Kubernetes native pipeline store by @mprahl in #11881
• fix(sdk): resolve issue when creating pipeline version from pipeline name using the cli. Fixes #11810 by @chjdev in #11866
• fix(backend): Stop logging the strack trace on benign user errors by @mprahl in #11883
• chore: update all owners files by @HumairAK in #11886
• feat(backend): Allow the launcher command to be configurable by @mprahl in #11888
• feat(backend/sdk): enable dsl.Collected for parameters & artifacts by @zazulam in #11725
• fix(sdk): fix pip install for dev by @dandawg in #11891
• chore: Adding Adopters file for CNCF graduation by @franciscojavierarceo in #11894
• feat(backend): parameterize retryStrategy input in Argo workflow by @alyssacgoins in #11861
• chore: Add mprahl to the samples owners file by @mprahl in #11897
• chore: handle empty tolerations dict/lists parrameterization by @HumairAK in #11898
• feat(backend): add the option to enable/disable cache globally by @hbelmiro in #11831
• feat(frontend): Add "Always Use Latest Version" option for recurring runs (fixes #11581) by @kaikaila in #11755
• add maintainer to kfp by @HumairAK in #11900
• chore: correct broken links in README.md by @Shekharrajak in #11896
• fix(sdk): Resolves issue when using ParallelFor with param and depending tasks by @mai-nakagawa in #11903
• chore(components): fix v1 model_evaluations components mingled under the preview module by @copybara-service[bot] in #11902
• feat(backend): Add migration script to create Pipeline and PipelineVersion objects from the database to Kubernetes API by @VaniHaripriya in #11884
• chore(components): GCPC 2.20.1 Release by @copybara-service[bot] in #11909
• chore: Add IBM Research Foundation Model Data Engineering team by @anishasthana in #11905
• chore(tests): fix KFP SDK tests by @hbelmiro in #11911
• chore(backend): break up driver logic by @HumairAK in #11885
• chore(components): Update OS packages in GCPC container image by @copybara-service[bot] in #11912
• feat(backend): Add CLI flags to support Kubernetes native API implementation by @VaniHaripriya in #11907
• chore(test): Update cache test timeout and polling intervals by @hbelmiro in #11916
• docs(KEP): Propose a new pipeline run workspace feature by @mprahl in #11875
• chore(backend): upgrade mysql to 8.4 by @HumairAK in #11917
• chore: Enable go fmt as a lint check for Go code by @cmdevoto in #11830
• chore(test): fixed log collection in tests by @hbelmiro in #11910
• feat(proto): Add WorkspaceConfig and KubernetesWorkspaceConfig message types to pipeline_spec proto by @VaniHaripriya in #11921
• • fix(launcher): missing executorInput parameter values caused by {{$}} evaluation order by @ntny in #11925
• chore(ci): clean up unnecessary parentheses and fix the formatting of the CI files by @lekaf974 in #11926
• docs(frontend): add ui dev docs by @HumairAK in #11931
• feat: add openshift env to manifests by @HumairAK in #11932
• chore: Updating KFP docs by @franciscojavierarceo in #11927
• docs: add GoLand-specific configs to backend Readme by @alyssacgoins in #11919
• test: Update the Kubernetes and Python version ranges in the CI by @mprahl in #11924
• feat(backend) implement retryStrategy for nested pipelines by @alyssacgoins in #11908
• chore(deps): bump tornado from 6.4.2 to 6.5.1 in /test/sample-test by @dependabot[bot] in #11936
• fix(tests): free up space in kfp samples test workflow by @alyssacgoins in #11942
• chore: pipeline api spec minor release by @HumairAK in #11939
• docs(frontend): added build instruction to readme by @rsun19 in #11943
• chore: Nest sdk docs by @anishasthana in #11945
• chore: Adding Devin's DeepWiki to README by @franciscojavierarceo in #11948
• docs(frontend): Added clean up section to openshift docs by @rsun19 in #11951
• chore: add c1 to ADOPTERS.md by @zazulam in #11954
• feat(sdk): update PipelineConfig to reflect new workspace Protobuf changes by @VaniHaripriya in #11934
• feat(backend/frontend): Add the name field for pipelines and pipeline versions by @mprahl in #11952
• feat(frontend) Use native k8s probes for pipeline-ui by @LaurentGoderre in #11955
• chore(deps): bump torch from 2.6.0 to 2.7.1 in /samples/contrib/ibm-samples/ffdl-seldon/source/seldon-pytorch-serving-image by @dependabot[bot] in #11959
• fix(cache): Optimize cache expiration query solves #11614 by @cbartram in #11920
• feat(backend/sdk): support PipelineTaskFinalStatus input by @alyssacgoins in #11953
• feat(backend): Use native k8s probes for api-server and visualization by @LaurentGoderre in #11960
• test(backend): Add unit test for NewLauncherV2 by @jpthompson23 in #11854
• chore(deps): bump requests from 2.18.4 to 2.32.4 in /components/contrib/arena/docker by @dependabot[bot] in #11963
• fix(backend): Fix the pipeline samples by @mprahl in #11967
• chore(deps): bump requests from 2.32.2 to 2.32.4 in /test/kfp-functional-test by @dependabot[bot] in #11966
• fix(test): Fix the frontend sample test by @mprahl in #11968
• chore(deps): bump brace-expansion in /test/frontend-integration-test by @dependabot[bot] in #11971
• chore(deps): bump brace-expansion from 1.1.11 to 1.1.12 in /frontend by @dependabot[bot] in #11972
• chore(deps): bump brace-expansion from 1.1.11 to 1.1.12 in /frontend/server by @dependabot[bot] in #11978
• chore(deps): bump protobuf from 4.25.3 to 4.25.8 in /test/kfp-functional-test by @dependabot[bot] in #11985
• user master for dev manifests by @HumairAK in #11977
• chore(deps): bump protobuf from 4.25.4 to 4.25.8 in /test/sample-test by @dependabot[bot] in #11986
• chore(deps): bump urllib3 from 1.26.15 to 2.5.0 in /components/aws/sagemaker by @dependabot[bot] in #11993
• chore: update workflows to test proto changes in ci by @zazulam in #11995
• fix(apiserver): fix typos in resource_manager.go: by @milosjava in #11998
• fix(backend): Fix boolean flag syntax for `--cache_d...

Version 2.5.0

What's Changed

• fix(frontend): Fix the frontend image build with Node 22 by @mprahl in #11524
• chore: switch to using gh container registry in manifests by @HumairAK in #11526
• fix(backend): Fix enable_caching issues when handling PVC creation/deletion by @rimolive in #11411
• chore: switch release/api generator images to ghcr by @HumairAK in #11528
• test: fix CI failure by upgrading upload-artifact version by @VaniHaripriya in #11529
• chore(release): bumped version to 2.4.0 by @HumairAK in #11531
• feat(docs): Add architecture diagram by @anishasthana in #11490
• chore: Remove ouadakarim from Kubeflow Pipelines reviewers by @ouadakarim in #11538
• fix(backend): removed old version comment by @dandawg in #11549
• feat(docs): Replace ADRs with KEPs for documentation by @anishasthana in #11535
• fix(frontend): fixes Default pipeline input params are missing from the GUI. Fixes #11515 by @ElayAharoni in #11518
• fix: Extend env variables in ml-pipeline-ui deployment by @akagami-harsh in #11552
• fix(sdk): dsl.component docstring typo by @dandawg in #11547
• chore: bump requests-toolbelt to 1.0.0 by @SimonDR-Boltzmann in #10762
• feat(backend): add support for uploading new sample pipeline vers by @HumairAK in #11553
• Fix typo in documentation for contribution and developer guide by @assu-2000 in #11537
• fix: Update broken api-connect link by @dandawg in #11521
• chore(test): Fixed namespace in tests by @hbelmiro in #11545
• docs(backend): improved backend README by @dandawg in #11511
• fix(CI): Use the correct image registry for replacements in integration tests by @mprahl in #11564
• feat(api): Add SemaphoreKey and MutexName fields to proto by @DharmitD in #11384
• feat(backend): Allow recurring runs to always use the latest pipeline version by @mprahl in #11560
• chore(deps): replace deprecated ioutil with os and io in tests by @dandawg in #11577
• chore(frontend) : Fix frontend failing ci test by @chahatsagarmain in #11575
• fix(backend): Replaced hardcoded ServiceAccount with default config by @hbelmiro in #11578
• docs: change node version from 14 to 22 in frontend project by @mahdikhashan in #11567
• feat(components): Introduce max_wait_duration to custom job to v1 GCPC custom job components/utils by @copybara-service in #11452
• chore(components): Bump image version for Structured Data pipelines by @copybara-service in #11588
• chore(backend): fixed imports on metadata_helpers.py by @dandawg in #11604
• fix(backend): the metacontroller is broken since #11474 by @juliusvonkohout in #11608
• feat: Introduce cache_key to sdk by @zzmao in #11466
• chore: Remove License checker by @HumairAK in #11609
• chore(deps): bump github.com/golang/glog from 1.2.0 to 1.2.4 by @dependabot in #11563
• chor(test) : Fix kfp-sdk-test for different python versions by @chahatsagarmain in #11559
• feat(backend): Add support for job and task placeholders in the KFP backend by @mprahl in #11599
• chore(backend): updated deprecated grpc func by @dandawg in #11603
• feat(components): Fix typos: 'statgey' -> 'strategy'in v1 GCPC custom job components/utils by @copybara-service in #11619
• docs(backend): small formatting improvements by @dandawg in #11570
• docs(backend): fixed docker cmd in docs for non-amd64 by @dandawg in #11610
• [chore][backend] Add workflow to validate affected generated files by @chahatsagarmain in #11539
• chore(components): GCPC 2.19.0 Release by @copybara-service in #11620
• Refer to central KEP template by @anishasthana in #11593
• chore: use local proto go packages for ci by @HumairAK in #11629
• fix(backend): upgrade go version to 1.22.12 to fix CVE-2024-45336 by @dandawg in #11631
• fix(backend) fix run retry for argo by @ntny in #11585
• (test) : Collect and upload logs when test fails for k8s by @chahatsagarmain in #11618
• fix(frontend): restrict file explorer to show only .yaml, .yml, .zip, and .tar.gz files by @muzzlol in #11623
• feat(backend): Add support for importing models stored in the Modelcar format (sidecar) by @mprahl in #11606
• fix(backend): fixes DAG status update to reflect completion of all tasks by @VaniHaripriya in #11651
• chore(test): Increased timeout in test by @hbelmiro in #11658
• feat(backend): configurable log level for driver / launcher images by @CarterFendley in #11278
• fix(backend): parallelFor resolve upstream inputs. Fixes #11520 by @zazulam in #11627
• fix(frontend): fixes optional pipeline inputs. Fixes #11632 by @izuku-sds in #11657
• fix(manifests): Upgrading metacontroller to v4.11.22 by @tarekabouzeid in #11656
• fix(backend): ignore unknown fields for pb json unmarshaling by @HumairAK in #11662
• Allow system.Model artifacts in the Modelcar format by @mprahl in #11674
• docs: Add a KEP for a Kubernetes native API for pipelines by @mprahl in #11551
• chore: nominate sdk/backend reviewers by @HumairAK in #11675
• chore(sdk): release kfp sdk 2.12.0 by @chensun in #11679
• docs: Add documentation for set_image_pull_secrets by @george-kuanli-peng in #11676
• Limit the number of parallel tests in SDK execution tests by @mprahl in #11680
• Use the correct SDK version in the SDK execution tests CI by @mprahl in #11683
• Fix format string in get_kfp_package_path by @mprahl in #11712
• chore(backend): partially revert 9afe23e by @HumairAK in #11713
• feat(sdk): Add Input Parameter support for configmap, secrets, node selectors, tolerations, pull secrets by @HumairAK in #11621
• chore: improved securitycontext for mysql by @juliusvonkohout in #11678
• Ignore empty PULL_NUMBER environment variables in SDK tests by @mprahl in #11714
• chore(sdk): Relax kfp-server-api upper bound to include 2.4 by @chensun in #11685
• chore(sdk): release kfp sdk 2.12.1 by @chensun in #11715
• fix(backend) fix execution-level retry on the Argo Workflows backend by @ntny in #11673
• remove unused function by @HumairAK in #11719
• fix(frontend): compatibility with pod_names v1 by @droctothorpe in #11682
• Chore: fix the PPC image for the dockerhub ratelimit and add the securitycontext by @juliusvonkohout in #11669
• chore(release): bumped version to 2.4.1 by @HumairAK in #11718
• feat(backend): Add types for KFP Kubernete Native API by @rimolive in #11672
• chore(frontend): update kubernetes client version to 0.12.1 by @droctothorpe in #11737
• fix: minio fsgroup for popular clusters by @juliusvonkohout in #11734
• chore(deps): bump golang.org/x/net from 0.33.0 to 0.36.0 in /test/tools/project-cleaner by @dependabot in #11748
• Fix Istio sidecar injection by moving from annotations to labels by @madmecodes in #11750
• docs(ui): add multi-user pro...

Version 2.4.1

What's Changed

• fix(backend): Replaced hardcoded ServiceAccount with default config by @hbelmiro in #11578
• chore: Remove License checker by @HumairAK in #11609
• fix(backend): the metacontroller is broken since by @juliusvonkohout in #11608
• fix(backend) fix run retry for argo by @ntny in #11585
• fix(backend): parallelFor resolve upstream inputs. by @zazulam in #11627
• fix(manifests): Upgrading metacontroller to v4.11.22 by @tarekabouzeid in #11656
• fix(backend): ignore unknown fields for pb json unmarshaling by @HumairAK in #11662
• chore: improved securitycontext for mysql by @juliusvonkohout in #11678
• chore(backend): partially revert 9afe23e by @HumairAK in in #11713
• fix(backend) fix execution-level retry on the Argo Workflows backend by @ntny in #11673
• update driver/launcher images for 2.4.1 by @HumairAK in #11717

Full Changelog: 2.4.0...2.4.1

Version 2.4.0

To deploy Kubeflow Pipelines in an existing cluster, follow the instruction in here

Install python SDK (python 3.9 above) by running:

python3 -m pip install kfp kfp-server-api --upgrade

See the Change Log

What's Changed

• chore: Update RELEASE.md to remove obsolete instructions by @chensun in #11183
• chore: Release kfp-pipeline-spec 0.4.0 by @chensun in #11189
• chore: release kfp-kubernetes 1.3.0 by @chensun in #11190
• chore: update kfp-kubernetes release scripts and instructions by @chensun in #11191
• feat(sdk)!: Pin kfp-pipeline-spec==0.4.0, kfp-server-api>=2.1.0,<2.4.0 by @chensun in #11192
• chore(sdk): release KFP SDK 2.9.0 by @chensun in #11193
• replaced deprecated image repos with registry.k8s.io by @HumairAK in #11152
• chore: Set Python 3.9 as the Minimum Supported Version by @diegolovison in #11159
• chore(docs): clean up kfp-kubernetes documentation index page by @gregsheremeta in #11207
• chore: move release instruction screenshots out of root directory by @chensun in #11215
• fix(ui): Fixes V1 Run detail unnecessary reloading. Fixes #10590 by @quinnovator in #11214
• fix(frontend): first time choosing a pipeline definition is VERY slow. Fixes #10897 by @ElayAharoni in #11130
• chore: clean up obsolete requirements.txt under root directory by @chensun in #11216
• chore(components): GCPC 2.17.0 Release by @copybara-service in #11213
• chore: fix CI by adding a dependency of python/sdk for the e2e-test by @rimolive in #11221
• refactor: fix invalid escape sequences by @TheKevJames in #11147
• fix(backend): upgrade PyYMAL to fix metadata_writer build error by @chensun in #11231
• chore(deps-dev): bump elliptic from 6.5.5 to 6.5.7 in /frontend by @dependabot in #11127
• chore(deps): bump fast-xml-parser and @aws-sdk/credential-providers in /frontend/server by @dependabot in #11056
• chore(deps): bump send and express in /frontend/server by @dependabot in #11229
• chore: add workflow to build images by @HumairAK in #11232
• fix(workflows): patch reversed launcher / driver in workflow matrix by @CarterFendley in #11238
• chore: Remove PaulinaPacyna from reviewers by @PaulinaPacyna in #11242
• test: Fixed kubeflow-pipelines-integration-v2 workflow by @hbelmiro in #11244
• feat(sdk): Allow disabling default caching via a CLI flag and env var by @DharmitD in #11222
• feat(frontend/backend): Allow the ability to sort experiments by last run creation. Fixes #10884 by @ElayAharoni in #11163
• chore: adding HumairAK to root OWNERS by @chensun in #11253
• feat(backend): add configurable S3 path style support by @ntny in #11246
• chore: adding @hbelmiro to backend reviewers by @hbelmiro in #11256
• chore: pinned estroz/rerun-actions version by @hbelmiro in #11257
• feat(backend): Remove PipelineSpec Template storage from ObjStore responsibilies. Fixes #10509 by @gmfrasca in #10790
• fix(backend): randomizing output uri path to avoid overwriting. Fixes #10186 by @b4sus in #11243
• correct lastrun unittest timestamps by @HumairAK in #11270
• chore(deps): bump rollup from 2.63.0 to 2.79.2 in /frontend by @dependabot in #11254
• chore(deps): bump webpack from 5.90.3 to 5.95.0 in /frontend by @dependabot in #11271
• chore(doc): Update kfp-kubenetes release instructions. by @chensun in #11276
• fix(backend): set default value to true for ForcePathStyle by @zazulam in #11281
• fix(components): remove default prediction column names in evaluation regression component to fix issues with bigquery data source by @copybara-service in #11282
• feat(workflows): use built images in Github workflows by @CarterFendley in #11284
• test: Integrate Git Hub Action Results into Tide by @VaniHaripriya in #11262
• chore(deps): bump golang.org/x/net from 0.0.0-20210726213435-c6fcb2dbf985 to 0.23.0 in /test/tools/project-cleaner by @dependabot in #10728
• Revert "chore(deps): bump golang.org/x/net from 0.0.0-20210726213435-… by @hbelmiro in #11297
• chore: update SinglePlatformSpec and ComponentSpec for Vertex Platform speci… by @chenyifan-vertex in #11298
• feat: add fields in SinglePlatformSpec by @chenyifan-vertex in #11299
• test: Resolve ci-check failures in long-running CI tests by @VaniHaripriya in #11295
• test: Fix failing SDK tests by creating Python virtual environment by @VaniHaripriya in #11300
• feat(testing): use kustomize to patch deployments before deploy by @CarterFendley in #11294
• test: Disabled ci-check and add-ci-passed-label workflows. by @hbelmiro in #11306
• chore(components): move old components to contrib folder by @chensun in #11315
• test: Enabled ci-check and add-ci-passed-label workflows by @VaniHaripriya in #11320
• fix(tests): remove redundant integration test wf by @droctothorpe in #11322
• adding chore as a new template. Fixes #11263 by @varodrig in #11317
• Add-Create-Experiment-button-when-selecting-experiement-final by @ElayAharoni in #11332
• feat(components): Add reservation_affinity support in v1.create_custom_training_job_from_component by @copybara-service in #11330
• feat(components)!: Deprecate preview.custom_job module by @copybara-service in #11336
• feat(api): add PipelineConfig to api to re-implement pipeline-level config by @gregsheremeta in #11333
• feat(component): execute in a virtual env by @leseb in #11326
• feat(sdk): add PipelineConfig to DSL to re-implement pipeline-level config by @gregsheremeta in #11112
• chore: fix release note by @chensun in #11337
• fix(components): Fix create_custom_training_job_from_component default location by @copybara-service in #11334
• chore(components): Update docker base image to marketplace.gcr.io/google/ubuntu2404, which will install newer Python by @copybara-service in #11338
• chore(backend): typo in max concurrency error text by @Gkrumbach07 in #11342
• feat(backend): implement subdag output resolution by @droctothorpe in #11196
• chore: upstream our authorization changes for oauth2-proxy by @juliusvonkohout in #11347
• test: Fix ci-passed label update on PR synchronization by @VaniHaripriya in #11358
• chore: Added cached layer for Go dependencies to Dockerfiles by @hbelmiro in #11348
• chore: Update the generated cache proto file by @rimolive in #11359
• feat(sdk)!: stop auto-populating metrics as dag output by @chensun in #11362
• chore(sdk): release KFP SDK 2.10.0 by @chensun in #11364
• chore(components): bump highest support KFP SDK version to 2.10.0 and highest support Python version to 3.13 by @copybara-service in #11369
• chore(sdk): remove kfp.deprecated from sdk, legacy samples, and legacy tests by @chensun in #11366
• chore: Updating Pytorch-Launcher component to work with pipelines v2 by @Fiona-Waters in #11273
• chore(components): remove upper limit on python_requires by @copybara-service in https://g...

Version 2.3.0

To deploy Kubeflow Pipelines in an existing cluster, follow the instruction in here or via UI here

Install python SDK (python 3.7 above) by running:

python3 -m pip install kfp kfp-server-api --upgrade

See the Change Log

What's Changed

• chore: Update RELEASE.md by @chensun in #10767
• fix(components): Add input param autorater_prompt_parameters to online_evaluation_pairwise component by @copybara-service in #10759
• feat(components): create infer preprocessor component by @copybara-service in #10765
• feat(components): create utility class for preprocessors and use it in rlhf preprocessor and infer preprocessor by @copybara-service in #10768
• chore(sdk): drop Python 3.7 for kfp sdk by @rickyxie0929 in #10750
• chore: Promote @rimolive as a backend approver by @chensun in #10780
• feat(components): use preprocessor utility methods for the upload model graph by @copybara-service in #10774
• chore: upgrade node to v18 by @HumairAK in #10794
• feat(GH workflow): migrate periodic functional tests to GH actions by @shruti2522 in #10751
• chore: Add Container Engine and Image Tag parameters to backend Makefiles by @gmfrasca in #10725
• chore: add meaningful logs to client initialization. by @HumairAK in #10801
• feat(kubernetes_platform): Update kubernetes_platform go package to include EnabledSharedMemory by @hsteude in #10703
• feat(components): Retry on batch prediction internal errors in AutoSxS by @copybara-service in #10799
• fix(components): Add staging and temp locations to prophet trainer component by @copybara-service in #10803
• fix(components): Remove unused import function_based from infer pipeline by @copybara-service in #10802
• docs(components): Bump the python version to 3.8 by @copybara-service in #10764
• chore(components): GCPC 2.14.1 Release by @copybara-service in #10813
• chore(components): Rollback GCPC 2.14.1 Release due to b/339911077 by @copybara-service in #10818
• chore(components): Add TaskError proto by @copybara-service in #10823
• chore(components): GCPC 2.14.1 Release by @copybara-service in #10821
• feat(components): Create the write_user_defined_error function by @copybara-service in #10835
• test: fix CI failure for periodic functional tests by @tmvfb in #10817
• chore(docs): Fix link by @haoxins in #7533
• chore: Update Periodic Functional Tests to run once a day by @DharmitD in #10842
• test: fix version conflict for functional tests by @tmvfb in #10837
• chore: server should skip sample load if path DNE by @DharmitD in #10833
• fix(frontend): reduce list run latency by @droctothorpe in #10797
• feat(components): Support parsing Gemini BP outputs in AutoSxS pipeline by @copybara-service in #10853
• feat(components): Use GetModel integration test to manually test write_user_defined_error function by @copybara-service in #10843
• docs: modified the ConfigMap as Env variable example in README.md with right key_name within use_config_map_as_env() block. by @vamsi-01 in #10855
• fix(sdk): Add required auth scopes to RegistryClient for GCP service accounts credentials by @PChambino in #10819
• chore(deps): bump requests from 2.31.0 to 2.32.2 in /backend/src/apiserver/visualization by @dependabot in #10858
• chore: Change command to apply manifests in the docs by @rimolive in #10864
• fix(manifests): Move metacontroller to the top in kustmization.yaml by @codablock in #10669
• test: Migrate Backend tests to a GHA workflow by @DharmitD in #10871
• feat(backend): mount EmptyDir volumes for launcher write locations by @HumairAK in #10857
• docs(components): internal by @copybara-service in #10874
• feat(components): Add role_field_name and model_name as input parameters to llm_evaluation_preprocessor component to support gemini model's input and output schema by @copybara-service in #10875
• fix(components): add check and add log to call out the fallback to the default model checkpoint and remove the model checkpoint check condition in RLHF GCPC by @copybara-service in #10828
• test: migrate kubeflow-pipeline-e2e-test to GitHub Actions by @hbelmiro in #10887
• feat(components): Support dynamic machine type paramters in CustomTrainingJobOp by @KevinGrantLee in #10883
• fix(kubernetes_platform): fix api-generator docker mount for SELinux by @gregsheremeta in #10890
• feat(components): Add Starry Net forecasting pipeline to public preview by @copybara-service in #10888
• chore(components): Update AutoSxS and RLHF image tags by @copybara-service in #10906
• feat(components): Update Starry Net image tags by @copybara-service in #10920
• chore: Updated Python image to 3.8 in backend/Dockerfile by @hbelmiro in #10941
• Expose starry_net yaml to GitHub by @chensun in #10943
• chore(sdk): release KFP SDK 2.8.0 by @chensun in #10940
• chore: Upgrade Python version to 3.8 in readthedocs config by @chensun in #10945
• feat(frontend&backend): Add UI support for object store customization and prefixes by @HumairAK in #10787
• chore(components): Update AutoSxS and RLHF image tags by @copybara-service in #10958
• chore(components): GCPC 2.15.0 Release by @copybara-service in #10951
• docs(backend): Remove deprecated v2 compatibility mode docs by @droctothorpe in #10956
• Upgrade go version to 1.21 by @DharmitD in #10911
• chore: Add ability to mount self-signed certs to kfp by @DharmitD in #10849
• feat(components): add persistent_resource_id to v1 GCPC custom job components/utils by @copybara-service in #10954
• chore: Added @rimolive, @hbelmiro and @DharmitD as approvers and reviewers for GitHub Actions by @hbelmiro in #10963
• test: Migrate KFP SDK YAPF test to a GHA Workflow by @DharmitD in #10961
• test: Created a new GitHub action to reuse the creation of a KFP cluster by @hbelmiro in #10946
• test: kubeflow-pipeline-upgrade-test migrated to GitHub Actions by @hbelmiro in #10932
• test: Migrate Frontend tests to a GHA Workflow by @DharmitD in #10972
• test: install kfp on github periodic functional tests workflow by @tmvfb in #10859
• chore: Add an option to re-run GHA workflows by @rimolive in #10930
• chore(components): Update Starry Net description and add nan- and zero-threshold args by @copybara-service in #10986
• test: Moved KFP backend visualization tests to GitHub Actions by @hbelmiro in #10984
• test: Migrate KFP SDK Upgrade Test to GHA by @DharmitD in #10989
• test: Migrate KFP SDK Runtime Code Test to GHA by @DharmitD in #10991
• chore(components): Bump image version for Structured Data pipelines by @copybara-service in #11000
• chore(components): Fix bug due to protobuf library being upgraded by pinning protobuf version by @copybara-service in #11007
• chore(components): Add target_field_name as input parameters to llm_evaluation_preprocessor component to support gemini model's input and output schema by @copybara-service in https://github.co...

Version 2.2.0

To deploy Kubeflow Pipelines in an existing cluster, follow the instruction in here or via UI here

Install python SDK (python 3.7 above) by running:

python3 -m pip install kfp kfp-server-api --upgrade

See the Change Log

What's Changed

• feat(components): Report TensorBoard metrics for preview.llm.rlhf_pipeline in real time by @copybara-service in #10595
• feat(kubernetes_platform): Update kubernetes_platform go package to include generic ephemerl volume by @abaland in #10602
• fix(metadata envoy): upgrade envoy and config from 1.12 to 1.27 by @freefood89 in #10589
• fix(components): Remove the unused resolve_data_paths from function_based by @copybara-service in #10606
• chore: Add Question issue template by @rimolive in #10557
• feat(kubernetes_platform): Update kubernetes_platform go package to include node affinities and pod (anti)affinities by @cjidboon94 in #10583
• chore(components): Add test machine spec support to preview.llm pipelines by @copybara-service in #10616
• fix(components): Ensure preview.llm.rlhf_pipeline runs if no tensorboard_id is provided by @copybara-service in #10626
• fix(components): Remove the unused functions from function_based by @copybara-service in #10613
• feat(components): Add model name preprocess component; Use publisher model if user uploaded model is non-tuned by @copybara-service in #10620
• feat(components): add task_type as a parameter to rlaif by @copybara-service in #10607
• feat(components): AutoSxS GA pending release by @copybara-service in #10628
• fix(components): Update service account comment by @copybara-service in #10636
• chore(components): GCPC 2.12.0 Release by @copybara-service in #10635
• fix(samples): Fix the loop_output.py example to handle the new parallel loop type requirement by @Tomcli in #10637
• feat(components): Added support for text-bison@002 to preview.llm.rlhf_pipeline by @copybara-service in #10641
• chore(components): GCPC 2.13.0 Release by @copybara-service in #10644
• fix(components): Make AutoSxS autorater_prompt_parameters required by @copybara-service in #10621
• docs(components): internal by @copybara-service in #10649
• chore(components): Update kserve component to v0.12.0 by @Tomcli in #10652
• fix(backend): Update backend common code and integration tests with updated API Service Params by @gmfrasca in #10640
• fix(Backend + SDK): Add missing optional field to SecretAsVolume and … by @revit13 in #10550
• chore(manifests): refactor manifests for kustomize5 compatibility. Part of #10053 by @rawc0der in #10087
• fix(components): Fix model eval import error in text generation/classification eval pipeline by @copybara-service in #10655
• chore(components): GCPC 2.13.1 Release by @copybara-service in #10666
• chore(deps): bump follow-redirects from 1.6.1 to 1.15.6 in /frontend by @dependabot in #10575
• chore(deps): bump pillow from 10.0.1 to 10.3.0 in /backend/src/apiserver/visualization by @dependabot in #10658
• fix(components): Remove the unused generate_default_instruction and resolve_upload_location from function_based by @copybara-service in #10638
• chore(deps): bump express from 4.18.2 to 4.19.2 in /frontend by @dependabot in #10639
• chore: Update argo images to 3.4.16 by @chensun in #10618
• feat(components): Use larger base reward model when tuning text and chat variants of bison@001 with the preview.llm.rlhf_pipeline by @copybara-service in #10663
• chore(deps): bump follow-redirects from 1.5.10 to 1.15.6 in /frontend/server by @dependabot in #10574
• chore(sample): add note about secret needing to be pre-created by @gregsheremeta in #10659
• Chore(components): Clean up old ibm components by @Tomcli in #10680
• chore(components): Update AutoSxS and RLHF image tags by @copybara-service in #10683
• feat(sdk+backend): Add support for generic ephemeral volume by @abaland in #10605
• feat(components): Use larger base reward model when tuning t5-xxl with the preview.llm.rlhf_pipeline by @copybara-service in #10665
• chore(backend): Promote @rimolive as the backend reviewer by @Tomcli in #10689
• chore(sdk): release kfp-kubernetes 1.2.0 by @connor-mccarthy in #10692
• chore(sdk): make kfp-kubernetes release instructions public by @connor-mccarthy in #10693
• chore(kfp-kubernetes): change type of affinity weight to int32 by @cjidboon94 in #10671
• fix(components): Fix image version parameter in rl pipelines by @copybara-service in #10698
• docs(components): internal by @copybara-service in #10694
• feat(backend): Merge kfp-tekton backend code by @rimolive in #10678
• feat(backend): Upgrade argo to v3.4.16 by @gmfrasca in #10568
• feat(backend): add namespace & prefix scoped credentials to kfp-launcher config for object store paths by @HumairAK in #10625
• chore(deps): bump sqlparse from 0.4.4 to 0.5.0 in /backend/src/apiserver/visualization by @dependabot in #10700
• feat(components): add resolve_reference_model_metadata to rlhf_preprocessor component by @copybara-service in #10612
• feat(components): Move AutoSxS pipeline to v1 directory by @copybara-service in #10701
• feat(components): add resolve_machine_spec and resolve_refined_image_uri to rlhf_preprocessor component by @copybara-service in #10608
• fix(components): remove default prediction column names in evaluation classification component to fix incorrect column names for bigquery data source by @copybara-service in #10708
• feat(components): Introduce placeholders: SERVICE_ACCOUNT_PLACEHOLDER, NETWORK_PLACEHOLDER, PERSISTENT_RESOURCE_ID_PLACEHOLDER and ENCYRPTION_SPEC_KMS_KEY_NAME_PLACEHOLDER. In addition, use PERSISTENT_RESOURCE_ID_PLACEHOLDER as the default value of persistent_resource_id for CustomTrainingJobOp and create_custom_training_job_op_from_component. With this change, custom job created without explicitly setting persistent_resource_id will inherit job level persistent_resource_id, if Persistent Resource is set as job level runtime by @copybara-service in #10650
• chore: Add kfp-tekton integration tests and manifests by @rimolive in #10702
• No public description by @copybara-service in #10726
• chore(components): Drop GCPC Python 3.7 by @copybara-service in #10730
• feat(components): use rlhf_preprocessor to replace the current value_exists call in rlhf by @copybara-service in #10584
• chore(components): Drop support for Python 3.7 in GCPC by @copybara-service in #10735
• feat(components): internal by @copybara-service in #10707
• feat(components): Expand regions supported by preview.llm.rlhf_pipeline by @copybara-service in #10710
• chore(components): Update AutoSxS and RLHF image tags by @copybara-service in #10749
• chore(components): Change the warning for Python 3.8 by @copybara-service in #10755
• feat(components): migrate function_based resolve_num_microbatches to rlhf_preprocessor component by @copybara-service in #10604
• chore(components): GCPC 2.14.0 Release by @copybara-service in #10754
• feat(components): migrate function_based convert_to_delimited_string to rlhf_preprocessor component by @copybara-service in https://github.com/kube...

Version 2.1.0

To deploy Kubeflow Pipelines in an existing cluster, follow the instruction in here or via UI here

Install python SDK (python 3.7 above) by running:

python3 -m pip install kfp kfp-server-api --upgrade

See the Change Log

What's Changed

• feat(components): Bump image tag used by preview.llm pipelines by @copybara-service in #10295
• feat(components): Add num_microbatches to _implementation.llm training components by @copybara-service in #10248
• chore(sdk): remove cleanup param in local init #localexecution by @connor-mccarthy in #10293
• feat(sdk): add local execution skeleton #localexecution by @connor-mccarthy in #10292
• fix(components): Use llama-2-7b for the base reward model when tuning llama-2-13 with the preview.llm.rlhf_pipeline by @copybara-service in #10249
• feat(sdk): add local task dispatcher validation and ExecutorInput construction #localexecution by @connor-mccarthy in #10298
• test(sdk): install kfp-pipeline-spec from source in kfp sdk tests by @connor-mccarthy in #10300
• fix(sdk): use kfp.dsl.types to replace kfp.components.types Fixes #10282 by @hsinhoyeh in #10283
• feat(components): Enable text generation pipeline to generate row based metrics by @copybara-service in #10296
• test(sdk): add placeholder_utils_test.py by @connor-mccarthy in #10301
• feat(component): Migrate AutoSxS pipeline to preview and move related files to _implementation/llm directory to help Model Eval team use side by side metrics as part of their pipeline by @copybara-service in #10294
• feat(components): Release new LLM Eval image version 0.5 by @copybara-service in #10313
• chore(components): release GCPC 2.7.0 by @copybara-service in #10303
• fix(components): Add autosxs_pipeline to the all variable for the preview/model_evaluation directory by @copybara-service in #10317
• chore(components): release GCPC 2.8.0 by @copybara-service in #10316
• feat(components): Add better docstrings for AutoSxS by @copybara-service in #10320
• test(components): INTERNAL by @copybara-service in #10314
• feat(sdk): add subprocess task handler #localexecution by @connor-mccarthy in #10302
• fix(components): Use large_model_reference as model_reference_name when uploading models from preview.llm.rlhf_pipeline instead of hardcoding value as text-bison@001 by @copybara-service in #10321
• test(sdk): add kfp-kubernetes execution tests by @connor-mccarthy in #10304
• fix(components): Resolve unique model display name on each preview.llm.rlhf_pipeline run instead of reusing cached result by @copybara-service in #10322
• fix(components): Upload the tuned adapter to Model Registry instead of model checkpoint from preview.llm.rlhf_pipeline by @copybara-service in #10323
• feat(sdk): add local execution output collection #localexecution by @connor-mccarthy in #10325
• chore(sdk): fix use of invalid escape sequence in tests by @connor-mccarthy in #10310
• feat(sdk): add local execution logging #localexecution by @connor-mccarthy in #10326
• chore(components): Fix argument description by @copybara-service in #10327
• feat(backend): preserve querystring in pipeline root (fixes #10318) by @TobiasGoerke in #10319
• feat(components): change output format to allow possible post eval by @copybara-service in #10281
• fix(frontend): Add disableParsingRawHTML option for markdown-to-jsx component by @zijianjoy in #10315
• feat(sdk): add DockerRunner #localexecution by @connor-mccarthy in #10328
• feat(sdk): add special dsl.OutputPath read logic #localexecution by @connor-mccarthy in #10334
• chore(sdk): write local execution logs to stdout #localexecution by @connor-mccarthy in #10330
• feat(sdk): support local Container Component execution #localexecution by @connor-mccarthy in #10333
• feat(components): Update RLHF env vars to handle empty string by @copybara-service in #10331
• feat(components): Output errors as a separate table from Arbiter by @copybara-service in #10329
• docs(components): Fix AutoSxS docstring formatting by @copybara-service in #10341
• chore(components): add json_escape placeholder util by @copybara-service in #10351
• fix(sdk): fix presentation of strings in local execution #localexecution by @connor-mccarthy in #10353
• fix(sdk): remove redundant newline character in local DockerRunner logs #localexecution by @connor-mccarthy in #10354
• test(sdk): improve KFP SDK local runner test safety #localexecution by @connor-mccarthy in #10336
• feat(backend): Enable logging for KFP components by @DharmitD in #10288
• feat(sdk): remove local execution feature flag #localexecution by @connor-mccarthy in #10355
• fix(sdk): permit empty local execution outputs #localexecution by @connor-mccarthy in #10338
• feat(sdk): support Concat and IfPresent placeholder in local container component execution #localexecution by @connor-mccarthy in #10348
• test(sdk): add test for local execution of None default parameter #localexecution by @connor-mccarthy in #10339
• chore(sdk): release KFP SDK 2.5.0 by @connor-mccarthy in #10364
• chore(sdk): depend on protobuf 4 in kfp-pipeline-spec by @connor-mccarthy in #10305
• docs(components): Document AutoML Tables util functions by @copybara-service in #10359
• feat(components): Implement the feature store grounding pipeline by @copybara-service in #10332
• feat(components): update eval pipeline documentation to clarify the required pipeline parameters by @copybara-service in #10366
• docs(components): update BigqueryQueryJobOp docs on container args limit by @copybara-service in #10380
• chore(components): GCPC 2.8.1 release by @copybara-service in #10381
• feat(kubernetes_platform): Update kubernetes_platform go package to include pod labels and annotations by @Tomcli in #10357
• chore(sdk): depend on protobuf 4 in kfp and kfp-kubernetes by @connor-mccarthy in #10307
• feat(components): Implement new output format of inference component by @copybara-service in #10375
• chore(components): rollback GCPC 2.8.1 release by @copybara-service in #10385
• chore(sdk): release KFP SDK 2.6.0 by @connor-mccarthy in #10386
• chore(components): Add v1.model.ModelGetOp components by @copybara-service in #10226
• chore(sdk): release kfp-kubernetes 1.1.0 by @connor-mccarthy in #10387
• chore(sdk): update kfp-kubernetes docs versions and release scripts by @connor-mccarthy in #10388
• docs(sdk): fix kfp-kubernetes docs build error by @connor-mccarthy in #10389
• chore(components): Sync AutoML components by @copybara-service in #10372
• fix(components): Update base image for KFP lightweight component for VPC SC compliance by @copybara-service in #10374
• chore: Fix metrics visualization v2 sample by @rimolive in #10399
• docs(sdk): add kfp.local to reference docs #localexecution by @connor-mccarthy in #10395
• chore(components): migrate GCPC to protobuf 4; require KFP>=2.6.0 by @copybara-service in #10401
• feat(components): Support scheduling and labels in utils.bui...

Version 2.0.5

To deploy Kubeflow Pipelines in an existing cluster, follow the instruction in here or via UI here

Install python SDK (python 3.7 above) by running:

python3 -m pip install kfp kfp-server-api --upgrade

See the Change Log

Version 2.0.4

To deploy Kubeflow Pipelines in an existing cluster, follow the instruction in here or via UI here

Install python SDK (python 3.7 above) by running:

python3 -m pip install kfp kfp-server-api --upgrade

See the Change Log