Skip to content

Commit

Permalink
fixup! Add wg-manifests
Browse files Browse the repository at this point in the history
  • Loading branch information
yanniszark committed Oct 28, 2020
1 parent 4f88e4a commit f1c7591
Showing 1 changed file with 20 additions and 43 deletions.
63 changes: 20 additions & 43 deletions wg-manifests/charter.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,79 +7,56 @@ outlined in [wg-governance].

- Provide a catalog (centralized repository) of Kubeflow application manifests.
- Provide a catalog of third-party apps for common services.
- Provide documentation to help users install a set, or all of the included
apps.

### In scope

#### Code, Binaries and Services

- Maintain a set of manifests that will allow users to install Kubeflow apps and
- Maintain tooling to automate copying manifests from upstream app repos.
- Maintain a catalog that will allow users to install Kubeflow apps and
common services easily on Kubernetes, either on the cloud or on-prem, without
depending on external cloud services or closed source solutions. These
manifests will be developed with certain principles in mind, which are
outlined in the last bullet. Those include:
depending on external cloud services or closed source solutions. Those
manifests are deployed using `kubectl` and `kustomize` and include:
1. A common set of manifests for the current five (5) official Kubeflow
applications:
- Training Operators
- Kubeflow Pipelines (KFP)
- Notebooks
- KFServing
- Katib
1. Manifests for a set of specific common services:
- Istio
- KNative
- Dex
- Cert-Manager
1. Manifests for common, currently unmaintained, apps:
- Central Dashboard
- Profile Controller
- PodDefaults Controller
- Maintain documentation and instructions for installing a set or all of the
apps. Documentation will not depend on any opinionated deployment tool. It
will only depend on `kustomize` and `kubectl`. This way, users and platform
owners can easily build on top and use their favorite opinionated deployment
tool (e.g., Argo, kapp, kpt, kfctl, FluxCD, etc.) with Kubeflow.
- The maintained manifests will follow certain principles:
- The flow of work / separation of responsibilities will be the following:
1. Application owners publish manifests in their repos.
1. WG copies and tracks upstream manifests in the manifests repo. They
form a base `kustomization`.
1. All kubeflow-specific changes (e.g., change the namespace) are done
in `kustomize` overlays and they don't touch the upstream files.
1. Periodically, the upstream manifests are updated by copying from a
later commit.
- Contrasted with the current state, in the aforementioned workflow:
- App owners are not required to do anything other than maintain working
installation manifests in their repo. This is something all app owners
already do, as it's needed for testing and developing the app.
- Manifests depend on the frequently updated manifests of the app repo,
instead of the out-of-date, kfctl-specific manifests in the current
`kubeflow/manifests` repo. [A recent proposal](https://www.google.com/url?q=http://bit.ly/kf_kustomize_v3&sa=D&ust=1603724692328000&usg=AOvVaw2qgtPzKUz5zqIjpn3Yoas7)
tried to make all of our manifests kustomize-native. However, it
reused the outdated kfctl-specific bases, instead of establishing a
procedure where upstream changes can easily find their way in
`kubeflow/manifests`. Thus, it suffered from poor maintainability.
- Users (ML Engineers) and platform owners can extend and re-use
kustomizations to integrate better with their platforms.
1. Manifests for a set of specific common services:
- Istio
- KNative
- Dex
- Cert-Manager

#### Cross-cutting and Externally Facing Processes

##### With Application Owners

- Aid applications owners in creating kustomize manifests for their application,
inside the app repo, if those don't exist already.
- Communicate with application owners to agree upon the version they want to be
included in the next Kubeflow release.

##### With Platform Owners
##### With Distribution Owners

- Coordinate with platform owners, to make sure they are in-sync about the
release schedule and have time to test and bring their platforms up-to-date.
- Coordinate with distribution owners, to make sure they are in-sync about the
release schedule and have time to test and bring their distributions
up-to-date.

### Out of scope

This WG is NOT going to:
- Maintain deployment-specific tools like `kfctl`.
- Maintain platform-specific manifests.
- Maintain distribution-specific manifests.
- Decide which applications to include in Kubeflow.
- Decide which variant of an application to include (e.g., KFP Standalone vs
KFP with Istio).
- Create and maintain one or more Kubeflow distributions.
- Support configurations with environment-specific requirements, like special
hardware, different versions of third-party apps (e.g., Istio, KNative, etc.)
or custom OIDC providers.
Expand Down

0 comments on commit f1c7591

Please sign in to comment.