diff --git a/OWNERS_ALIASES b/OWNERS_ALIASES index 7108ddaf6b5..f3e9d7577d6 100644 --- a/OWNERS_ALIASES +++ b/OWNERS_ALIASES @@ -141,6 +141,9 @@ aliases: wg-policy-leads: - JimBugwadia - rficcaglia + wg-serving-leads: + - SergeyKanzhelev + - TBD wg-structured-logging-leads: - mengjiao-liu - pohly diff --git a/liaisons.md b/liaisons.md index 3d9877e1887..fd412174e33 100644 --- a/liaisons.md +++ b/liaisons.md @@ -60,6 +60,7 @@ members will assume one of the departing members groups. | [WG IoT Edge](wg-iot-edge/README.md) | Patrick Ohly (**[@pohly](https://github.com/pohly)**) | | [WG LTS](wg-lts/README.md) | Nabarun Pal (**[@palnabarun](https://github.com/palnabarun)**) | | [WG Policy](wg-policy/README.md) | Patrick Ohly (**[@pohly](https://github.com/pohly)**) | +| [WG Serving](wg-serving/README.md) | TODO TODO (**[@TODO](https://github.com/TODO)**) | | [WG Structured Logging](wg-structured-logging/README.md) | Nabarun Pal (**[@palnabarun](https://github.com/palnabarun)**) | | [Committee Code of Conduct](committee-code-of-conduct/README.md) | Nabarun Pal (**[@palnabarun](https://github.com/palnabarun)**) | | [Committee Security Response](committee-security-response/README.md) | Stephen Augustus (**[@justaugustus](https://github.com/justaugustus)**) | diff --git a/sig-apps/README.md b/sig-apps/README.md index f66544965b3..56c48d97fc5 100644 --- a/sig-apps/README.md +++ b/sig-apps/README.md @@ -59,6 +59,7 @@ subprojects, and resolve cross-subproject technical issues and decisions. The following [working groups][working-group-definition] are sponsored by sig-apps: * [WG Batch](/wg-batch) * [WG Data Protection](/wg-data-protection) +* [WG Serving](/wg-serving) ## Subprojects diff --git a/sig-architecture/README.md b/sig-architecture/README.md index da653feb055..f466eb1111c 100644 --- a/sig-architecture/README.md +++ b/sig-architecture/README.md @@ -59,6 +59,7 @@ The following [working groups][working-group-definition] are sponsored by sig-ar * [WG API Expression](/wg-api-expression) * [WG LTS](/wg-lts) * [WG Policy](/wg-policy) +* [WG Serving](/wg-serving) * [WG Structured Logging](/wg-structured-logging) diff --git a/sig-autoscaling/README.md b/sig-autoscaling/README.md index b55e96f2bf7..759b6c19513 100644 --- a/sig-autoscaling/README.md +++ b/sig-autoscaling/README.md @@ -43,6 +43,7 @@ The Chairs of the SIG run operations and processes governing the SIG. The following [working groups][working-group-definition] are sponsored by sig-autoscaling: * [WG Batch](/wg-batch) +* [WG Serving](/wg-serving) ## Subprojects diff --git a/sig-instrumentation/README.md b/sig-instrumentation/README.md index 9e5b1ffc9a9..f2e24bd46e0 100644 --- a/sig-instrumentation/README.md +++ b/sig-instrumentation/README.md @@ -53,6 +53,7 @@ subprojects, and resolve cross-subproject technical issues and decisions. ## Working Groups The following [working groups][working-group-definition] are sponsored by sig-instrumentation: +* [WG Serving](/wg-serving) * [WG Structured Logging](/wg-structured-logging) diff --git a/sig-list.md b/sig-list.md index 35b5dd885ae..32b17ebd334 100644 --- a/sig-list.md +++ b/sig-list.md @@ -67,6 +67,7 @@ When the need arises, a [new SIG can be created](sig-wg-lifecycle.md) |[IoT Edge](wg-iot-edge/README.md)|[iot-edge](https://github.com/kubernetes/kubernetes/labels/wg%2Fiot-edge)|* Multicluster
* Network
|* [Steve Wong](https://github.com/cantbewong), VMware
* [Cindy Xing](https://github.com/cindyxing), Microsoft
* [Dejan Bosanac](https://github.com/dejanb), Red Hat
|* [Slack](https://kubernetes.slack.com/messages/wg-iot-edge)
* [Mailing List](https://groups.google.com/forum/#!forum/kubernetes-wg-iot-edge)|* APAC WG Meeting: [Wednesdays at 5:00 UTC (every four weeks)](https://zoom.us/j/91251176046?pwd=cmdqclovM3R3eDB1VlpuL1ZGU1hnZz09)
* Regular WG Meeting (Pacific Time): [Wednesdays at 09:00 PT (every four weeks)](https://zoom.us/j/92778512626?pwd=MXhlemwvYnhkQmkxeXllQ0Z5VGs4Zz09)
|[LTS](wg-lts/README.md)|[lts](https://github.com/kubernetes/kubernetes/labels/wg%2Flts)|* Architecture
* Cluster Lifecycle
* K8s Infra
* Release
* Security
* Testing
|* [Jeremy Rickard](https://github.com/jeremyrickard), Microsoft
* [Jordan Liggitt](https://github.com/liggitt), Google
* [Micah Hausler](https://github.com/micahhausler), Amazon
|* [Slack](https://kubernetes.slack.com/messages/wg-lts)
* [Mailing List](https://groups.google.com/a/kubernetes.io/g/wg-lts)|* Regular WG Meeting: [Tuesdays at 07:00 PT (Pacific Time) (biweekly)](https://zoom.us/j/92480197536?pwd=dmtSMGJRQmNYYTIyZkFlQ25JRngrdz09)
|[Policy](wg-policy/README.md)|[policy](https://github.com/kubernetes/kubernetes/labels/wg%2Fpolicy)|* Architecture
* Auth
* Multicluster
* Network
* Node
* Scheduling
* Storage
|* [Jim Bugwadia](https://github.com/JimBugwadia), Kyverno/Nirmata
* [Robert Ficcaglia](https://github.com/rficcaglia), SunStone
|* [Slack](https://kubernetes.slack.com/messages/wg-policy)
* [Mailing List](https://groups.google.com/forum/#!forum/kubernetes-wg-policy)|* Regular WG Meeting: [Wednesdays at 8:00 PT (Pacific Time) (semimonthly)](https://zoom.us/j/7375677271)
+|[Serving](wg-serving/README.md)|[serving](https://github.com/kubernetes/kubernetes/labels/wg%2Fserving)|* Apps
* Architecture
* Autoscaling
* Instrumentation
* Network
* Node
* Scheduling
* Storage
|* [Sergey Kanzhelev](https://github.com/SergeyKanzhelev), Google
* [TBD](https://github.com/TBD), TBD
|* [Slack](https://kubernetes.slack.com/messages/wg-serving)
* [Mailing List](https://groups.google.com/a/kubernetes.io/g/wg-serving)|* Regular Meeting ([Calendar](https://calendar.google.com/calendar/embed?src=TODO)): [TODOs at TODO PT (Pacific Time) (monthly)](https://zoom.us/j/TODO)
|[Structured Logging](wg-structured-logging/README.md)|[structured-logging](https://github.com/kubernetes/kubernetes/labels/wg%2Fstructured-logging)|* API Machinery
* Architecture
* Cloud Provider
* Instrumentation
* Network
* Node
* Scheduling
* Storage
|* [Mengjiao Liu](https://github.com/mengjiao-liu), DaoCloud
* [Patrick Ohly](https://github.com/pohly), Intel
|* [Slack](https://kubernetes.slack.com/messages/wg-structured-logging)
* [Mailing List](https://groups.google.com/forum/#!forum/kubernetes-wg-structured-logging)| ### Committees diff --git a/sig-network/README.md b/sig-network/README.md index a1a9dc2d61d..5df6db5ec57 100644 --- a/sig-network/README.md +++ b/sig-network/README.md @@ -74,6 +74,7 @@ subprojects, and resolve cross-subproject technical issues and decisions. The following [working groups][working-group-definition] are sponsored by sig-network: * [WG IoT Edge](/wg-iot-edge) * [WG Policy](/wg-policy) +* [WG Serving](/wg-serving) * [WG Structured Logging](/wg-structured-logging) diff --git a/sig-node/README.md b/sig-node/README.md index 2ec6273443e..4850473da24 100644 --- a/sig-node/README.md +++ b/sig-node/README.md @@ -54,6 +54,7 @@ subprojects, and resolve cross-subproject technical issues and decisions. The following [working groups][working-group-definition] are sponsored by sig-node: * [WG Batch](/wg-batch) * [WG Policy](/wg-policy) +* [WG Serving](/wg-serving) * [WG Structured Logging](/wg-structured-logging) diff --git a/sig-scheduling/README.md b/sig-scheduling/README.md index edeb5b8a8fd..6b72173f975 100644 --- a/sig-scheduling/README.md +++ b/sig-scheduling/README.md @@ -64,6 +64,7 @@ subprojects, and resolve cross-subproject technical issues and decisions. The following [working groups][working-group-definition] are sponsored by sig-scheduling: * [WG Batch](/wg-batch) * [WG Policy](/wg-policy) +* [WG Serving](/wg-serving) * [WG Structured Logging](/wg-structured-logging) diff --git a/sig-storage/README.md b/sig-storage/README.md index a171fc09771..d9ecb6f3602 100644 --- a/sig-storage/README.md +++ b/sig-storage/README.md @@ -58,6 +58,7 @@ subprojects, and resolve cross-subproject technical issues and decisions. The following [working groups][working-group-definition] are sponsored by sig-storage: * [WG Data Protection](/wg-data-protection) * [WG Policy](/wg-policy) +* [WG Serving](/wg-serving) * [WG Structured Logging](/wg-structured-logging) diff --git a/sigs.yaml b/sigs.yaml index d4d6f76df9f..d28f2a263f4 100644 --- a/sigs.yaml +++ b/sigs.yaml @@ -3467,6 +3467,47 @@ workinggroups: liaison: github: pohly name: Patrick Ohly +- dir: wg-serving + name: Serving + mission_statement: > + Discuss and enhance the support of inference serving for accelerated workloads + in Kubernetes. Make Kubernetes the natural choice for hosting production inference + reliably, and improve all serving workloads along the way. + + charter_link: charter.md + stakeholder_sigs: + - Apps + - Architecture + - Autoscaling + - Instrumentation + - Network + - Node + - Scheduling + - Storage + label: serving + leadership: + chairs: + - github: SergeyKanzhelev + name: Sergey Kanzhelev + company: Google + - github: TBD + name: TBD + company: TBD + meetings: + - description: Regular Meeting ([Calendar](https://calendar.google.com/calendar/embed?src=TODO)) + day: TODO + time: TODO + tz: PT (Pacific Time) + frequency: monthly + url: https://zoom.us/j/TODO + archive_url: https://docs.google.com/document/d/1aExJFtaLnO-TM6_2uILgI8NI0IjOm7FcwLABBKEMEo0/edit + recordings_url: https://www.youtube.com/playlist?list=TODO + contact: + slack: wg-serving + mailing_list: https://groups.google.com/a/kubernetes.io/g/wg-serving + liaison: + github: TODO + name: TODO TODO - dir: wg-structured-logging name: Structured Logging mission_statement: > diff --git a/wg-serving/README.md b/wg-serving/README.md new file mode 100644 index 00000000000..6ec77325b6d --- /dev/null +++ b/wg-serving/README.md @@ -0,0 +1,43 @@ + +# Serving Working Group + +Discuss and enhance the support of inference serving for accelerated workloads in Kubernetes. Make Kubernetes the natural choice for hosting production inference reliably, and improve all serving workloads along the way. + +The [charter](charter.md) defines the scope and governance of the Serving Working Group. + +## Stakeholder SIGs +* [SIG Apps](/sig-apps) +* [SIG Architecture](/sig-architecture) +* [SIG Autoscaling](/sig-autoscaling) +* [SIG Instrumentation](/sig-instrumentation) +* [SIG Network](/sig-network) +* [SIG Node](/sig-node) +* [SIG Scheduling](/sig-scheduling) +* [SIG Storage](/sig-storage) + +## Meetings +*Joining the [mailing list](https://groups.google.com/a/kubernetes.io/g/wg-serving) for the group will typically add invites for the following meetings to your calendar.* +* Regular Meeting ([Calendar](https://calendar.google.com/calendar/embed?src=TODO)): [TODOs at TODO PT (Pacific Time)](https://zoom.us/j/TODO) (monthly). [Convert to your timezone](http://www.thetimezoneconverter.com/?t=TODO&tz=PT%20%28Pacific%20Time%29). + * [Meeting notes and Agenda](https://docs.google.com/document/d/1aExJFtaLnO-TM6_2uILgI8NI0IjOm7FcwLABBKEMEo0/edit). + * [Meeting recordings](https://www.youtube.com/playlist?list=TODO). + +## Organizers + +* Sergey Kanzhelev (**[@SergeyKanzhelev](https://github.com/SergeyKanzhelev)**), Google +* TBD (**[@TBD](https://github.com/TBD)**), TBD + +## Contact +- Slack: [#wg-serving](https://kubernetes.slack.com/messages/wg-serving) +- [Mailing list](https://groups.google.com/a/kubernetes.io/g/wg-serving) +- [Open Community Issues/PRs](https://github.com/kubernetes/community/labels/wg%2Fserving) +- Steering Committee Liaison: TODO TODO (**[@TODO](https://github.com/TODO)**) + + + diff --git a/wg-serving/charter.md b/wg-serving/charter.md new file mode 100644 index 00000000000..ccc86867d7b --- /dev/null +++ b/wg-serving/charter.md @@ -0,0 +1,87 @@ +# WG Serving Charter + +This charter adheres to the conventions described in the [Kubernetes Charter README] and uses +the Roles and Organization Management outlined in [wg-governance]. + +[Kubernetes Charter README]: /committee-steering/governance/README.md + +## Scope + +Discuss and enhance the support for AI/ML inference workloads in Kubernetes. + + +### In scope + +- Gather requirements for serving workloads (inference primarily, but benefiting + other non-batch use cases where possible) that have broad community alignment + from practitioners, distros, and vendors. Provide concrete input to other SIGs + and WGs around needs for identified requirements. Do it in partnership + with existing ecosystem projects like kServe, Seldon, Kaito, and + others to identify, extract, or implement common shared problems (like Kueue + abstracted deferred scheduling for multiple batch frameworks). +- Specific areas of improvement include: + - Directly improve key kubernetes workload controllers when used with + accelerators and the most common inference serving frameworks and model + servers. + - Explore new projects that improve orchestration, scaling, and load balancing + of inference workloads and compose well with other workloads on Kubernetes + - Being able to run serving workloads safely while giving up + available slack capacity to batch frameworks + +### Out of scope + +- Training and batch inference, which are covered by WG Batch. +- Addition of new API kinds that serve a specific models. The focus should be on + general APIs that frameworks can build on top of. +- Ability to describe the workflows for serving workloads is out of scope, + Kubernetes will offer building blocks to MLOps platforms to build those. + +## Stakeholders + +Stakeholders in this working group span multiple SIGs that own parts of the +code in core kubernetes components and addons. + +- SIG Apps as an primary SIG +- SIG Architecture +- SIG Node +- SIG Scheduling +- SIG Autoscaling +- SIG Network +- SIG Instrumentation +- SIG Storage + +## Deliverables + +The list of deliverables include the following high level features: + +- To SIG Apps: + - Ability to express the model serving workloads with easy to understand logical + objects with the ability to scale to multi-host +- To SIG Scheduling and Autoscaling + - Faster scaling up and down + - Ability to preempt workloads +- To SIG Node: + - Runtime support for Pods preemption + - Runtime support for devices partitioning + +## Roles and Organization Management + +This wg adheres to the Roles and Organization Management outlined in [wg-governance] +and opts-in to updates and modifications to [wg-governance]. + +[wg-governance]: /committee-steering/governance/wg-governance.md + +Additionally, the wg commits to: + +- maintain a solid communication line between the Kubernetes groups and the wider CNCF community; +- submit a proposal to the KubeCon/CloudNativeCon maintainers track; + +## Timelines and Disbanding + +As a first mandate, the wg will define a roadmap in the first quarter of operation. +We believe there will be a set of features the Working Group can identify and deliver +that will enable the majority of frameworks operate natively on Kubernetes. + +There is no expectations that the Working Group will be converted into SIG long term, +however, there is a chance that a separate project or a sizeable sub-component of SIG Apps can be +created as a result of a Working Group.