feat(helm): add affinity and tolerations to epp-deployment #1504

hhk7734 · 2025-08-29T15:54:23Z

What type of PR is this?

/kind feature

What this PR does / why we need it:

This PR adds support for configuring affinity and tolerations in the epp-deployment chart template.

Which issue(s) this PR fixes:

Fixes #

Does this PR introduce a user-facing change?:

NONE

Signed-off-by: Hyeonki Hong <hhk7734@gmail.com>

netlify · 2025-08-29T15:54:29Z

✅ Deploy Preview for gateway-api-inference-extension ready!

Name	Link
🔨 Latest commit	`d6f2bcc`
🔍 Latest deploy log	https://app.netlify.com/projects/gateway-api-inference-extension/deploys/68b5376f3a57170008b71286
😎 Deploy Preview	https://deploy-preview-1504--gateway-api-inference-extension.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

k8s-ci-robot · 2025-08-29T15:54:31Z

Welcome @hhk7734!

It looks like this is your first PR to kubernetes-sigs/gateway-api-inference-extension 🎉. Please refer to our pull request process documentation to help your PR have a smooth ride to approval.

You will be prompted by a bot to use commands during the review process. Do not be afraid to follow the prompts! It is okay to experiment. Here is the bot commands documentation.

You can also check if kubernetes-sigs/gateway-api-inference-extension has its own contribution guidelines.

You may want to refer to our testing guide if you run into trouble with your tests not passing.

If you are having difficulty getting your pull request seen, please follow the recommended escalation practices. Also, for tips and tricks in the contribution process you may want to read the Kubernetes contributor cheat sheet. We want to make sure your contribution gets all the attention it needs!

Thank you, and welcome to Kubernetes. 😃

k8s-ci-robot · 2025-08-29T15:54:32Z

Hi @hhk7734. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

nirrozenbaum · 2025-08-29T17:48:47Z

Hi @hhk7734,
Thanks for the PR! 🙏🏼

can you please explain the motivation or use case for this change?

kfswain · 2025-08-29T18:24:50Z

I think this is for guiding the EPP image to land on specific nodes, which makes sense. We probably want this

hhk7734 · 2025-08-29T19:16:46Z

@nirrozenbaum @kfswain

I created node groups based on their intended purpose and added labels and taints. To place epp in the desired groups, I added affinity and tolerations.

nirrozenbaum · 2025-08-29T19:47:48Z

@nirrozenbaum @kfswain

I created node groups based on their intended purpose and added labels and taints. To place epp in the desired groups, I added affinity and tolerations.

yeah, makes sense.

nirrozenbaum · 2025-08-29T19:48:19Z

/ok-to-test

nirrozenbaum

overall looks good.

I left just a few nits.
in addition, can you also add the relevant documentation here: https://github.com/kubernetes-sigs/gateway-api-inference-extension/tree/main/config/charts/inferencepool

nirrozenbaum · 2025-08-29T19:49:28Z

config/charts/inferencepool/templates/epp-deployment.yaml

      - name: plugins-config-volume
        configMap:
          name: {{ include "gateway-api-inference-extension.name" . }}
+      {{- with .Values.inferenceExtension.affinity }}


can we replace the “with” with “if”?

Suggested change

{{- with .Values.inferenceExtension.affinity }}

{{- if .Values.inferenceExtension.affinity }}

.Values.inferenceExtension.extraContainerPorts and .Values.inferenceExtension.env also use with. Since this follows the same pattern, I’d suggest keeping it as is. Do you agree?

To be honest, I’d change those to using “if” as well. I less like the use of “placeholders” and empty vars in the values file.
I personally prefer to have these options well documented and removed when not used.
I’ll defer to @kfswain to get his opinion.

d6f2bcc
I modified it to use if instead of with.

nirrozenbaum · 2025-08-29T19:49:53Z

config/charts/inferencepool/templates/epp-deployment.yaml

+      affinity:
+        {{- toYaml . | nindent 8 }}
+      {{- end }}
+      {{- with .Values.inferenceExtension.tolerations }}


ditto

Suggested change

{{- with .Values.inferenceExtension.tolerations }}

{{- if .Values.inferenceExtension.tolerations }}

Fixed here as well.

nirrozenbaum · 2025-08-29T19:50:53Z

config/charts/inferencepool/values.yaml

    - name: v
      value: 1

+  affinity: {}


and then we can remove empty values of affinity and tolerations from values.yaml

I think it should remain to show that affinity and tolerations can be set.

Signed-off-by: Hyeonki Hong <hhk7734@gmail.com>

nirrozenbaum · 2025-09-01T07:52:08Z

@hhk7734 looks great!
I assume it was tested?

hhk7734 · 2025-09-01T08:32:23Z

@hhk7734 looks great! I assume it was tested?

I only tested it with Helm templating, like the example below.

test-values.yaml

inferenceExtension:
  affinity:
    nodeAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
        nodeSelectorTerms:
          - matchExpressions:
              - key: kubernetes.io/arch
                operator: In
                values:
                  - amd64

helm template gaie ./config/charts/inferencepool -f test-values.yaml > test.yaml

test.yaml

# ...
      volumes:
      - name: plugins-config-volume
        configMap:
          name: gaie-epp
      affinity:
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
            - matchExpressions:
              - key: kubernetes.io/arch
                operator: In
                values:
                - amd64
---
# ...

nirrozenbaum · 2025-09-01T08:50:00Z

/lgtm
/approve

Thanks @hhk7734!
Congrats on your first Inference Gateway PR! 🥳

k8s-ci-robot · 2025-09-01T08:50:08Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: hhk7734, nirrozenbaum

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [nirrozenbaum]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

feat(helm): add affinity and tolerations to epp-deployment

0c7442c

Signed-off-by: Hyeonki Hong <hhk7734@gmail.com>

k8s-ci-robot added kind/feature Categorizes issue or PR as related to a new feature. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels Aug 29, 2025

k8s-ci-robot requested review from kfswain and liu-cong August 29, 2025 15:54

k8s-ci-robot added needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels Aug 29, 2025

k8s-ci-robot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Aug 29, 2025

nirrozenbaum reviewed Aug 29, 2025

View reviewed changes

hhk7734 added 2 commits September 1, 2025 15:04

docs(helm): add affinity and tolerations to inferencepool chart README

7b52733

Signed-off-by: Hyeonki Hong <hhk7734@gmail.com>

chore(helm): use if instead of with

d6f2bcc

Signed-off-by: Hyeonki Hong <hhk7734@gmail.com>

hhk7734 force-pushed the epp-chart branch from f839343 to d6f2bcc Compare September 1, 2025 06:04

k8s-ci-robot assigned nirrozenbaum Sep 1, 2025

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Sep 1, 2025

k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Sep 1, 2025

k8s-ci-robot merged commit a634467 into kubernetes-sigs:main Sep 1, 2025
10 checks passed

hhk7734 deleted the epp-chart branch September 1, 2025 08:52

	{{- with .Values.inferenceExtension.affinity }}
	{{- if .Values.inferenceExtension.affinity }}

	{{- with .Values.inferenceExtension.tolerations }}
	{{- if .Values.inferenceExtension.tolerations }}

feat(helm): add affinity and tolerations to epp-deployment #1504

feat(helm): add affinity and tolerations to epp-deployment #1504

Uh oh!

Conversation

hhk7734 commented Aug 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

netlify bot commented Aug 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Deploy Preview for gateway-api-inference-extension ready!

Uh oh!

k8s-ci-robot commented Aug 29, 2025

Uh oh!

k8s-ci-robot commented Aug 29, 2025

Uh oh!

nirrozenbaum commented Aug 29, 2025

Uh oh!

kfswain commented Aug 29, 2025

Uh oh!

hhk7734 commented Aug 29, 2025

Uh oh!

nirrozenbaum commented Aug 29, 2025

Uh oh!

nirrozenbaum commented Aug 29, 2025

Uh oh!

nirrozenbaum left a comment

Choose a reason for hiding this comment

Uh oh!

nirrozenbaum Aug 29, 2025

Choose a reason for hiding this comment

Uh oh!

hhk7734 Aug 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

nirrozenbaum Aug 30, 2025

Choose a reason for hiding this comment

Uh oh!

hhk7734 Sep 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

nirrozenbaum Aug 29, 2025

Choose a reason for hiding this comment

Uh oh!

hhk7734 Sep 1, 2025

Choose a reason for hiding this comment

Uh oh!

nirrozenbaum Aug 29, 2025

Choose a reason for hiding this comment

Uh oh!

hhk7734 Aug 30, 2025

Choose a reason for hiding this comment

Uh oh!

nirrozenbaum commented Sep 1, 2025

Uh oh!

hhk7734 commented Sep 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

nirrozenbaum commented Sep 1, 2025

Uh oh!

k8s-ci-robot commented Sep 1, 2025

Uh oh!

Uh oh!

Uh oh!

hhk7734 commented Aug 29, 2025 •

edited

Loading

netlify bot commented Aug 29, 2025 •

edited

Loading

hhk7734 Aug 30, 2025 •

edited

Loading

hhk7734 Sep 1, 2025 •

edited

Loading

hhk7734 commented Sep 1, 2025 •

edited

Loading