Skip to content

feat(blog): Add new blog featuring adoption of in-place Pod resource updates#976

Open
vitanovs wants to merge 9 commits into
gardener:masterfrom
vitanovs:feat/doc-adoption-of-in-place-pod-resource-updates
Open

feat(blog): Add new blog featuring adoption of in-place Pod resource updates#976
vitanovs wants to merge 9 commits into
gardener:masterfrom
vitanovs:feat/doc-adoption-of-in-place-pod-resource-updates

Conversation

@vitanovs
Copy link
Copy Markdown
Member

How to categorize this PR?

/kind enhancement

What this PR does / why we need it:

This PR features a new brief blog post about the adoption of in-place Pod resource updates in Gardener.

Which issue(s) this PR fixes:

Part of gardener/gardener#12955

vitanovs added 2 commits May 18, 2026 11:41
… updates

This commit features a blog post about the adoption of in-place Pod
resource updates in Gardener.
… section

This commit updates the Vertical Pod Autoscaler section with few wording
improvements.

Also:
- Fix graph edges titles to reflect logic operation rather than
  technical endpoint/function calls.
- Rewording in the in-place updates benefits section.
@netlify
Copy link
Copy Markdown

netlify Bot commented May 18, 2026

Deploy Preview for gardener-docs ready!

Name Link
🔨 Latest commit efc8ed4
🔍 Latest deploy log https://app.netlify.com/projects/gardener-docs/deploys/6a0d7c2156e4be00087b33be
😎 Deploy Preview https://deploy-preview-976--gardener-docs.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.
🤖 Make changes Run an agent on this branch

To edit notification comments on pull requests, go to your Netlify project configuration.

@gardener-prow gardener-prow Bot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. kind/enhancement Enhancement, improvement, extension labels May 18, 2026
@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented May 18, 2026

Important

Review skipped

Review was skipped due to path filters

⛔ Files ignored due to path filters (2)
  • website/blog/2026/05/05-19-in-place-pod-resource-updates.md is excluded by !website/**
  • website/blog/2026/05/images/in-place-pod-resource-updates/vpa-updater-dashboard-overview.png is excluded by !**/*.png, !website/**

CodeRabbit blocks several paths by default. You can override this behavior by explicitly including those paths in the path filters. For example, including **/dist/** will override the default block on the dist directory, by removing the pattern from both the lists.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yml

Review profile: CHILL

Plan: Pro

Run ID: f903eda3-7110-4dc7-a20e-8381a90f0f4b

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@gardener-prow gardener-prow Bot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label May 18, 2026
@gardener-prow
Copy link
Copy Markdown

gardener-prow Bot commented May 18, 2026

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign n-boshnakov for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@vitanovs vitanovs changed the title Feat/doc adoption of in place pod resource updates feat(blog): Add new blog featuring adoption of in-place Pod resource updates May 18, 2026
@gardener-prow gardener-prow Bot added the cla: yes Indicates the PR's author has signed the cla-assistant.io CLA. label May 18, 2026
@vitanovs vitanovs marked this pull request as ready for review May 18, 2026 11:41
@vitanovs vitanovs requested a review from a team as a code owner May 18, 2026 11:41
@gardener-prow gardener-prow Bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label May 18, 2026
Copy link
Copy Markdown
Contributor

@n-boshnakov n-boshnakov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the blog, @vitanovs! I only have a minor suggestion to leave.

Comment thread website/blog/2026/05/05-19-in-place-pod-resource-updates.md Outdated
Co-authored-by: Nikolay Boshnakov <nikolay.boshnakov@sap.com>
@ialidzhikov
Copy link
Copy Markdown
Member

/assign

Copy link
Copy Markdown
Member

@ialidzhikov ialidzhikov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for taking the time write a blog pod about the biggest achievement in VPA since its introduction - in-place updates! 🎉

Suggestions inline ⬇️


![VPA Updater Dashboard Overview](./images/in-place-pod-resource-updates/vpa-updater-dashboard-overview.png)

With sections covering `VerticalPodAutoscaler` resource overviews (segregated by _update mode_) and panels displaying success rates per resource, the new dashboard can be used for both monitoring and generating status reports on applied resource recommendations.
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could add a section with the achievements on our side. We together checked with you that in dev environments 90% of the Pod evictions were replaced with in-place updates.
We should share this success story with concrete numbers and screenshots from the monitoring dashboard. Maybe the number is even bigger now.

- __Reduced scheduling overhead__: No need to re-schedule `Pod`s across the cluster
- __Reduced initialization overhead__: Applications __do not__ go through full initialization all over again
- __Preserved `Pod` identity__: `Pod` names, IPs, and volumes remain unchanged
- __Improved resource efficiency__: More granular and responsive resource optimization
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One more strong benefit is the following. Not all workload can gracefully handle restarts/evictions due to bugs or limitations. IMO, we should link issues we were facing that were caused by such evictions.
One such issue is kubernetes/kubernetes#126921. We had VPA for the csi-node-plugin DaemonSet. VPA was evicting the csi-node-plugin Pods to update their resources. Deletion of the csi-node-plugin Pod removes the driver information (with the max allocatable disks) from the CSINode object. On startup, the csi-node-plugin adds the driver to the CSINode object. In the time window the csi-node-plugin is down, the CSINode object does not have information about the volume attachment limits. This makes kube-scheduler to schedule Pods with volumes to Nodes which already exhausted their volume attachment limit. This ends up in Pod PVC to stuck in attaching forever until an operator fixes the issue.
Another major bug that occurred in the past due to eviction/restart: kubernetes/autoscaler#7726.
We also had number of other bugs due to the restarts but we could only mention the above occurrences.


Having the ability to bypass the rollout process when updating `Pod` resources drastically improves the scaling efficiency. Eliminating the overhead of `Pod` scheduling and application initialization is among the primary benefits of the new _update_ mechanism. The following points summarize the key factors when considering using _in-place_ updates:

- __Zero-downtime scaling__: Resources are adjusted without `Pod` recreation or service interruption
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here, we could make it stronger by explaining the components in Gardener which have downtime with Pod evictions:

  • The non-HA Shoot's etcd - when the single replica etcd pod is evicted, the whole control place has downtime
  • vali and prometheus run single replica no matter the Shoot is HA or not. Any eviction by VPA of vali/prometheus results in short downtime of the logging/monitoring backend.

Comment thread website/blog/2026/05/05-19-in-place-pod-resource-updates.md Outdated
Comment thread website/blog/2026/05/05-19-in-place-pod-resource-updates.md Outdated
Comment thread website/blog/2026/05/05-19-in-place-pod-resource-updates.md Outdated
Comment thread website/blog/2026/05/05-19-in-place-pod-resource-updates.md Outdated
Comment thread website/blog/2026/05/05-19-in-place-pod-resource-updates.md Outdated
Comment thread website/blog/2026/05/05-19-in-place-pod-resource-updates.md Outdated
Like any cluster-wide configuration change, migrating the `vpa` resources' _update mode_ presented a unique challenge that solved this by leveraging the [Gardener Resource Manager](https://github.com/gardener/gardener/blob/master/docs/concepts/resource-manager.md) and its extensible architecture.

We developed a dedicated `MutatingWebhook` that automatically filters relevant `vpa` resources and applies the _update mode_ change, making the migration seamless. The webhook is deployed through the `VPAInPlaceUpdates` _feature gate_ (available in both `gardenlet` and [Gardener Operator](https://github.com/gardener/gardener/blob/master/docs/concepts/operator.md)).
We also built a _rollback_ mechanism to ensure safety—if the feature gate is disabled, migration scripts automatically revert the _update mode_ changes during `gardenlet` or `operator` initialization. These changes have been available since Gardener [v1.137](https://github.com/gardener/gardener/releases/tag/v1.137.0), making _in-place_ update mode adoption fully operational and ready for production use.
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"Migration scripts automatically revert…during initialization" — calling them "scripts" undersells what's happening; this is reconciliation logic in gardenlet/operator startup. Consider: "the gardenlet / operator initialization logic automatically reverts the update mode."

vitanovs and others added 6 commits May 20, 2026 12:15
Co-authored-by: Ismail Alidzhikov <9372594+ialidzhikov@users.noreply.github.com>
Co-authored-by: Ismail Alidzhikov <9372594+ialidzhikov@users.noreply.github.com>
Co-authored-by: Ismail Alidzhikov <9372594+ialidzhikov@users.noreply.github.com>
Co-authored-by: Ismail Alidzhikov <9372594+ialidzhikov@users.noreply.github.com>
Co-authored-by: Ismail Alidzhikov <9372594+ialidzhikov@users.noreply.github.com>
Co-authored-by: Ismail Alidzhikov <9372594+ialidzhikov@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cla: yes Indicates the PR's author has signed the cla-assistant.io CLA. kind/enhancement Enhancement, improvement, extension size/L Denotes a PR that changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants