fix: health should remove prometheus metrics after collectors removal#1302
Merged
Conversation
Signed-off-by: ianisimov <ianisimov@nvidia.com>
Signed-off-by: ianisimov <ianisimov@nvidia.com>
|
🌿 Preview your docs: https://nvidia-preview-pull-request-1302.docs.buildwithfern.com/infra-controller |
poroh
approved these changes
May 1, 2026
rpowers-nv
pushed a commit
to rpowers-nv/ncx-infra-controller-core
that referenced
this pull request
May 5, 2026
…NVIDIA#1302) ## Description After refactoring of Hardware Health service, Prometheus metrics was moved into seprate Sink, this introduced regression when stale metrics was not removed (after Collector stopped). This PR fixes this regression by adding new message which tells all Sinks what collector is removed and they can handle it. Right now only Prometheus collector react to this message ## Type of Change <!-- Check one that best describes this PR --> - [ ] **Add** - New feature or capability - [ ] **Change** - Changes in existing functionality - [x] **Fix** - Bug fixes - [ ] **Remove** - Removed features or deprecated functionality - [ ] **Internal** - Internal changes (refactoring, tests, docs, etc.) ## Related Issues (Optional) NVIDIA#989 ## Breaking Changes - [ ] This PR contains breaking changes <!-- If checked above, describe the breaking changes and migration steps --> ## Testing <!-- How was this tested? Check all that apply --> - [x] Unit tests added/updated - [ ] Integration tests added/updated - [ ] Manual testing performed - [ ] No testing required (docs, internal refactor, etc.) ## Additional Notes <!-- Any additional context, deployment notes, or reviewer guidance --> --------- Signed-off-by: ianisimov <ianisimov@nvidia.com> Signed-off-by: rpowers <rpowers@nvidia.com>
mkoci
added a commit
to mkoci/infra-controller
that referenced
this pull request
May 28, 2026
mkoci
added a commit
to mkoci/infra-controller
that referenced
this pull request
May 28, 2026
mkoci
added a commit
to mkoci/infra-controller
that referenced
this pull request
May 28, 2026
mkoci
added a commit
to mkoci/infra-controller
that referenced
this pull request
May 28, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
After refactoring of Hardware Health service, Prometheus metrics was moved into seprate Sink, this introduced regression when stale metrics was not removed (after Collector stopped). This PR fixes this regression by adding new message which tells all Sinks what collector is removed and they can handle it. Right now only Prometheus collector react to this message
Type of Change
Related Issues (Optional)
#989
Breaking Changes
Testing
Additional Notes