xena backports #301
Merged
xena backports #301
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Contributor
markgoddard
commented
Dec 19, 2022
markgoddard
commented
- Add SMART Monitoring with dash and alerts
- Increase job timeout for kolla image build GHA
- Fix oom-killer graph
- Rephrase the match logic for interfaces monitored for package drops
Enabled Textfile collector in node exporter in kolla/globals.yml Added smartmon script as is from the prometheus-community github and then removed NVME support from this script in favour of using the nvme-cli script, which has also been added in. This is because the nvme-cli script provides better metrics than the smartmon script does. The script also adds the serial number of the disk as a label to all SMART metrics. Added a Kayobe custom playbook to easily deploy the script and associated cron job. This playbook installs smartmontool and nvmecli then copies these over to the hosts and sets up a cronjob which runs the scripts and stores the metrics in the docker volume for node exporter. The playbook changes the way the metrics are saved to a file by making use of the mv command as it is atomic. This was needed as at times prometheus would read a partially completed file. Added a prometheus alert to alert when a drive is reported as not healthy for more than 10 minutes. Added a Grafana dashboard to display the number of healthy and unhealthy drives reported in prometheus. (cherry picked from commit d83ecde) Add docs for SMART Monitoring (cherry picked from commit 595429a) Update doc/source/configuration/monitoring.rst Fix kayobe command Co-authored-by: Will Szumski <will@stackhpc.com> (cherry picked from commit 9a5fc53) Update doc/source/configuration/monitoring.rst Fix Spelling Co-authored-by: Will Szumski <will@stackhpc.com> (cherry picked from commit ef25d6f) Add release note (cherry picked from commit 3d4d011) Amend docs and add release note (cherry picked from commit b6cb511) Move SMART prometheus alert to own file (cherry picked from commit b353fd3) Fix typo (cherry picked from commit 611f2fb) fixup
(cherry picked from commit 807e935)
Changes the oom-killer graph from a smoothed irate to a discrete delta function. Change-Id: I2e4a8576c628610409ade4aad2bd98754bec3860 (cherry picked from commit ef1a449)
OVS bridge interfaces drop packets during normal operation. Change the regex to filter out interfaces that don't matter for packet drops. (cherry picked from commit 9c3f15a)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.