-
Notifications
You must be signed in to change notification settings - Fork 39
Add SHIP Status Dashboard component-monitor deployment #920
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Add SHIP Status Dashboard component-monitor deployment #920
Conversation
Add build script, config, and systemd service for running a dedicated component-monitor instance on the ERT VM to report health status of ERT services to the SHIP Status Dashboard. The component-monitor probes the MCP server health endpoint and reports status to the dashboard API. A separate instance is needed because the app.ci cluster cannot reach the internal VM network.
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
|
/cc @smg247 |
| @@ -0,0 +1,25 @@ | |||
| #!/bin/bash | |||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The component-monitor is available as a public image at: quay.io. The latest tag is automatically updated whenever a change is made.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
that's great, I can extract the latest binary from the image, don't have to build it from the source code
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The binary in the container image is dynamically linked and requires GLIBC_2.32, but RHEL8 ships with GLIBC 2.28. The binary was likely built on a newer base image.
/tmp/component-monitor: /lib64/libc.so.6: version `GLIBC_2.32' not found (required by /tmp/component-monitor)
I have Go 1.24.6 installed on RHEL8 vm, seems like I have to rollback image based logic
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I will update the service file to launch it in container directly. so this file is not needed
Use the pre-built component-monitor image from quay.io/openshiftci instead of compiling from source. This removes the Go toolchain dependency on the VM.
The pre-built container image from quay.io/openshiftci avoids the need to install Go or extract the binary. The systemd service uses podman run with the config and token mounted as volumes.
Remove User directive since cloud-user lacks rootless podman permissions. System-level podman services run as root by default.
|
Updated based on review feedback:
The GLIBC incompatibility with extracting the binary is avoided entirely by running the container image as-is. |
|
@rioliu-rh: all tests passed! Full PR test history. Your PR dashboard. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here. |
Summary
/usr/local/binContext
The SHIP Status Dashboard component-monitor on app.ci cannot reach the internal VM network where the ERT MCP server runs. This deploys a dedicated
ert-component-monitorinstance on the VM itself to probe the local health endpoint and report status to the dashboard.Ref: OCPERT-324