Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

psc-seq: don't repeat StickyFault every time. #1799

Merged
merged 1 commit into from
May 22, 2024

Conversation

cbiffle
Copy link
Collaborator

@cbiffle cbiffle commented May 22, 2024

We're currently in a situation where the dogfood rack has the power whip feeding PSUs 3-5 disconnected. This means the PSUs are sitting there in faulted state refusing to enable. Since we can't distinguish this from actual PSU failures, currently we're frantically chatting about this situation in the logs. This is annoying, and overwrites other data.

This change alters the behavior to note a sticky fault condition once and then quietly keep trying to resolve it without the spam.

@cbiffle cbiffle requested a review from hawkw May 22, 2024 21:19
@hawkw
Copy link
Member

hawkw commented May 22, 2024

This seems non-compiley --- it looks like you've forgotten the already_sticky field here: https://github.com/oxidecomputer/hubris/actions/runs/9198535640/job/25301455881?pr=1799#step:6:175

Copy link
Member

@hawkw hawkw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems good to me, once it compiles! I think we probably ought to actually add a mechanism for reading out the present state of the rectifiers, as well, but that can be done separately.

We're currently in a situation where the dogfood rack has the power whip
feeding PSUs 3-5 disconnected. This means the PSUs are sitting there in
faulted state refusing to enable. Since we can't distinguish this from
actual PSU failures, currently we're frantically chatting about this
situation in the logs. This is annoying, and overwrites other data.

This change alters the behavior to note a sticky fault condition _once_
and then quietly keep trying to resolve it without the spam.
@cbiffle cbiffle force-pushed the cbiffle/psc-sticky-fault-spam branch from dd33272 to aa57907 Compare May 22, 2024 21:57
@cbiffle cbiffle merged commit 0b1b95c into master May 22, 2024
104 checks passed
@cbiffle cbiffle deleted the cbiffle/psc-sticky-fault-spam branch May 22, 2024 22:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants