Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Missing metric for current forging mode on a blockproducer node #5751

Open
gitmachtl opened this issue Apr 4, 2024 · 4 comments · Fixed by #5780
Open

Missing metric for current forging mode on a blockproducer node #5751

gitmachtl opened this issue Apr 4, 2024 · 4 comments · Fixed by #5780
Labels

Comments

@gitmachtl
Copy link
Contributor

gitmachtl commented Apr 4, 2024

With the push to P2P also on the BlockProducer node, SPOs need to change there backup/failover infrastructure.

A test with node 8.9.1 showed, turning the blockproduction off on a node via a SIGHUP signal (and removed credential files), all last metrics of the blockproducer stays. So there is currently no way to detect the forging mode of a running node that way.

We would need an additional metric (accessable via prometheus interface or a cli query) with value 1 = forging and 0 = not forging, to be sure we know what the node is currently doing. And the backup/failover infra can handle the workmode accordingly.

Trying to get this data out of the logfiles is not a nice way, and i am sure we can add this metric to all the other available values.

@gitmachtl gitmachtl added the type: bug Something is not working label Apr 4, 2024
@coot coot added type: feature request Request a new functionality good first issue and removed type: bug Something is not working labels Apr 5, 2024
@coot
Copy link
Contributor

coot commented Apr 5, 2024

We install a SIGHUP handler in cardano-node (see) which contains the necessary information; this only requires adding an EKG counter in the node, which makes it quite easy to implement (no need to modify anywhere deeper in the stack, e.g. ouroboros-consensus).

@gitmachtl
Copy link
Contributor Author

gitmachtl commented Apr 7, 2024

@coot thx, please make sure that the "isForging" EKG/Prometheus metric is also reported correctly as false(0) if the blockproducer node was started with the --non-producing-node option. that should be the default in an active/standby blockproducer backup infra. start it up with the credentials but in non producing mode. check if the state to promote it as an active producer is ok (like is the node on tip, other blockproducers not active), if so, reload the settings via a SIGHUP signal and start forging.

@coot
Copy link
Contributor

coot commented Apr 7, 2024

I don't think I will be working on it, but I'll bring it to the attention of the core team.

@coot coot linked a pull request Apr 25, 2024 that will close this issue
@coot coot changed the title [Bug] - Missing metric for current forging mode on a blockproducer node Missing metric for current forging mode on a blockproducer node Apr 25, 2024
Copy link

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 120 days.

@github-actions github-actions bot added the Stale label May 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
Status: 🔖 Ready
Development

Successfully merging a pull request may close this issue.

2 participants