Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Turn hydra-notify into a daemon #668

Merged
merged 11 commits into from Aug 14, 2019

Conversation

@edolstra
Copy link
Member

commented Aug 12, 2019

hydra-notify is now a daemon that receives events from hydra-queue-runner by listening to PostgreSQL notifications. This makes it a lot more efficient: not starting a hydra-notify process for each event avoids the ~1s startup time loading all the plugins. Also, it allows future plugins to be written in any language as long as they can listen to PostgreSQL notifications, and to run on different machines from the queue runner.

The PostgreSQL notifications are:

  • step_started
  • step_finished
  • build_started
  • build_finished
  • eval_started
  • eval_failed
  • eval_cached
  • eval_added
@AmineChikhaoui

This comment has been minimized.

Copy link
Member

commented Aug 12, 2019

Awesome !

@edolstra edolstra changed the base branch from flake to master Aug 13, 2019

edolstra added some commits Aug 9, 2019

Turn hydra-notify into a daemon
It now receives notifications about started/finished builds/steps via
PostgreSQL. This gets rid of the (substantial) overhead of starting
hydra-notify for every event. It also allows other programs (even on
other machines) to listen to Hydra notifications.
Send notifications when evaluations start/finish/fail
* 'eval_started' has the format '<tmpId>\t<project>\t<jobset>'.

* 'eval_failed' has the format '<tmpId>'. (The cause of the error can
  be found in the database.)

* 'eval_added' has the format '<tmpId>:<evalId>'.
Plugins: Add isEnabled method
Plugins are now disabled at startup time unless there is some relevant
configuration in hydra.conf. This avoids hydra-notify having to do a
lot of redundant work (a lot of plugins did a lot of database queries
*before* deciding they were disabled).

Note: BitBucketStatus users will need to add 'enable_bitbucket_status
= 1' to hydra.conf.
Avoid fetching Projects/Jobsets just to get the name column
In particular, doing a 'select * from Jobsets where ...' must be
avoided, because the 'errormsg' column can be very big.

@edolstra edolstra force-pushed the notifications branch from 14434a5 to 92d8d6b Aug 13, 2019

@edolstra edolstra merged commit f17cd94 into master Aug 14, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants
You can’t perform that action at this time.