Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enabled multiprocessing in Cerberus monitoring #60

Merged

Conversation

yashashreesuresh
Copy link
Contributor

@yashashreesuresh yashashreesuresh commented May 21, 2020

This commit:

  • Enables Cerberus to perform all checks enabled in the config
    parallelly using multiprocessing.
  • Uses multiprocessing to monitor namespaces in watch_namespaces
    to check for failed components.
  • Uses multiprocessing to collect detailed logs of failed namespaces.
  • Uses multiprocessing to check for pod crash/restart in each
    namespaces during the sleep interval.

The detailed description of how multiprocessing helps in speeding the Cerberus monitoring and the results for the same can be found in this doc.

Fixes #23 as well

@yashashreesuresh
Copy link
Contributor Author

Please have a look @chaitanyaenr @jtaleric @mffiedler

@rht-perf-ci
Copy link

Can one of the admins verify this patch?

@chaitanyaenr chaitanyaenr added the Needs Rebase Needs to be rebased before merging label May 22, 2020
Copy link
Collaborator

@chaitanyaenr chaitanyaenr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work @yashashreesuresh.

start_cerberus.py Outdated Show resolved Hide resolved
@yashashreesuresh yashashreesuresh force-pushed the complete_multiprocessing branch 2 times, most recently from b27ece9 to 17153b1 Compare May 27, 2020 11:49
config/config.yaml Outdated Show resolved Hide resolved
@chaitanyaenr chaitanyaenr removed the Needs Rebase Needs to be rebased before merging label May 27, 2020
@yashashreesuresh yashashreesuresh force-pushed the complete_multiprocessing branch 2 times, most recently from 0d152f5 to 98d4c08 Compare June 1, 2020 08:32
@chaitanyaenr chaitanyaenr added the Needs Rebase Needs to be rebased before merging label Jun 4, 2020
@chaitanyaenr chaitanyaenr removed the Needs Rebase Needs to be rebased before merging label Jun 15, 2020
@yashashreesuresh yashashreesuresh force-pushed the complete_multiprocessing branch 2 times, most recently from e6c8d40 to d9e7d6c Compare June 16, 2020 04:45
@chaitanyaenr chaitanyaenr added the Needs Rebase Needs to be rebased before merging label Jun 22, 2020
Copy link
Collaborator

@chaitanyaenr chaitanyaenr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@chaitanyaenr chaitanyaenr removed the Needs Rebase Needs to be rebased before merging label Jul 8, 2020
This commit:
- Enables Cerberus to perform all checks provided in the config
  parallely using multiprocessing.
- Uses multiprocessing to monitor namespaces in watch_namespaces
  to check for failed components.
- Uses multiprocessing to collect detailed logs of failed namespaces.
- Uses multiprocessing to check for pod crash/restart in each
  namespaces during the sleep interval.
@chaitanyaenr chaitanyaenr merged commit 603a18b into krkn-chaos:master Jul 21, 2020
@yashashreesuresh yashashreesuresh deleted the complete_multiprocessing branch August 19, 2020 06:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Each Watch should become its own thread
3 participants