New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Improve ncd status logging #13
Conversation
(modification may break parsing by scripts)
- ncd output streamed if >= verbose level - Redundant debug level 3 ncd output suppressed
@@ -563,7 +563,7 @@ sub launch_ncd { | |||
|
|||
my $result = 0; # Assume success | |||
|
|||
my @cmd = ( '/usr/sbin/ncm-ncd', '--configure' ); | |||
my $p = CAF::Process->new(['/usr/sbin/ncm-ncd', '--configure'] , log => $this_app ); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
mayeb make /usr/sbin/ncm-ncd
a conatsnat or readonly and use it here and below?
As for me, the implementation is complete and now ready for merging... after review! |
return; | ||
} | ||
|
||
my $comp_state_dir = $this_app->option('state'); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
so i wasn't aware of the state dir, but i just looked at the default on one of our nodes
[root@node2160 run]# ls -lrt /var/run/quattor-components/
total 4
-rw-r--r-- 1 root root 0 Jul 18 20:31 download
-rw-r--r-- 1 root root 1 Aug 27 10:57 spma
[root@node2160 run]# grep '' /var/run/quattor-components/
[root@node2160 run]# grep '' /var/run/quattor-components/*
/var/run/quattor-components/spma:
so this contains 2 files, one empty one and one with just a newline (not really useful); but they have very different timestamps. we should make sure that this dir is cleaned up etc etc; the documentations suggests it's only cleaned up when components become inactive, so we'd need some support of "recent" failures.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
May be we should fix the documentation but for me (checked on my test system), there is a file left by ncm-ncd only for failed components (components with errors, not warning) and the file is removed after a succesful execution of the component.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not that the file is created when ncm-ncd determined that the component should be run and is empty until it actually ran and encounters some error.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
so what i have is some leftovers i guess. well, if it causes issues, i'll open an issue with ncm-ncd
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The only issue you'll have with leftover if that they will appear as failed components even if they are not... but only in the case they are other failed components (ncm-ncd returns a non 0 exit status). When ncm-ncd succeeds, there is no attempt to look at the state directory.
If all your components ran successfully, for the time being I'd suggest cleaning up this directory manually.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I confirm that ncm-ncd
is removing state files only for components it ran successfully. If there is a file created in the state directory or a state file left from a component no longer part of the configuration, the file will never be removed by ncm-ncd
. It has to be cleaned up manually.
test this please |
ok to test |
test this please |
Merged build triggered. |
Merged build started. |
Merged build finished. |
retest this please |
Merged build triggered. |
Merged build started. |
Merged build finished. |
Last commit address @stdweird's remark about possible empty configuration module state files. If it happens, it ensures the message logged states it explicitely. |
Fixes #12.
In addition, restore the original formatting of ncd status line in
/var/log/ncm-cdispd
to avoid breaking existing parser for this file (I know that line is parsed to raise attention when a component failed).Still work in progress, would like to add the list of failed components as described in #12.