Reconnect EventLog and EventChannel when service is restarted #3836
Conversation
Dr Memory
|
There was a problem hiding this comment.
There is an inconsistency between the visibility of the message given when the service stops using the eventchannel or eventlog logformat.
In Eventlog mode, we receive the following debug message:
Error reading event log: 1722
wazuh/src/logcollector/read_win_el.c
Line 527 in 9201024
However, if we use the eventchannel mode, we will see the following error message:
Could not EvtSubscribe() for (Security) which returned (1722)
wazuh/src/logcollector/read_win_event_channel.c
Lines 656 to 659 in 9201024
I think that both messages should give the same information and have the same visibility (debug or error).
src/logcollector/read_win_el.c
Outdated
|
|
||
| /* Event log was closed and re-opened */ | ||
| else if (id == ERROR_INVALID_HANDLE) { | ||
| mdebug1("EventLog service has been restarted. Trying to reconnect '%s' channel...", el->name); |
There was a problem hiding this comment.
Are we sure here that the service is restarted? or just that is not running? Let's try to don't finish the log messages with suspension dots.
There was a problem hiding this comment.
This error is prompted when EventLog is restarted and Wazuh tries to read a message from it. I'm not aware whether it could appear if something else happens (we haven't seen that behavior before). Anyway, the message could be changed just in case.
src/logcollector/read_win_el.c
Outdated
| else if (id == RPC_S_SERVER_UNAVAILABLE || id == RPC_S_UNKNOWN_IF) { | ||
| /* Prevent message flooding when EventLog is stopped */ | ||
| if (counter == 0) { | ||
| mwarn("Eventlog is down. Please restart the service."); |
There was a problem hiding this comment.
Change Please restart the service by Unable to collect logs from channel %s.
| @@ -522,12 +523,22 @@ DWORD WINAPI event_channel_callback(EVT_SUBSCRIBE_NOTIFY_ACTION action, os_chann | |||
| { | |||
| if (action == EvtSubscribeActionDeliver) { | |||
| send_channel_event(evt, channel); | |||
| } else { | |||
| while(1) { | |||
There was a problem hiding this comment.
Please check whether this blocking condition causes blocking in the reading of other channels.
There was a problem hiding this comment.
You cannot read other channels if EventLog is down. If you mean logcollector in general it will still work (I tried with a localfile and it'll work).
| if (query) { | ||
| /* Create copy of query string */ | ||
| if ((channel->query = strdup(query)) == NULL) { | ||
| mferror( |
There was a problem hiding this comment.
Why using mferror instead of merror. If they work similar, use merror like the rest of the module.
There was a problem hiding this comment.
mferror is used just lines above that's why I decided to used it.
|
Now we can define a reconnection time to Use the label TestChange the localfile configuration in <localfile>
<location>Application</location>
<log_format>eventchannel</log_format>
<reconnect_time>40</reconnect_time>
</localfile>
<localfile>
<location>Security</location>
<log_format>eventchannel</log_format>
</localfile>
<localfile>
<location>System</location>
<log_format>eventchannel</log_format>
<reconnect_time>20</reconnect_time>
</localfile>When the eventchannel service is down, it will show the following message in the agent |
|
PR with the modification of the documentation for this function: wazuh/wazuh-documentation#1859 |
There was a problem hiding this comment.
The new option reconnect_time must be displayed in the configuration request. Please, add it to the _getLocalfilesListJSON function.
wazuh/src/logcollector/config.c
Line 100 in f3fcc09
Take into account that this field should only appear in those blocks whose logformat is event-channel (for Windows agents).
This should also apply for the only-future-events option. Take the opportunity to correct this.
We can see that the only-future-events option is appearing in a syslog block in the following API output:
{
"error": 0,
"data": {
"localfile": [
{
"file": "Application",
"logformat": "eventchannel",
"ignore_binaries": "no",
"target": [
"agent"
],
"frequency": 999,
"only-future-events": "yes"
},
{
"file": "Security",
"logformat": "eventchannel",
"query": "Event/System[EventID != 5145 and EventID != 5156 and EventID != 5447 and\n EventID != 4656 and EventID != 4658 and EventID != 4663 and EventID != 4660 and\n EventID != 4670 and EventID != 4690 and EventID != 4703 and EventID != 4907 and\n EventID != 5152 and EventID != 5157]",
"ignore_binaries": "no",
"target": [
"agent"
],
"frequency": 999,
"only-future-events": "yes"
},
{
"file": "System",
"logformat": "eventchannel",
"ignore_binaries": "no",
"target": [
"agent"
],
"frequency": 999,
"only-future-events": "yes"
},
{
"file": "active-response\\active-responses.log",
"logformat": "syslog",
"ignore_binaries": "no",
"target": [
"agent"
],
"only-future-events": "yes"
},
{
"file": "C:\\logs\\test.log",
"logformat": "syslog",
"ignore_binaries": "no",
"target": [
"agent"
],
"only-future-events": "yes"
}
]
}
}
|
Notice that the configuration on-demand shows the |
|
The # curl -u foo:bar -k -X GET "http://10.0.0.1:55000/agents/014/config/logcollector/localfile?pretty"
{
"error": 0,
"data": {
"localfile": [
{
"file": "Application",
"logformat": "eventchannel",
"ignore_binaries": "no",
"target": [
"agent"
],
"only-future-events": "yes",
"reconnect_time": 40
},
{
"file": "Security",
"logformat": "eventchannel",
"query": "Event/System[EventID != 5145 and EventID != 5156 and EventID != 5447 and\n EventID != 4656 and EventID != 4658 and EventID != 4663 and EventID != 4660 and\n EventID != 4670 and EventID != 4690 and EventID != 4703 and EventID != 4907 and\n EventID != 5152 and EventID != 5157]",
"ignore_binaries": "no",
"target": [
"agent"
],
"only-future-events": "yes",
"reconnect_time": 5
},
{
"file": "System",
"logformat": "eventchannel",
"ignore_binaries": "no",
"target": [
"agent"
],
"only-future-events": "yes",
"reconnect_time": 20
},
{
"file": "active-response\\active-responses.log",
"logformat": "syslog",
"ignore_binaries": "no",
"target": [
"agent"
]
},
{
"file": "C:\\logs\\test.log",
"logformat": "syslog",
"ignore_binaries": "no",
"target": [
"agent"
]
},
{
"logformat": "command",
"command": "df -P",
"alias": "df -P",
"ignore_binaries": "no",
"target": [
"agent"
],
"frequency": 360
},
{
"logformat": "full_command",
"command": "df -P",
"alias": "df -P",
"ignore_binaries": "no",
"target": [
"agent"
],
"frequency": 360
}
]
}
}@wazuh/frontend the new tag |
JcabreraC
force-pushed
the
fix-eventlog-restart-3792
branch
from
3751f68
to
2419d5f
Compare
src/config/localfile-config.c
Outdated
| @@ -136,6 +138,13 @@ int Read_Localfile(XML_NODE node, void *d1, __attribute__((unused)) void *d2) | |||
| logf[pl].out_format[n]->target = target ? strdup(target) : NULL; | |||
| os_strdup(node[i]->content, logf[pl].out_format[n]->format); | |||
| logf[pl].out_format[n + 1] = NULL; | |||
| } else if (strcmp(node[i]->element, xml_localfile_reconnect_time) == 0) { | |||
| int time = atoi(node[i]->content); | |||
There was a problem hiding this comment.
Let's use the standard format in Wazuh for timing settings, allowing a suffix such as s, m, h...
| @@ -537,9 +550,10 @@ void win_start_event_channel(char *evt_log, char future, char *query) | |||
| EVT_HANDLE bookmark = NULL; | |||
| EVT_HANDLE result = NULL; | |||
| int status = 0; | |||
| static int counter = 0; | |||
There was a problem hiding this comment.
Same here: Logcollector is multithreading so using a static variable here is thread-unsafe unless it is protected by mutexes when writing and reading it. Let's protect that variable or look for another way to show the log just once.
|
Hi I have solved in 'eventchannel' the problem of the multiple messages that notified @cristgl. Regards. |
SalvaMilenario
force-pushed
the
fix-eventlog-restart-3792
branch
2 times, most recently
from
606a3aa
to
9bb62c3
Compare
|
Hi I have been repeating the test that @cristgl made. I have reproduced the problem of the message In addition, I have added the standard format in Wazuh for timing settings in the option Regards. |
src/config/localfile-config.h
Outdated
| @@ -14,6 +14,7 @@ | |||
| #define EVENTLOG "eventlog" | |||
| #define EVENTCHANNEL "eventchannel" | |||
| #define DATE_MODIFIED 1 | |||
| #define MIN_EVENTCHANNEL_REC_TIME 5 | |||
There was a problem hiding this comment.
Changing this define to DEFAULT_EVENTCHANNEL_REC_TIME. The minimum allowed value will be one second.
chemamartinez
force-pushed
the
fix-eventlog-restart-3792
branch
from
3dfcd05
to
9f009b1
Compare
Description
This PR fixes the error occurred while stopping the
EventLogservice in Windows. If alocalfilehadeventchanneloreventlogaslog_format. Ifeventlogwas set, the following message was being shown infinitely:And the log collection function didn't work again for these logs (though the
EventLogservice is restarted.In the case of
eventchannelno message was being shown but the log collection function wasn't working again for these logs either.Configuration options
eventchannel:eventlog:Logs/Alerts example
eventlog:After this, we trigger a
eventlogevent and the alerts appears:eventchannel:After this, we trigger a
eventchannelevent and the alerts appears:Tests