New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Winlogbeat - Test Usage on Windows Event Collector #1031

Closed
andrewkroh opened this Issue Feb 24, 2016 · 9 comments

Comments

Projects
None yet
3 participants
@andrewkroh
Member

andrewkroh commented Feb 24, 2016

I want to make sure users have a good experience when deploying Winlogbeat on a event collector machine. In this architecture event logs from multiple machines are forwarded to the collector and stored in the "Forwarded Events" event log.

In particular, verify that the message field is rendered without error for events from the source machines.

Setup summary:

"enable winrm on the forwarding hosts, open the firewall ports, add the collector machine account to the forwarding machine's local eventviewers group then create the subscription on the collector"

@davidhowell-tx

This comment has been minimized.

Show comment
Hide comment
@davidhowell-tx

davidhowell-tx Mar 4, 2016

No issues with message fields coming from my event collector.

davidhowell-tx commented Mar 4, 2016

No issues with message fields coming from my event collector.

@andrewkroh

This comment has been minimized.

Show comment
Hide comment
@andrewkroh

andrewkroh Mar 4, 2016

Member

@davidhowell-tx Thank you for commenting! Are you using a pull subscription (event collector pulls from event sources) or a push (event sources push to event collector)? And what operating systems are you using?

Member

andrewkroh commented Mar 4, 2016

@davidhowell-tx Thank you for commenting! Are you using a pull subscription (event collector pulls from event sources) or a push (event sources push to event collector)? And what operating systems are you using?

@davidhowell-tx

This comment has been minimized.

Show comment
Hide comment
@davidhowell-tx

davidhowell-tx Mar 4, 2016

My Event Collector is a Windows Server 2012 R2 system, and my source systems are all Windows Server 2012 R2 as well. I am using Source initiated subscriptions, so push.

davidhowell-tx commented Mar 4, 2016

My Event Collector is a Windows Server 2012 R2 system, and my source systems are all Windows Server 2012 R2 as well. I am using Source initiated subscriptions, so push.

@Kevin-Valle

This comment has been minimized.

Show comment
Hide comment
@Kevin-Valle

Kevin-Valle Mar 14, 2016

I am having an issue with the message field not being rendered.
Instead, the message_error field is filled with "The system cannot find the file specified".
The other fields are filled in correctly, the message is just not rendered.

I am running WinlogBeat on an event collector with Source-initiated subscriptions that store into the "Forwarded Events" event log

Kevin-Valle commented Mar 14, 2016

I am having an issue with the message field not being rendered.
Instead, the message_error field is filled with "The system cannot find the file specified".
The other fields are filled in correctly, the message is just not rendered.

I am running WinlogBeat on an event collector with Source-initiated subscriptions that store into the "Forwarded Events" event log

@andrewkroh

This comment has been minimized.

Show comment
Hide comment
@andrewkroh

andrewkroh Mar 14, 2016

Member

I wonder if the format setting could cause that problem. See /cf:FORMAT on https://msdn.microsoft.com/en-us/library/bb736545(v=vs.85).aspx. Maybe try toggling that setting and see if it has any effect.

Member

andrewkroh commented Mar 14, 2016

I wonder if the format setting could cause that problem. See /cf:FORMAT on https://msdn.microsoft.com/en-us/library/bb736545(v=vs.85).aspx. Maybe try toggling that setting and see if it has any effect.

@Kevin-Valle

This comment has been minimized.

Show comment
Hide comment
@Kevin-Valle

Kevin-Valle Mar 14, 2016

Found my problem: The event collector did not have the event manifests from the application that created the original events on the source computer. Installing the application on the event collector resolved the issue. Thanks for responding!

Kevin-Valle commented Mar 14, 2016

Found my problem: The event collector did not have the event manifests from the application that created the original events on the source computer. Installing the application on the event collector resolved the issue. Thanks for responding!

@andrewkroh

This comment has been minimized.

Show comment
Hide comment
@andrewkroh

andrewkroh Jun 14, 2016

Member

I did find a small issue in 5.X when testing. It causes message_error to be always be present but otherwise it is working as expected. See my comment here for more details:
https://discuss.elastic.co/t/winlogbeat-message-error-the-system-cannot-find-the-file-specified/48125/11

I am targeting a fix for that issue for 5.0.0-beta1.

Member

andrewkroh commented Jun 14, 2016

I did find a small issue in 5.X when testing. It causes message_error to be always be present but otherwise it is working as expected. See my comment here for more details:
https://discuss.elastic.co/t/winlogbeat-message-error-the-system-cannot-find-the-file-specified/48125/11

I am targeting a fix for that issue for 5.0.0-beta1.

@andrewkroh

This comment has been minimized.

Show comment
Hide comment
@andrewkroh

andrewkroh Jun 22, 2016

Member

There's one more thing that should be fixed so I'm leaving this open. I think the state that we persist to disk needs to be more than just a single number in order to be able to correctly resume reading the ForwardedEvents log on restart. There are multiple log sources contained in ForwardedEvents and each source has its own record number counter.

If we store a bookmark (XML string) this should allow Winlogbeat to properly resume after restart for the ForwardedEvents log.

Member

andrewkroh commented Jun 22, 2016

There's one more thing that should be fixed so I'm leaving this open. I think the state that we persist to disk needs to be more than just a single number in order to be able to correctly resume reading the ForwardedEvents log on restart. There are multiple log sources contained in ForwardedEvents and each source has its own record number counter.

If we store a bookmark (XML string) this should allow Winlogbeat to properly resume after restart for the ForwardedEvents log.

@andrewkroh andrewkroh reopened this Jun 22, 2016

@andrewkroh

This comment has been minimized.

Show comment
Hide comment
@andrewkroh

andrewkroh Jun 24, 2016

Member

I tested the bookmarks provided by Windows with the ForwardedEvents log and it doesn't help. The bookmarks do not account for the fact that there can be a unique record number iterator for each remote event log source. This may result in some forwarded events not being shipped if Winlogbeat is restarted.

Here's the bookmark it created:

<BookmarkList>
  <Bookmark Channel='ForwardedEvents' RecordId='708' IsCurrent='true'/>
</BookmarkList>

Further confirming this behavior is this thread on the Technet forums.

Member

andrewkroh commented Jun 24, 2016

I tested the bookmarks provided by Windows with the ForwardedEvents log and it doesn't help. The bookmarks do not account for the fact that there can be a unique record number iterator for each remote event log source. This may result in some forwarded events not being shipped if Winlogbeat is restarted.

Here's the bookmark it created:

<BookmarkList>
  <Bookmark Channel='ForwardedEvents' RecordId='708' IsCurrent='true'/>
</BookmarkList>

Further confirming this behavior is this thread on the Technet forums.

@andrewkroh andrewkroh closed this Jun 24, 2016

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment