Windows Events Not Being Decoded With Eventchanel

[mstarks01]

When using eventchannel over eventlog, no Windows alerts will be logged in alerts.log and alerts sent. So, basically, it's not usable.

The issue seems to be that the log is sent in a multi-line format. From archives.log:

2014 Jun 08 10:20:16 (promenade) 192.168.1.12->WinEvtLog 2014 Jun 08 10:14:47 WinEvtLog: Security: Information(4732): no source: no user: no domain: promenade: A member
 was added to a security-enabled local group.

Subject:
        Security ID:            S-1-5-21-4082012827-2132021701-1268594856-1002
        Account Name:           me
        Account Domain:         promenade
        Logon ID:               0x2926e5b

Member:
        Security ID:            S-1-5-21-4082012827-2132021701-1268594856-501
        Account Name:           -

Group:
        Security ID:            S-1-5-32-544
        Group Name:             Administrators
        Group Domain:           Builtin

Additional Information:
        Privileges:             -

When using eventlog as the format, the line looks like this:

2014 Jun 08 11:10:57 (promenade) 192.168.1.12->WinEvtLog 2014 Jun 08 11:06:11 WinEvtLog: Security: AUDIT_SUCCESS(4732): Microsoft-Windows-Security-Auditing: (no user): no domain: promenade: A member was added to a security-enabled local group. Subject:  Security ID:  S-1-5-21-4082012827-2132021701-1268594856-1002  Account Name:  me  Account Domain:  promenade  Logon ID:  0x2926e5b  Member:  Security ID:  S-1-5-21-4082012827-2132021701-1268594856-501  Account Name:  -  Group:  Security ID:  S-1-5-32-544  Group Name:  Administrators  Group Domain:  Builtin  Additional Information:  Privileges:  -

And everything works normally.

[theflakes]

Seeing this issue as well with the 2.8 client. Logs with new lines in them from the System, application, and security logs are sent ok. But, when using eventchannel to subscribe the OSSEC client to an Application and Services log each line of a multi-line event is treated as a seperate log entry. For example event ID 1149 in the Microsoft-Windows-TerminalServices-RemoteConnectionManager/Operational log.

2014 Aug 20 13:33:13 (testServer) 1.2.3.4->WinEvtLog 2014 Aug 20 09:33:13 WinEvtLog: Microsoft-Windows-TerminalServices-RemoteConnectionManager/Operational: Information(1
149): no source: NETWORK SERVICE: NT AUTHORITY: testServer.com: Remote Desktop Services: User authentication succeeded:

User: username
Domain: testDomain
Source Network Address: 1.2.3.4

[awiddersheim]

@gaelmuller Any chance you could take a look at this? Seems like a fairly significant issue.

[defensivedepth]

What can I do to help get this issue fixed?

[jrossi]

If you know C programming drive in. Some Google and playing around and it could be solved.

If you don't know C, but still want to help and are willing to learn C. I will help and where I can and we can do pair programming over the internet on this problem and some other problems that simpler and easier to being someone up to speed with.

Not enough time yourself you can start looking for people that might want to help in the above two ways. I will do just about anything I can help people get up to speed on development of ossec.

[defensivedepth]

Unfortunately I don't have strong C experience, nor the time to dive into it, but I could potentially offer a financial incentive to someone to take care of this...

[awiddersheim]

Sent you an email.

[awiddersheim]

I made a first pass at this over in awiddersheim@d890425. The output of the old eventlog and new eventchannel more closely match one another now but there are still slight differences. I'm not sure how much affect those differences have on the decoders and what people need to do. Frankly, I'm not very good with that part of OSSEC so any help would be appreciated.

Here is some sample output of the two right on top of one another. The eventlog format is first and the eventchannel output is second.

2014/11/22 17:12:02 2014 Nov 22 17:12:01 WinEvtLog: Security: AUDIT_SUCCESS(4732): Microsoft-Windows-Security-Auditing: (no user): no domain: example.server.com: A member was added to a security-enabled local group. Subject:  Security ID:  S-1-5-21-1674808879-2897079356-3500711641-6763  Account Name:  awiddersheim  Account Domain:  BLAH  Logon ID:  0xf0384b  Member:  Security ID:  S-1-5-21-2142426957-3221971033-3828417012-1024  Account Name:  -  Group:  Security ID:  S-1-5-32-545  Group Name:  Users  Group Domain:  Builtin  Additional Information:  Privileges:  -
2014/11/22 17:11:01 2014 Nov 22 17:11:00 WinEvtLog: Security: Information(4732): no source: no user: no domain: example.server.com: A member was added to a security-enabled local group.    Subject:   Security ID:  S-1-5-21-1674808879-2897079356-3500711641-6763    Account Name:  awiddersheim     Account Domain:  BLAH   Logon ID:  0xF0384B    Member:      Security ID:  S-1-5-21-2142426957-3221971033-3828417012-1023    Account Name:  -    Group:      Security ID:  S-1-5-32-545      Group Name:  Users      Group Domain:  Builtin    Additional Information:   Privileges:  -

[awiddersheim]

Made some more improvements to get these more close aligned. Here is what the latest output when using eventchannel looks like:

2014/11/23 12:16:00 2014 Nov 23 12:15:59 WinEvtLog: Security: Information(4732): Microsoft-Windows-Security-Auditing: (no user): no domain: example.server.com.: A member was added to a security-enabled local group.    Subject:   Security ID:  S-1-5-21-1674808879-2897079356-3500711641-6763   Account Name:  awiddersheim   Account Domain:  BLAH Logon ID:  0x1A59C5B    Member:   Security ID:  S-1-5-21-2142426957-3221971033-3828417012-1041   Account Name:  -    Group:   Security ID:  S-1-5-32-545   Group Name:  Users   Group Domain:  Builtin    Additional Information:   Privileges:  -

There are still differences but they are mostly white space. I'm hoping they are able to handle that extra white space or can at least be made to.

https://github.com/awiddersheim/ossec-hids/compare/fix_windows_event_channel

I've been using nxlog pretty heavily as a source of inspiration and mostly a place to steal. They have some code to get the event types right that we might be able to use.

[awiddersheim]

Made some more changes. Think things are ready for inclusion into the main line OSSEC code. The differences between eventchannel and eventlog are now only white space. Here is the log comparison now after all changes.

2014/12/01 14:06:01 2014 Dec 01 14:05:59 WinEvtLog: Security: AUDIT_SUCCESS(4732): Microsoft-Windows-Security-Auditing: (no user): no domain: example.server.com: A member was added to a security-enabled local group.    Subject:   Security ID:  S-1-5-21-1674808879-2897079356-3500711641-6763   Account Name:  awiddersheim   Account Domain:  BLAH   Logon ID:  0x4AB6A5D    Member:   Security ID:  S-1-5-21-2142426957-3221971033-3828417012-1062   Account Name:  -    Group:   Security ID:  S-1-5-32-545   Group Name:  Users   Group Domain:  Builtin    Additional Information:   Privileges:  -
2014/12/01 14:14:27 2014 Dec 01 14:14:26 WinEvtLog: Security: AUDIT_SUCCESS(4732): Microsoft-Windows-Security-Auditing: (no user): no domain: example.server.com: A member was added to a security-enabled local group. Subject:  Security ID:  S-1-5-21-1674808879-2897079356-3500711641-6763  Account Name:  awiddersheim  Account Domain:  BLAH  Logon ID:  0x4ab6a5d  Member:  Security ID:  S-1-5-21-2142426957-3221971033-3828417012-1063  Account Name:  -  Group:  Security ID:  S-1-5-32-545  Group Name:  Users  Group Domain:  Builtin  Additional Information:  Privileges:  -

[mstarks01]

Thanks for your work. This should benefit a lot of people. This is why the OSSEC log format needs to be standardized and documented. It's a bit ironic that third-party integrators would have to reverse-engineer the format considering the pain OSSEC decoder writers (myself included) have gone through trying to bring in support for other products.

[awiddersheim]

Yeah, I know @jrossi has been talking about coming up with a new standard using JSON and experimenting with better communication methods between the agents and the master.

Perhaps something we can all sit down, figure out and implement in version 3.0 or something. That should help with adding new stuff in the future.

[mstarks01]

I'm not sure my past offer to help with this was fully understood. It's either that or Jeremy and I simply disagree on how the project should be handled in this area, which is fine. My proposal was to create a simple written standard, similar to something like Arcsight CEF (but much, much simpler) that defines fields, delimiters, headers, data types, lengths and so on. That would hopefully prevent issues like this because the person implementing event channel would know, for example, that line feeds should not have been used, but perhaps tabs. JSON could go a long way here, but you still have to define field names, etc. and in the case of Windows at least, mappings.

[awiddersheim]

Closing this since #457 was merged which should address the original issue.