Integrate winevt-kb extract and export functionality into plaso tool

[joachimmetz]

Per libyal/winevt-kb#10 request to keep winevt-kb tooling and plaso closely synced. Best approach is to have the functionality to extract Windows EventLog resources embedded in Plaso.

• add pyexe and pywrc dependencies to gift, l2tdevtools and l2tbinaries
  • GIFT
  • l2tbinaries win32
  • l2tbinaries win64
  • l2tbinaries macosx
• add WindowsEventLogProviders artifact definition - Added WindowsEventLogProviders definition ForensicArtifacts/artifacts#422
• add PreprocessingWarning
• release new version of ForensicArtifacts - https://github.com/ForensicArtifacts/artifacts/releases/tag/20210620
• extract and store Windows EventLog providers in knowledge base (WindowsEventLogProviderArtifact, WindowsEventLogProvidersPlugin) - Changed preprocessing to extract Windows EventLog providers #636 #3755
• Move event log providers out of system configuration - Moved EventLog providers out of system configuration #636 #3819
• extract EventLog message strings - Changed pe parser to extract message strings #636 #3853
  • looks like pefile has no native support to read the message strings https://github.com/erocarrera/pefile/blob/wiki/ReadingResourceStrings.md
  • winevtrc model:
    • message files table (columns: file_id, path), message string table [name: file_id+lcid] (columns: message id, string)
  • only extract message strings from files defined by event providers
  • only extract message strings corresponding to the preferred language
• move winevtrc-db and "native" winevtrc into output helper class - Added Windows EventLog resources output helper #636 #3856
• output extracted message strings - Changes Windows EventLog resources output helper #636 #3860
• add log2timeline option to set preferred language - Added preferred language extraction option #636 #3869
• add log2timeline option to disable message string extraction - Added extract Windows EventLog resources option #636 #3871
• extract message strings when the winevt or winevtx parser is enabled - Changes for Windows EventLog resources support #636 #3882
  • for now issue a warning if pe parser is disabled

Next steps and improvements captured in #163

[rodgermoore]

Hi @joachimmetz . Is this going to be implemented in the 1.6.0 release? Would be nice 🥇

[joachimmetz]

@rodgermoore this is not likely to get done before the next mid year release (1.6.0)

• A couple of things we've dropped the x.y.z version schema and are working on a more frequent release schedule.
• We have to overhaul the storage, which should allow us to directly store this and similar information into the plaso storage file

[joachimmetz]

Storage changes have a higher priority at the moment, but should make it easier to implement this in plaso during extraction.

[joachimmetz]

Unlikely to be implemented before Jan 25, bumping release

[Onager]

Not going to make release, removing milestone.

Please do not re-assign to a milestone until this issue has been cleaned up.

[joachimmetz]

how to handle an event provider defined as event source:

Microsoft-Windows-Resource-Exhaustion-Detector  System          %SystemRoot%\system32\radardt.dll

and winevt publisher

Microsoft-Windows-Resource-Exhaustion-Detector                  %SystemRoot%\system32\radardt.dll

Looks like the 0xb0000000 mask is also used

10073|124|1033|2952791017|The Windows Resource Exhaustion Detector started.|0xb00003e9
10074|124|1033|2952791018|The Windows Resource Exhaustion Detector stopped.|0xb00003ea
10075|124|1033|2952791019|The Windows Resource Exhaustion Detector received a notification that the computer is low on virtual memory.|0xb00003eb
10076|124|1033|2952791021|The Windows Resource Exhaustion Detector failed to start due to an error.|0xb00003ed
10077|124|1033|2952791022|The Windows Resource Exhaustion Detector failed to stop due to an error.|0xb00003ee
10078|124|1033|2952791023|The Windows Resource Exhaustion Detector experienced a memory allocation failure.|0xb00003ef
10079|124|1033|2952791024|Windows failed to diagnose a low virtual memory condition.|0xb00003f0

[joachimmetz]

Regarding the use of 0xb0000000 it seems to be mixed e.g.

Log source              : Microsoft-Windows-Application-Experience
Identifier              : {eef54e71-0661-422d-9a98-82fd4940b820}
Log type                : Application
Event message files     : %SystemRoot%\system32\aeevts.dll

2012-05-16T11:04:41.000000+00:00,Content Modification Time,EVT,WinEVTX,[201 / 0x00c9] Source Name: Microsoft-Windows-Application-Experience [Provider identifier: {eef54e71-0661-422d-9a98-82fd4940b820}] [Message identifier: 0x000000c9] Strings: [] Computer Name: test-PC Record Number: 1486 Event Level: 4,winevtx,NTFS:\Windows\System32\winevt\Logs\System.evtx,-

<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
  <System>
    <Provider Name="Microsoft-Windows-Application-Experience" Guid="{EEF54E71-0661-422D-9A98-82FD4940B820}"/>
    <EventID>201</EventID>
    <Version>0</Version>
    <Level>4</Level>
    <Task>0</Task>
    <Opcode>0</Opcode>
    <Keywords>0x8000000000000000</Keywords>
    <TimeCreated SystemTime="2012-05-16T11:04:41.440000000Z"/>
    <EventRecordID>1486</EventRecordID>
    <Correlation/>
    <Execution ProcessID="768" ThreadID="2784"/>
    <Channel>System</Channel>
    <Computer>test-PC</Computer>
    <Security UserID="S-1-5-18"/>
  </System>
  <EventData/>
</Event>

138669|103|1033|201|The Program Compatibility Assistant service started successfully.|0x000000c9
138691|103|1033|2952790223|The Program Compatibility Assistant was requested to monitor {0:s}, but ignored the request because the application is excluded in the registry.|0xb00000cf

[joachimmetz]

Also seen 0xb1000000:

140344|128|1033|2969570237|A summary of the Client Side Caching counters has been generated. The counter list can be found in the event details.|0xb1000bbd

  <System>
    <Provider Name="Microsoft-Windows-BranchCacheSMB" Guid="{4A933674-FB3D-4E8D-B01D-17EE14E91A3E}"/>
    <EventID>3005</EventID>
    <Version>0</Version>
    <Level>4</Level>
    <Task>0</Task>
    <Opcode>0</Opcode>
    <Keywords>0x8000000000000800</Keywords>
    <TimeCreated SystemTime="2012-05-16T09:59:40.199600000Z"/>
    <EventRecordID>1</EventRecordID>
    <Correlation/>
    <Execution ProcessID="880" ThreadID="1076"/>
    <Channel>Microsoft-Windows-BranchCacheSMB/Operational</Channel>
    <Computer>37L4247D28-05</Computer>
    <Security UserID="S-1-5-18"/>
  </System>

Is this related to channels?

Key path: CMI-CreateHive{3D971F19-49AB-4000-8D39-A6D9C673D809}\Microsoft\Windows\CurrentVersion\WINEVT\Publishers\{4a933674-fb3d-4e8d-b01d-17ee14e91a3e}\ChannelReferences\0
Name: 0
Last written time: Jul 14, 2009 07:51:33.220737800 UTC

Value: 0 (default)
Type: string (REG_SZ)
Data size: 90
Data: Microsoft-Windows-BranchCacheSMB/Operational

Value: 1 Id
Type: 32-bit integer little-endian (REG_DWORD_LITTLE_ENDIAN)
Data size: 4
Data: 16

Value: 2 Flags
Type: 32-bit integer little-endian (REG_DWORD_LITTLE_ENDIAN)
Data size: 4
Data: 0

Key path: CMI-CreateHive{3D971F19-49AB-4000-8D39-A6D9C673D809}\Microsoft\Windows\CurrentVersion\WINEVT\Publishers\{4a933674-fb3d-4e8d-b01d-17ee14e91a3e}\ChannelReferences\1
Name: 1
Last written time: Jul 14, 2009 07:51:33.220737800 UTC

Value: 0 (default)
Type: string (REG_SZ)
Data size: 84
Data: Microsoft-Windows-BranchCacheSMB/Analytic

Value: 1 Id
Type: 32-bit integer little-endian (REG_DWORD_LITTLE_ENDIAN)
Data size: 4
Data: 17

Value: 2 Flags
Type: 32-bit integer little-endian (REG_DWORD_LITTLE_ENDIAN)
Data size: 4
Data: 0

[joachimmetz]

Maybe related to 2 event providers using the same message resource file?

Log source              : Microsoft-Windows-OfflineFiles
Identifier              : {95353826-4fbe-41d4-9c42-f521c6e86360}
Log type                : System
Event message files     : %systemroot%\system32\cscsvc.dll

Log source              : Microsoft-Windows-BranchCacheSMB
Identifier              : {4a933674-fb3d-4e8d-b01d-17ee14e91a3e}
Event message files     : %systemroot%\system32\cscsvc.dll

Link to WEVT_TEMPLATE PE/COFF resource? Is possible

libfwevt_event_read: identifier                                         : 0x0001
libfwevt_event_read: version                                            : 0
libfwevt_event_read: channel                                            : 16
libfwevt_event_read: level                                              : 4
libfwevt_event_read: opcode                                             : 0
libfwevt_event_read: task                                               : 0
libfwevt_event_read: keywords                                           : 0x4000000000000010
libfwevt_event_read: message identifier                                 : 0xb0000001
libfwevt_event_read: template offset                                    : 0x00000000
libfwevt_event_read: opcode offset                                      : 0x000033b4
libfwevt_event_read: level offset                                       : 0x000033fc
libfwevt_event_read: task offset                                        : 0x00000000
libfwevt_event_read: unknown3                                           : 0x00000001
libfwevt_event_read: unknown4                                           : 0x00003c28
libfwevt_event_read: flags                                              : 0x000000bc

More research is needed here added to #163

[joachimmetz]

EventLog provides with multiple names:

Log source              : Microsoft-Windows-WMI
Identifier              : {1edeee53-0afe-4609-b846-d8c0b2075b1f}
Event message files     : %SystemRoot%\system32\wbem\WinMgmtR.dll

Log source              : WinMgmt
Identifier              : {1edeee53-0afe-4609-b846-d8c0b2075b1f}
Log type                : Application

[joachimmetz]

Note to self to check for unsupported EventLog providers

grep '\[Message identifier: ' output.log | grep -v ' Message string: ' | sed 's/^.* Source Name: //;s/ \[Message identifier: .*$//' | sort | uniq