VirusTotal integration - Unable to run integration if the file name doesn’t start with a BOM

[CamiRomero]

Wazuh version	Component	Install type	Install method	Platform
4.4.0	Virus Total	Manager	Packages/Sources	CentOS-8
4.4.0	Virus Total	Agent	Packages/Sources	Windows

Description

The virustotal.py script for the integration component is failing when the file name to be processed doesn’t start with a BOM.

Evidences

2022/06/03 14:35:05 wazuh-integratord: ERROR: Unable to run integration for virustotal -> integrations
2022/06/03 14:35:05 wazuh-integratord: ERROR: While running virustotal -> integrations. Output: UnicodeError: UTF-16 stream does not start with BOM
2022/06/03 14:35:05 wazuh-integratord: ERROR: Exit status was: 1

Steps to reproduce

1. Edit the /var/ossec/etc/ossec.conf configuration on manager adding:

<integration>
  <name>virustotal</name>
  <api_key>API_KEY</api_key> <!-- Replace with your VirusTotal API key -->
  <group>syscheck</group>
  <alert_format>json</alert_format>
</integration>

3. Edit the /var/ossec/etc/ossec.conf configuration on agent - Using FIM to monitor a directory:

<directories check_all="yes" realtime="yes">DIRECTORY_TO_MONITOR</directories>

4. Restart the agent and manager in order to apply the changes
5. Create a file with name that does not contain utf-8 characters in the folder under scanning.

Example name: testing.txt

---

Current Result: Unable to run integration for VirusTotal

Expected Result: Run integration for VirusTotal

[CamiRomero]

This fix was applied in branch 8199-virustotal-enconding.

Test on Windows

Reproduce Test Case on master

Package details

Test information

Wazuh version	Wazuh revision	Component	Install type	Install method	Platform
4.3.4	40316	virustotal-scan	Manager	Packages	CentOS-8
4.3.4	40316	virustotal-scan	Agent	Packages	Windows

Reproduce failure in Virus total integration 🟢

1. Edit the /var/ossec/etc/ossec.conf configuration on manager adding:

   <integration>
     <name>virustotal</name>
     <api_key>API_KEY</api_key> <!-- Replace with your VirusTotal API key -->
     <group>syscheck</group>
     <alert_format>json</alert_format>
   </integration>
   

2. Edit the C:\Program Files (x86)\ossec-agent\ossec.conf configuration on agent - Using FIM to monitor a directory:

   <directories check_all="yes" realtime="yes">DIRECTORY_TO_MONITOR</directories>
   

3. Restart the agent and manager in order to apply the changes

4. Create a file with a name that does not contain UTF-8 characters in the folder under scanning.

   Example name: tésting.txt
   

5. We can see the below error in ossec.log:

   2022/06/15 15:20:39 rootcheck: INFO: Ending rootcheck scan.
   2022/06/15 15:21:06 wazuh-integratord: ERROR: Unable to run integration for virustotal -> integrations
   2022/06/15 15:21:06 wazuh-integratord: ERROR: While running virustotal -> integrations. Output: UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe9 in position 569: invalid continuation byte
    
   2022/06/15 15:21:06 wazuh-integratord: ERROR: Exit status was: 1
   
   

Reproduce Test Case on Branch 8199-virustotal-enconding

Package details

Test information

Wazuh version	Wazuh revision	Component	Install type	Install method	Platform
4.4.0	40400	virustotal-scan	Manager	Packages	CentOS-8
4.4.0	40400	virustotal-scan	Agent	Packages	Windows

Test Virus total integration 🟢

1. Edit the /var/ossec/etc/ossec.conf configuration on manager adding:

   <integration>
     <name>virustotal</name>
     <api_key>API_KEY</api_key> <!-- Replace with your VirusTotal API key -->
     <group>syscheck</group>
     <alert_format>json</alert_format>
   </integration>
   

2. Edit the C:\Program Files (x86)\ossec-agent\ossec.conf configuration on agent - Using FIM to monitor a directory:

   <directories check_all="yes" realtime="yes">DIRECTORY_TO_MONITOR</directories>
   

3. Restart the agent and manager in order to apply the changes

4. Create a file with a name that does not contain UTF-8 characters in the folder under scanning.

   Example name: tésting.txt
   

5. We can see the below log in /var/ossec/logs/integrations.log:

   Wed Jun 15 15:50:18 UTC 2022 /tmp/virustotal-1655308218--1043570507.alert df80e208b8628e60266c375f95936c55bf11c5ccc1336e229ef406883cd6eb60   > /dev/null 2>&1
   

6. We can see the below alert in /var/ossec/logs/alerts/alerts.log

   ** Alert 1655308563.1857299: - virustotal,
   2022 Jun 15 15:56:03 (WIN-JLGVA4CR4VI) FE80:0000:0000:0000:B1E6:42AF:E199:1398->virustotal
   Rule: 87104 (level 3) -> 'VirusTotal: Alert - c:\users\vagrant\desktop\testing\tsting.txt - No positives found'
   {"virustotal": {"found": 1, "malicious": 0, "source": {"alert_id": "1655308562.1856042", "file": "c:\\users\\vagrant\\desktop\\testing\\tsting.txt", "md5": "7215ee9c7d9dc229d2921a40e899ec5f", "sha1": "b858cb282617fb0956d960215c8e84d1ccf909c6"}, "sha1": "b858cb282617fb0956d960215c8e84d1ccf909c6", "scan_date": "2022-06-14 12:10:08", "positives": 0, "total": 56, "permalink": "https://www.virustotal.com/gui/file/36a9e7f1c95b82ffb99743e0c5c4ce95d83c9a430aac59f84ef3cbfab6145068/detection/f-36a9e7f1c95b82ffb99743e0c5c4ce95d83c9a430aac59f84ef3cbfab6145068-1655208608"}, "integration": "virustotal"}
   virustotal.found: 1
   virustotal.malicious: 0
   virustotal.source.alert_id: 1655308562.1856042
   virustotal.source.file: c:\users\vagrant\desktop\testing\tsting.txt
   virustotal.source.md5: 7215ee9c7d9dc229d2921a40e899ec5f
   virustotal.source.sha1: b858cb282617fb0956d960215c8e84d1ccf909c6
   virustotal.sha1: b858cb282617fb0956d960215c8e84d1ccf909c6
   virustotal.scan_date: 2022-06-14 12:10:08
   virustotal.positives: 0
   virustotal.total: 56
   virustotal.permalink: https://www.virustotal.com/gui/file/36a9e7f1c95b82ffb99743e0c5c4ce95d83c9a430aac59f84ef3cbfab6145068/detection/f-36a9e7f1c95b82ffb99743e0c5c4ce95d83c9a430aac59f84ef3cbfab6145068-1655208608
   integration: virustotal
   

---

Note: Please note that the alert suppress character with non UTF-8 - Reported in Issue
'VirusTotal: Alert - c:\users\vagrant\desktop\testing\tsting.txt - No positives found'