[bug] Cannot process directory of evtx files with space in directory path

[joswr1ght]

Describe the bug
Running Hayabusa using a directory of EVTX files -d where the directory path includes a space causes Hayabusa to return an error response.

Step to Reproduce
Steps to reproduce the behavior:

PS C:\tools\hayabusa> .\hayabusa.exe csv-timeline -d 'C:\Users\Sec504\Desktop\T1078-Valid accounts\' -o evtxtomitre.csv

╔╗ ╔╦═══╦╗  ╔╦═══╦══╗╔╗ ╔╦═══╦═══╗
║║ ║║╔═╗║╚╗╔╝║╔═╗║╔╗║║║ ║║╔═╗║╔═╗║
║╚═╝║║ ║╠╗╚╝╔╣║ ║║╚╝╚╣║ ║║╚══╣║ ║║
║╔═╗║╚═╝║╚╗╔╝║╚═╝║╔═╗║║ ║╠══╗║╚═╝║
║║ ║║╔═╗║ ║║ ║╔═╗║╚═╝║╚═╝║╚═╝║╔═╗║
╚╝ ╚╩╝ ╚╝ ╚╝ ╚╝ ╚╩═══╩═══╩═══╩╝ ╚╝
   by Yamato Security

Start time: 2023/08/22 16:43

[ERROR] No .evtx files were found.

Elapsed time: 00:00:00.003
Errors were generated. Please check ./logs/errorlog-20230822_164335.log for details.


PS C:\tools\hayabusa> get-content ./logs/errorlog-20230822_164335.log
user input: C:\tools\hayabusa\hayabusa.exe csv-timeline -d C:\Users\Sec504\Desktop\T1078-Valid accounts" -o evtxtomitre.csv
[ERROR] The filename, directory name, or volume label syntax is incorrect. (os error 123)
PS C:\tools\hayabusa> Get-ChildItem 'C:\Users\Sec504\Desktop\T1078-Valid accounts\'


    Directory: C:\Users\Sec504\Desktop\T1078-Valid accounts


Mode                 LastWriteTime         Length Name
----                 -------------         ------ ----
-a----         5/17/2023   5:36 PM          69632 ID1149-RDP success logins to multiple hosts.evtx
-a----         5/17/2023   5:36 PM          69632 ID33205-SQL Server failed login with disabled SA account.evtx
-a----         5/17/2023   5:36 PM          69632 ID4625-failed login with denied access due to account
                                                  restriction.evtx

In this example, if I rename the Desktop folder from T1078-Valid accounts to validaccounts or similar, Hayabusa recognizes the directory of files and processes the EVTX files as expected.

Expected behavior

When a directory path includes spaces and is presented as a quoted argument, Hayabusa should interpret the directory path as a literal path.

Environment (please complete the following information):

• OS: Microsoft Windows [Version 10.0.19044.1682]
• hayabusa version 2.6.0

[fukusuket]

Thank you for reporting the issue :)
When I ran it with cmd on Windows 11, the result was as follows. (single quote fails, double quote succeeds)

C:\tmp\hayabusa-2.7.0-win-64-bit>hayabusa-2.7.0-win-x64.exe csv-timeline -d 'C:\test space\hayabusa-sample-evtx'
error: unexpected argument 'space\hayabusa-sample-evtx'' found

Usage: hayabusa-2.7.0-win-x64.exe csv-timeline [OPTIONS]

C:\tmp\hayabusa-2.7.0-win-64-bit>hayabusa-2.7.0-win-x64.exe csv-timeline -d "C:\test space\hayabusa-sample-evtx"

╔╗ ╔╦═══╦╗  ╔╦═══╦══╗╔╗ ╔╦═══╦═══╗
║║ ║║╔═╗║╚╗╔╝║╔═╗║╔╗║║║ ║║╔═╗║╔═╗║
║╚═╝║║ ║╠╗╚╝╔╣║ ║║╚╝╚╣║ ║║╚══╣║ ║║
║╔═╗║╚═╝║╚╗╔╝║╚═╝║╔═╗║║ ║╠══╗║╚═╝║
║║ ║║╔═╗║ ║║ ║╔═╗║╚═╝║╚═╝║╚═╝║╔═╗║
╚╝ ╚╩╝ ╚╝ ╚╝ ╚╝ ╚╩═══╩═══╩═══╩╝ ╚╝
   by Yamato Security

Start time: 2023/08/24 20:56

Total event log files: 585
Total file size: 137.2 MB

Loading detections rules. Please wait.

Windows(cmd/Powershell) seems to interpret single quotes as simple values... :(
(not as a pair of quotes)

• https://powerbi-cli.github.io/content/howto-quotes.html?tabs=bash
• https://stackoverflow.com/questions/51080215/differences-between-single-and-double-quotes-in-cmd/51090074

[fukusuket]

Sorry, I misunderstood behavior... The above #1166 (comment) is true for cmd but doesn't seem to be true for PowerShell.
The reproduction conditions with PowerShell seem to be the following three points.

• Directory path contains space
• The last character of the value passed to the -d option is a backslash
• Other options are passed after -d option(like -o option)

I have confirmed the following on Windows 10.

PS C:\tmp\hayabusa-2.6.0-win-64-bit> Get-Host


Name             : ConsoleHost
Version          : 5.1.19041.3031
InstanceId       : a225adad-2d8a-4371-a067-c14dc7a1c62b
UI               : System.Management.Automation.Internal.Host.InternalHostUserInterface
CurrentCulture   : ja-JP
CurrentUICulture : ja-JP
PrivateData      : Microsoft.PowerShell.ConsoleHost+ConsoleColorProxy
DebuggerEnabled  : True
IsRunspacePushed : False
Runspace         : System.Management.Automation.Runspaces.LocalRunspace

PS C:\tmp\hayabusa-2.6.0-win-64-bit> .\hayabusa-2.6.0-win-x64.exe csv-timeline -d "C:\tmp\data\EVTX-to-MITRE-Attack\TA0001-Initial access\T1078-Valid accounts" -o 1.csv

...
   by Yamato Security

Start time: 2023/08/24 23:01

Total event log files: 3
Total file size: 208.9 KB

Loading detections rules. Please wait.

PS C:\tmp\hayabusa-2.6.0-win-64-bit> .\hayabusa-2.6.0-win-x64.exe csv-timeline -d 'C:\tmp\data\EVTX-to-MITRE-Attack\TA0001-Initial access\T1078-Valid accounts' -o 2.csv

...
   by Yamato Security

Start time: 2023/08/24 23:01

Total event log files: 3
Total file size: 208.9 KB

Loading detections rules. Please wait.

PS C:\tmp\hayabusa-2.6.0-win-64-bit> .\hayabusa-2.6.0-win-x64.exe csv-timeline -d "C:\tmp\data\EVTX-to-MITRE-Attack\TA0001-Initial access\T1078-Valid accounts\" -o 3.csv

...
   by Yamato Security

Start time: 2023/08/24 23:02

[ERROR] No .evtx files were found.

Elapsed time: 00:00:00.011
Errors were generated. Please check ./logs/errorlog-20230824_230206.log for details.

PS C:\tmp\hayabusa-2.6.0-win-64-bit> .\hayabusa-2.6.0-win-x64.exe csv-timeline -d 'C:\tmp\data\EVTX-to-MITRE-Attack\TA0001-Initial access\T1078-Valid accounts\' -o 4.csv

...
   by Yamato Security

Start time: 2023/08/24 23:02

[ERROR] No .evtx files were found.

Elapsed time: 00:00:00.014
Errors were generated. Please check ./logs/errorlog-20230824_230222.log for details.

@joswr1ght
Are the above reproduction conditions the same in your environment?

[YamatoSecurity]

@fukusuket Thanks for finding out the cause of this!
To sum it up, when using Hayabusa in the Command Prompt or PowerShell prompt with directories with spaces in them, you need to 1) Enclose the path with double quotes and 2) Make sure the path does not end with a backslash.
(To be most specific, PowerShell prompt allows single quotes but not if it ends in a backslash)
Is that right?

By the way, do you know why it fails if the path ends in a backslash and is this something we can handle in Hayabusa?
It seems like a problem with Command/PowerShell prompt as I can end the path with a backslash on my Mac terminal and it works. I also think Windows Terminal lets you end the path with a backslash, just not the old terminals in Windows.

@joswr1ght Could you let us know if this works for you?
I think this is a common hurdle users come across so I will add something to the readme to explain about this.

[fukusuket]

@YamatoSecurity

Is that right?

Yes, You are right!

By the way, do you know why it fails if the path ends in a backslash and is this something we can handle in Hayabusa?

The version is a little old, but the cause similar to the following may be related.

• Strings with a backslash before the ending quote results in a quote being included in the final arg value clap-rs/clap#2791

I'll check if Hayabusa can handle it :)

[fukusuket]

@YamatoSecurity
It seems that this behavior is based on the following specifications.
https://learn.microsoft.com/en-us/cpp/cpp/main-function-command-line-args?view=msvc-160#parsing-c-command-line-arguments

A double quote mark preceded by a backslash (") is interpreted as a literal double quote mark (").

Due to the above, it seems difficult to deal with on the Hayabusa side :(

Therefore, In the README,
it would be better to add a note not to use a backslash as the last character when the -d option is enclosed in quotation marks.

The only exception is PowerShell 7.3 and newer, which worked even with a backslash as the last character.
https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.core/about/about_parsing?view=powershell-7.3#passing-arguments-that-contain-quote-characters

PowerShell 7.3 changed the way the command line is parsed for native commands.

[joswr1ght]

For a better user experience, I suggest amending the error in the output log file to indicate that trailing backslashes should not be used in a directory path specification.

Imagine a user is running Hayabusa and they use PowerShell tab completion to specify a directory path. PowerShell will autocomplete the directory name that will lead to Hayabusa's inability to process the directory path. While mentioning this behavior in the README is nice, it's better to have the directions in the place where the user will be able to immediately correct the problem (and not just blame Hayabusa unfairly for this problem).

Here is the original error log file output:

user input: C:\tools\hayabusa\hayabusa.exe csv-timeline -d C:\Users\Sec504\Desktop\T1078-Valid accounts" -o evtxtomitre.csv
[ERROR] The filename, directory name, or volume label syntax is incorrect. (os error 123)

Suggest something like this:

user input: C:\tools\hayabusa\hayabusa.exe csv-timeline -d C:\Users\Sec504\Desktop\T1078-Valid accounts" -o evtxtomitre.csv
[ERROR] The filename, directory name, or volume label syntax is incorrect. (os error 123). When specifying a directory name, do not include a trailing slash at the end of the path.

[YamatoSecurity]

@joswr1ght That is a good suggestion. However, the message The filename, directory name, or volume label syntax is incorrect. might not be something we can easily customize.

@fukusuket @hitenkoku Do one of you know how we could add the message When specifying a directory path in Windows, do not include a trailing slash at the end of the path. to the log file when we get an error 123?

[hitenkoku]

@YamatoSecurity I will check it.

Do one of you know how we could add the message When specifying a directory path in Windows, do not include a trailing slash at the end of the path. to the log file when we get an error 123?

[YamatoSecurity]

@fukusuket @hitenkoku I noticed that this error of not being able to load evtx file only happens if the user specifies -o output.csv. If -o is not used and results are output to screen, then it will load the evtx files correctly.
Do you know why it would fail when -o is used but succeeds when -o is not used. (Does hayabusa load evtx files differently when -o is used?)
It seems like we could load the evtx files even in Windows and with spaces.

[fukusuket]

@YamatoSecurity
I tried it with the following simple Rust program (without clap)!

use std::env;

fn main() {
    let args: Vec<String> = env::args().collect();
    println!("{:?}", args);
}

The results are as follows. It seems that the cause is that all arguments after the backslash are treated as one argument.
I thinks this is the same behavior as described in Microsoft's document.
(The number of commas is the same as the number of arguments Rust recognizes.)

PS C:\tmp> .\sample1.exe test -d "C:\tmp\data space\"
["C:\\tmp\\sample1.exe", "test", "-d", "C:\\tmp\\data space\""]

PS C:\tmp> .\sample1.exe test -d "C:\tmp\data space\" -o out.csv
["C:\\tmp\\sample1.exe", "test", "-d", "C:\\tmp\\data space\" -o out.csv"]

PS C:\tmp> .\sample1.exe test -d "C:\tmp\data space" -o out.csv
["C:\\tmp\\sample1.exe", "test", "-d", "C:\\tmp\\data space", "-o", "out.csv"]

I confirmed the above behavior with the following PowerShell version.

PS C:\tmp> $PSVersionTable

Name                           Value
----                           -----
PSVersion                      5.1.20348.1850
...

Rust received the wrong arguments from PowerShell, so I think it's hard to handle wrong arguments on the Rust side :(
(Strictly speaking, I think it can be resolved by further parsing the invalid argument above, but this correction method seems complicated ... 😭)

[YamatoSecurity]

@fukusuket Ah, I see now. Thanks for testing it out! In that case, there is not much we can do on the Hayabusa side. We will deal with it just with explanation and error message.