Crash when finding a broken link

[j75]

I am on Linux (Ubuntu); when running kit_hunter_2.py if it finds a broken symbolic link somewhere in the indicated path, then the application stops:

Traceback (most recent call last):
  File "kit_hunter_2.py", line 560, in <module>
    folder_files = get_contents_of_folder_files(directory_path, supported_file_formats)
  File "kit_hunter_2.py", line 252, in get_contents_of_folder_files
    file_contents = open(file_path, "rb").read().splitlines()
FileNotFoundError: [Errno 2] No such file or directory: ...

[SteveD3]

Can you tell me what the directory is you're scanning, and what the symbolic link is pointed at? I've never tested it with symbolic links, just the directory where the files are or just outside of public_html (or /var/www/). Will make a note of this and check it out.

Edit to add: What happens when you run ./kit_hunter2.py or python 3 kit_hunter.py ?

[SteveD3]

So I'm trying to recreate this. I've created a test scan where I've included six different broken symbolic links.

---

steved3@steved3-lab:/media/steved3/PHISHING/Kit-Hunter-2/Scanner/scan$ tree
.
├── kit_hunter_2.py
└── scan2
├── myfile -> source_file
├── myfile2 -> source_file
└── scan3
├── myfile -> source_file
├── myfile2 -> source_file
└── scan4
├── censor.be_CA 3.0.zip
├── myfile -> source_file
└── myfile2 -> source_file

3 directories, 8 files

When I scan, the output and function works as expected.

---

steved3@steved3-lab:/media/steved3/PHISHING/Kit-Hunter-2/Scanner/scan$ ./kit_hunter_2.py -qlm

Kit Hunter Starting...

Examining Folder: /media/steved3/PHISHING/Kit-Hunter-2/Scanner/scan

Examining Folder: /media/steved3/PHISHING/Kit-Hunter-2/Scanner/scan/scan2

Examining Folder: /media/steved3/PHISHING/Kit-Hunter-2/Scanner/scan/scan2/scan3

Examining Folder: /media/steved3/PHISHING/Kit-Hunter-2/Scanner/scan/scan2/scan3/scan4

Examining Archive: /media/steved3/PHISHING/Kit-Hunter-2/Scanner/scan/scan2/scan3/scan4/censor.be_CA 3.0.zip

=========================

Done! All file processing is complete.

=========================

Kit Hunter processed all files in 00h : 00m : 00.12s

=========================

The finished report is located at:

/media/steved3/PHISHING/Kit-Hunter-2/Scanner/scan/Kit_Hunter_Report_2021-Sep-18-0243.log

=========================

[SteveD3]

I tried it with a valid symbolic link, and got the same results. It skipped the broken symbolic links and the valid one.

[SteveD3]

Tried this using ./kit_hunter.py as well. No flags, full scan. Same results. Scan worked as expected.

Debian 10

steved3@steved3-lab:~$ which python
/usr/bin/python

steved3@steved3-lab:~$ python --version
Python 3.7.3

[j75]

$ tree /tmp/test_kit_hunter/
/tmp/test_kit_hunter/
├── date1
└── dir1
    ├── date2
    └── dir2
        ├── date1 -> ../../date1
        └── dir3
            └── date.txt

3 directories, 4 files

$ python3 kit_hunter_2.py -d
What directory will you scan? Remember: /you/must/use/a/full/path/ :  /tmp/test_kit_hunter/

Kit Hunter Starting...

Traceback (most recent call last):
  File "kit_hunter_2.py", line 561, in <module>
    process_files(directory_path, compressed_files, folder_files)
  File "kit_hunter_2.py", line 514, in process_files
    extra_tag_file_contents = get_contents_of_tag_files(kh_tag_path)
  File "kit_hunter_2.py", line 218, in get_contents_of_tag_files
    for file in os.listdir(directory_path):
FileNotFoundError: [Errno 2] No such file or directory: '/path/to/the/primary/tag/folder/'

The date files were simply created with date > file.txt.
I forget to specify, I am on Ubuntu 20.04.03 and pyhon3 in 3.8.10 version.

Running the script from the /tmp/test_kit_hunter and with the -qlm option produces the same error.

IMHO it would be nice:

1. to display the real path of the folder that triggered the error
2. to be able to pass directly the path as a parameter i.e. `python3 kit_hunter_2.py -d /tmp/test_kit_hunter/'

[SteveD3]

The error is showing me that you don't have your tag paths set.

FileNotFoundError: [Errno 2] No such file or directory: '/path/to/the/primary/tag/folder/'

At the top of the kit_hunter_2.py (lines 27 and 28), there is a config option, where you set the paths of your tag files. Mine looks like this:

kh_quick_scan = '/media/steved3/PHISHING/Kit-Hunter-2/tag_files/quick_scan/'

and

kh_full_scan = '/media/steved3/PHISHING/Kit-Hunter-2/tag_files/'

Set those options and let me know if you get the same error.

I'll place the direct path parameter on the development list for sure. Will be something good for me to learn. Thanks for taking the time to report this error though, and offer feedback. I really appreciate it.

[SteveD3]

2. to be able to pass directly the path as a parameter i.e. `python3 kit_hunter_2.py -d /tmp/test_kit_hunter/'

Quick update, got this aspect working for you tonight, will test it for a bit and then release it. So far it is working as expected, but I'm still seeing what I can throw at it.

[j75]

So, I modified the 2 values (kh_quick_scan and kh_full_scan) it still stops when a broken link is found !

% tree /tmp/test_kit_hunter
/tmp/test_kit_hunter
└── dir1
    ├── date.txt -> ../date.txt
    └── dir2

% python3 kit_hunter_2.py -d
What directory will you scan? Remember: /you/must/use/a/full/path/ :  /tmp/test_kit_hunter/
Traceback (most recent call last):
  File "kit_hunter_2.py", line 560, in <module>
    folder_files = get_contents_of_folder_files(directory_path, supported_file_formats)
  File "kit_hunter_2.py", line 252, in get_contents_of_folder_files
    file_contents = open(file_path, "rb").read().splitlines()
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/test_kit_hunter/dir1/date.txt'

As the folders tag_files/ and tag_files/quick_scan/ are subfolders of the folder where kit_hunter_2.py is located, couldn't it use its own path to set them (at least by default)?

[SteveD3]

It's taken some time, but I've almost got this fixed. Will be releasing a new version soon with error checking, which will log errors, but allow the scan to continue.

Once I am done with testing, I will release it. Also, your request for switching has been added as well. So -d /path/to/folder/ will work, that change is live now in the repository. As for the tag paths, those are directed like that so that the tags can be stored anywhere and called, but I will consider making that change in a future release.

-Steve

Example:

| ===============================================================================================
| Scan Error Report:
| ===============================================================================================
| The following errors occurred during processing:
| ===============================================================================================
| Error Location:
|      /steved3/Kit-Hunter-2/Scanner/Lab/scan/scan2/j75_invalid.txt
|
| Error Type:
|       [Errno 2] No such file or directory: '/steved3/Kit-Hunter-2/Scanner/Lab/scan/scan2/j75_invalid.txt'
| ===============================================================================================

[j75]

OK, I'll be glad to test the new version ASAP! :-)

About the symlinks, I am wondering if it wouldn't be more appropriate to skip (and log) those that point outside the checked directory...

[SteveD3]

It will log errors. So if the Symlinks are broken, and pointed outside, or internal, it should log them all.
Version 2.6.0 should fix the problem you've reported. Please download that, and try it out. I'll leave this ticket open for a few days so you can test.

[j75]

Yes, now it works as expected - thanks!