Output of script not being saved to file or piped

[X-l-l-l]

Hi, I am trying to save the output of the script, for example like this:
python3 LFImap/lfimap.py -U "http://localhost/vulnerabilities/fi/?page=include.php" -C "..." -a > fi.txt
Or by using subprocess.Popen and then using iter to get the output, but id does not print anything.
Also tryed with subprocess.run and capture_output, and printing it at the end... Still nothing.

Am I doing something wrong?

[hansmach1ne]

Hello, thank You for taking interest in LFImap.
I have reproduced this issue and will provide the fix ASAP, perhaps with the new switch to allow users to output to a specified file.

[nrathaus]

@hansmach1ne

I think the best option is to replace print(...) which is used to print things out with a logging module, this can allow you to switch between printing to console and "print" to a file (or both) without much more than a configuration setting during runtime

I can work on it if you give me the green light

BTW: I strongly suggest to consider linting, many lines of work are very long (way over 100 characters), some are written in if .. something ... else without newlines, making streamline reading difficult

[nrathaus]

You can see a basic print( to logging.info replacement here:
#55

And the outcome of it that creates example.log:

INFO:root:
�[93m[i]�[0m Testing GET 'page' parameter...
INFO:root:
�[93m[i]�[0m Testing GET 'page' parameter...
INFO:root:
�[93m[i]�[0m Testing GET 'page' parameter...
INFO:root:
�[93m[i]�[0m Testing GET 'page' parameter...
INFO:root:
�[93m[i]�[0m Testing GET 'page' parameter...
INFO:root:
----------------------------------------
LFImap finished with execution.
INFO:root:Parameters tested: 1
INFO:root:Requests sent: 14
INFO:root:Vulnerabilities found: 4
INFO:root:
�[93m[i]�[0m Testing GET 'page' parameter...
INFO:root:
----------------------------------------
LFImap finished with execution.
INFO:root:Parameters tested: 1
INFO:root:Requests sent: 14
INFO:root:Vulnerabilities found: 4
INFO:root:
�[93m[i]�[0m Testing GET 'page' parameter...
INFO:root:
----------------------------------------
LFImap finished with execution.
INFO:root:Parameters tested: 1
INFO:root:Requests sent: 14
INFO:root:Vulnerabilities found: 4
INFO:root:
�[93m[i]�[0m Testing GET 'page' parameter...
INFO:root:
----------------------------------------
LFImap finished with execution.
INFO:root:Parameters tested: 1
INFO:root:Requests sent: 14
INFO:root:Vulnerabilities found: 4
INFO:root:
�[93m[i]�[0m Testing GET 'page' parameter...
INFO:root:
�[93m[i]�[0m Testing GET 'page' parameter...
INFO:root:�[92m[+]�[0m LFI -> 'http://localhost:4280/vulnerabilities/fi/?page=php%3A%2F%2Ffilter%2Fresource%3D%2Fetc%2Fpasswd'
INFO:root:�[92m[+]�[0m RCE -> 'http://localhost:4280/vulnerabilities/fi/?page=php%3a%2f%2finput&cmd=cat%20%2Fetc%2Fpasswd' -> HTTP POST -> '<?php echo(shell_exec($_GET['cmd']));?>'
INFO:root:�[92m[+]�[0m LFI -> 'http://localhost:4280/vulnerabilities/fi/?page=file%3A%2F%2F%2Fetc%2Fpasswd'
INFO:root:�[92m[+]�[0m LFI -> 'http://localhost:4280/vulnerabilities/fi/?page=/etc/passwd'
INFO:root:
----------------------------------------
LFImap finished with execution.
INFO:root:Parameters tested: 1
INFO:root:Requests sent: 14
INFO:root:Vulnerabilities found: 4

There is more work to do though, like color removal, understand why it prints the outcome a few times, etc

[hansmach1ne]

@nrathaus

Hey, first of, thanks for the suggestion.

This would be great, however we would need to account the ANSI escape sequences, which color the certain output in a colored way. Logging library will print these ANSI strings (unformatted) directly to the file.

With that said, planning to continue to have the colored output support, because when the user tests large amount of URLs, the output is too clustered. The colors solve this problem. Additionally, in the future there will be command-line switches so that user can output to the XML, HTML, and TXT files sort of like a output 'report'. The TXT switch would output the LFImap output as is to the specified file location.

[X-l-l-l]

For me at least, the ANSI part wouldn't be a problem, as I have already written a small function that deletes any ANSI sequences in a string, I could put it here if it's any help. What I really needed was a way to capture the exact output of the script and process it in some way.

[hansmach1ne]

@X-l-l-l Of course, if you have it already written and ready, mind pasting it here? Thanks. :)

[X-l-l-l]

Sure thing, here it is:

def rm_ansi(line):
    ansi_escape = re.compile(r'\x1b\[([0-9]{1,2}(;[0-9]{1,2})?)?[m|K]')
    plain_text = ansi_escape.sub('', line)
    return plain_text  

Haven't tested it in every situation, but worked well in the cases i needed it.

[hansmach1ne]

The problem is identified as incorrect buffering done by the python for some unknown reasons (Guessing because we use ANSI escape sequences, buffers are not flushed correctly -> not a hundred percent on that).

However, executing export PYTHONUNBUFFERED=1 before running the script, outputs STDOUT correctly and confirms the issue is related to buffering.

[hansmach1ne]

@X-l-l-l Could you git pull the latest update and test if it now works for you?
Added flushing after printing to STDOUT - should fix the problem.

Default colored:

No colors:

[X-l-l-l]

Tested it now. That seems like it fixed it. Tried it as you did, with outputting to a file and also tried it with subprocess.run and Popen and piping the stdout to other scripts or areas. Thank you very much! Amazing work!

[hansmach1ne]

👍 Thanks for raising this