Logging functionality

[pyneda]

What does it implement/fix? Explain your changes.

1. Logging functionality to allow log to console and file using python's logging module.

• Custom formatter to maintain actual output format which prepends the info, run... prefixes automatically based on the logging level
• Extra logging levels:
  run: Just to maintain current output format
  good: Just to maintain current output format
  vuln: To just print vulnerabilities as requested in issue log to file #146

Note that current prints have not been replaced yet as wanted to have some feedback before doing it and also that either the formatters, chosen prefixes and logging levels can be changed as desired

2. Basic setup.py file

Where has this been tested?

Python Version: 3.7
Operating System: Debian based

Does this close any currently open issues?

Yes, issue: #146 and #160

Does this add any new dependency?

No

Does this add any new command line switch/option?

Yes, new command line options:

• console-log-level: Minimum log level to display in console
• file-log-level: Minimum log level to write to file
• log-file: File to save logs to

Any other comments you would like to make?

Just seen there's already an started implementation for this feature which follows a different approach and although I think not reinventing the wheel and using python's logging system may be better, feel completely free to discard this pr and use the currently started implementation.

Some Questions

• I have documented my code.
• I have tested my build before submitting the pull request.

[s0md3v]

Go on 👌
Remove the setup.py btw

[pyneda]

@s0md3v When you have some free time, could you please review this?

[s0md3v]

Thanks for putting this much of effort into it, I appreciate it.

I ran the following command and saw no changes to the output:
python3 xsstrike.py -u http://*********.***/search.php?q=x --console-log-level DEBUG

Then I ran this command:
python3 xsstrike.py -u http://*********.***/search.php?q=x --file-log-level GOOD --log-file output.txt

The output was all the same and the output.txt was empty.

I haven't checked the code yet and I understand that you may not have enabled this functionality yet.
The implementation is pretty neat tho.

[pyneda]

It should be fixed now, moved xsstrike.py imports as module loggers were initialized before parsing arguments thus only default handlers were used.

Additionally, added logger.debug_json method to allow debug data as json.

[s0md3v]

Still doesn't work man. I tried setting the --console-log-level to GOOD and nothing changed.

[pyneda]

To log to a file you should at least specify --file-log-level option, if you just use --console-log-level the file handler will not be used. Additionally, I suggest trying it with DEBUG or INFO log levels.

[s0md3v]

I don't want to log to the file. I just want to limit the console output to certain type e.g. GOOD. That's what --console-log-level does right?

[pyneda]

Yeah, --console-log-level allows to set the minimum logging level.

Just been trying it with GOOD logging level and it seems to be working good for me, INFO and DEBUG messages such as crawling status or requests details are not displayed and only logs with GOOD level or higher are shown. Could you please give more details on the problem you have so I can verify and fix it?

[s0md3v]

[pyneda]

I will try to look at it, but not sure which can be the problem you face or how to reproduce it. As you can see in pictures below the output I get is correct and moreover emiting log records based on level is done by the python logging module rather than my implementation, anyways, I will try to reproduce it.

VULN:

WARNING:

GOOD:

INFO:

DEBUG:

[s0md3v]

Sorry mate, my bad. It works.
It didn't work for some reason when I downloaded the logger branch via the web interface but it works as intended with git checkout

[s0md3v]

As you are the author of this new logging mechanism, I suggest you write a documentation for the newly added options.
I will add them to the user guide.
And yea, we are ready to merge.

[pyneda]

That's good! Find the requested documentation below, feel free to modify it as you think.

For Developers
log.py
It allows to get a new logger instance by using the setup_logger function. The logger is configured according to the arguments that xsstrike has received on startup and stored in console_log_level, file_log_level and log_file vars and will use a formatter to automatically prepend the required prefix depending on the created log record level. Additionally, the logger instance has RUN, GOOD and VULN custom logging levels and the following available methods:

• no_format: Creates a new log record without automatically adding any prefix. Log level can be specified through the level parameter whose default value is INFO

• debug_json: Converts the received dict to json and appends it to the end of the new log record which is always created with DEBUG level.

• red_line: Creates a red line which can be used as separator. Amount of - characters and log level can be specified through amount and level paramets.

Usage
Logging

Option: --console-log-level | Default: INFO

It is possible to choose a minimum logging level to display xsstrike logs in the console:
python xsstrike.py -u "http://example.com/search.php?q=query" --console-log-level WARNING

Option: --file-log-level | Default: None

If specified, xsstrike will also write all logs with equal logging level or higher to a file:
python xsstrike.py -u "http://example.com/search.php?q=query" --console-log-level DEBUG

Option: --log-file | Default: xsstrike.log

Name of the file where logs will be stored. Note that if --file-log-levelis not specified, this option will not have any effect.
python xsstrike.py -u "http://example.com/search.php?q=query" --file-log-level INFO --log-file output.log

[sniff122]

i am not seeing the new commands added, when i try to run with the commands, i get

[s0md3v]

re-clone it

[sniff122]

Also tried