Add configurable logging to the console and support stdout sinks

[adfoster-r7]

As an alternative to the rather specific implementation of Add stdout logging functionality to json rpc, this PR adds a configuration option to the console and the rpc service for specifying the required metasploit logsink

Verification

Console verification

Verify that default logging works, and the default Flatfile sink works as expected:

bundle exec ./msfconsole
cat ~/.msf4/logs/framework.log

Ensure that the logger can be configured with arbitrary loggers:

bundle exec ./msfconsole --logger Stdout -x 'exit' -q
bundle exec ./msfconsole --logger StdoutWithoutTimestamps -x 'exit' -q
bundle exec ./msfconsole --logger Stderr -x 'exit' -q
bundle exec ./msfconsole --logger Flatfile -x 'exit' -q
bundle exec ./msfconsole --logger TimestampColorlessFlatfile -x 'exit' -q
bundle exec ./msfconsole --logger LoggerThatDoesntExist -x 'exit' -q

For example, the stdout/stderr loggers should resemble:

$ bundle exec ./msfconsole --logger Stdout -q
[11/25/2020 17:54:12] [e(0)] core: Dependency for windows/encrypted_shell_reverse_tcp is not supported
[11/25/2020 17:54:12] [e(0)] core: Dependency for windows/x64/encrypted_shell_reverse_tcp is not supported
[11/25/2020 17:54:12] [e(0)] core: Dependency for windows/encrypted_reverse_tcp is not supported
[11/25/2020 17:54:12] [e(0)] core: Dependency for windows/x64/encrypted_reverse_tcp is not supported
[11/25/2020 17:54:13] [e(0)] core: Unable to load module /Users/adfoster/Documents/code/metasploit-framework/modules/auxiliary/gather/office365userenum.py - LoadError  Try running file manually to check for errors or dependency issues.
[*] Using configured payload php/meterpreter/reverse_tcp
[*] Starting persistent handler(s)...
msf6 exploit(multi/http/drupal_drupageddon) > 

The Flatfile loggers will go to ~/.msf4/logs/framework.log as normal.

Additionally verifying invalid options work:

$ bundle exec ./msfconsole --logger InvalidLogger -x 'exit' -q
bundler: failed to load command: ./msfconsole (./msfconsole)
Rex::ArgumentError: An invalid argument was specified. Could not find logger InvalidLogger, expected one of StdoutWithoutTimestamps, FlatfileWithoutColors, Stderr, Stdout, Flatfile, Stream
...

JSON RPC support

This functionality is additionally supported by the rpc layer:

MSF_WS_DATA_SERVICE_LOGGER=Stdout bundle exec thin --rackup msf-json-rpc.ru --address localhost --port 8081 --environment production --tag msf-json-rpc start 

Curling with an additional request:

curl --request POST --url http://localhost:8081/api/v1/json-rpc --header 'content-type: application/json' --data '{ "jsonrpc": "2.0", "method": "core.version", "id": 1, "params": [] }'

Verifying the server's output:

$ MSF_WS_DATA_SERVICE_LOGGER=Stdout bundle exec thin --rackup msf-json-rpc.ru --address localhost --port 8081 --environment production --tag msf-json-rpc start
[11/25/2020 17:34:53] [e(0)] core: Dependency for windows/encrypted_shell_reverse_tcp is not supported
[11/25/2020 17:34:53] [e(0)] core: Dependency for windows/x64/encrypted_shell_reverse_tcp is not supported
[11/25/2020 17:34:53] [e(0)] core: Dependency for windows/encrypted_reverse_tcp is not supported
[11/25/2020 17:34:53] [e(0)] core: Dependency for windows/x64/encrypted_reverse_tcp is not supported
[11/25/2020 17:34:54] [e(0)] core: Unable to load module /Users/adfoster/Documents/code/metasploit-framework/modules/auxiliary/gather/office365userenum.py - LoadError  Try running file manually to check for errors or dependency issues.
Thin web server (v1.7.2 codename Bachmanity)
Maximum connections set to 1024
Listening on localhost:8081, CTRL+C to stop
[11/25/2020 17:35:17] [d(0)] core: Already established connection to postgresql, so reusing active connection.
[11/25/2020 17:35:17] [e(0)] core: DB.connect threw an exception - ActiveRecord::AdapterNotSpecified database configuration does not specify adapter
[11/25/2020 17:35:17] [e(0)] core: Failed to connect to the database: database configuration does not specify adapter

[jmartin-tech]

Looks good, some minor thoughts noted.

[jmartin-tech]

I think some in progress PR work snuck into this, can you rebase on master and reduces just to your commits please?

[dwelch-r7]

Release Notes

Added a configuration option to the console and the RPC service for specifying the required metasploit logsink.