Prefer Rex::Text.dehex over Rex::Text.hex_to_raw in msfvenom

[wvu]

Rex::Text.dehex allows character literals interleaved with escaped hex, while Rex::Text.hex_to_raw takes only escaped hex. Enhances -b/--bad-chars and --encrypt-key.

Please see rapid7/rex-text#22 and #9869.

-b/--bad-chars

wvu@kharak:/rapid7/metasploit-framework:feature/msfvenom$ ./msfvenom -p generic/custom -e generic/none -b " " PAYLOADSTR="hello world"
[-] No platform was selected, choosing Msf::Module::Platform from the payload
[-] No arch selected, selecting arch: x86 from the payload
Found 1 compatible encoders
Attempting to encode payload with 1 iterations of generic/none
generic/none failed with Encoding failed due to a bad character (index=5, char=0x20)
Error: An encoding exception occurred.
wvu@kharak:/rapid7/metasploit-framework:feature/msfvenom$

--encrypt-key

wvu@kharak:/rapid7/metasploit-framework:feature/msfvenom$ ./msfvenom -p generic/custom --encrypt xor --encrypt-key "\x0f" PAYLOADSTR="hello world"
[-] No platform was selected, choosing Msf::Module::Platform from the payload
[-] No arch selected, selecting arch: x86 from the payload
No encoder or badchars specified, outputting raw payload
Payload size: 11 bytes
gjcc`/x`}ck
wvu@kharak:/rapid7/metasploit-framework:feature/msfvenom$

wvu@kharak:/rapid7/metasploit-framework:feature/msfvenom$ ./msfvenom -p generic/custom --encrypt xor --encrypt-key "\x00\x00\x00\x00\x00\x0c" PAYLOADSTR="hello world"
[-] No platform was selected, choosing Msf::Module::Platform from the payload
[-] No arch selected, selecting arch: x86 from the payload
No encoder or badchars specified, outputting raw payload
Payload size: 11 bytes
hello,world
wvu@kharak:/rapid7/metasploit-framework:feature/msfvenom$

wvu@kharak:/rapid7/metasploit-framework:feature/msfvenom$ ./msfvenom -p generic/custom --encrypt xor --encrypt-key "hello world" PAYLOADSTR="hello world" | xxd -g 1
[-] No platform was selected, choosing Msf::Module::Platform from the payload
[-] No arch selected, selecting arch: x86 from the payload
No encoder or badchars specified, outputting raw payload
Payload size: 11 bytes

00000000: 00 00 00 00 00 00 00 00 00 00 00                 ...........
wvu@kharak:/rapid7/metasploit-framework:feature/msfvenom$

Fixes e344adb.

[wvu]

Release Notes

The msfvenom's bad character and encryption key specification now accepts character literals and escaped hex.