Handle invalid payloads more clearly

[busterb]

Currently, if you have an error in a payload module (e.g. you have a syntax error in a module), the error is not noticed early enough in the generation process by the parameter validator, leading to a mysterious message like so from msfvenom, due to it failing to instantiate the payload module later in the process:

Error: undefined method `platform' for nil:NilClass

This change cleans up some of the error handling, checks to see if the payload
module can be instantiated, and gives a more useful error output for the
different cases. This also tweaks some of the literal string outputs to match
other exception strings from payload_generator.

Verification

• Generate an invalid payload: ./msfvenom -p linux/x86/shell_bind_tcp_random_portal
• Verify you get an error message:

$ ./msfvenom -p linux/x86/shell_bind_tcp_random_portal
Error: invalid payload: linux/x86/shell_bind_tcp_random_portal

• Modify a payload to have a syntax error (delete an end from a block, then try generating it: ./msfvenom -p linux/x86/shell_bind_tcp_random_port
• Verify you get a less-mysterious error message (you still need to look at the logs to see why it was unloadable):

$ ./msfvenom -p linux/x86/shell_bind_tcp_random_port
Error: unloadable payload: linux/x86/shell_bind_tcp_random_port

[asoto-r7]

I'm seeing a handful of spec errors, both via Travis here on Github, and locally:

rspec ./spec/lib/msf/core/payload_generator_spec.rb:182 # Msf::PayloadGenerator when creating a new generator when given a payload through stdin should not raise Exception
rspec ./spec/lib/msf/core/payload_generator_spec.rb:158 # Msf::PayloadGenerator when creating a new generator when given an invalid payload should raise ArgumentError with "Invalid Payload Selected"
rspec ./spec/lib/msf/core/payload_generator_spec.rb:134 # Msf::PayloadGenerator when creating a new generator when not given a payload should raise ArgumentError with "Invalid Payload Selected"
rspec ./spec/lib/msf/core/payload_generator_spec.rb:206 # Msf::PayloadGenerator when creating a new generator when given an invalid format should raise ArgumentError with "Invalid Format Selected"
rspec ./spec/lib/msf/core/payload_generator_spec.rb:518 # Msf::PayloadGenerator#generate_raw_payload when passing a payload through stdin returns the payload from stdin
rspec ./spec/lib/msf/core/payload_generator_spec.rb:494 # Msf::PayloadGenerator#generate_raw_payload when passing a payload through stdin when no platform has been selected raises an IncompatiblePlatform error
rspec ./spec/lib/msf/core/payload_generator_spec.rb:469 # Msf::PayloadGenerator#generate_raw_payload when passing a payload through stdin when no arch has been selected raises an IncompatibleArch error

I think most of the issue is that a few of the test cases will need updating, but I'm unclear as to why some seemingly unrelated test cases are failing now.

I'm happy to dig deeper later, but I'm going to call it a night for now.

[busterb]

Thanks, I had notice the test failures earlier, will update them to match.

[busterb]

Updated tests (and it found 1 bug, so fixed those too!)

[bwatters-r7]

effect

[bwatters-r7]

Release notes

Error messaging have been improved for fatal conditions that occur while generating payloads.