Debug cmd_exec again #15362

bwatters-r7 · 2021-06-22T16:39:37Z

Please wait for me to do some automated testing to verify this works across all supported Windows versions and no regressions were introduced to Linux. Automated testing is easier on PRs, so I've only done some basic tests locally. I'll leave it in draft until I finish the automated testing.

This is yet another attempt to fix some of the bugs in cmd_exec. I noticed that the cmd_exec tests were failing on Windows shells and after digging a bit, I don't think the write_file or append_file functionality ever worked except under some very contrived circumstances, namely, it would not work on binary files that contained non-ansi characters or any file larger than ~ 7K.

Further, because of some interesting sharing of code between *nix and windows shells, the tokenization is sloppy with regard to newlines- sometimes Windows newlines, sometimes *nix-style newlines, depending on whether we send the token, or Windows echos the token. Rather than go back and split the code that sends the token, I just normalize the newlines before processing responses. It is certainly feasible to fix this on the sending end, but this approach also works. It also allows me to remove the abomination I added in #12463 because the trailing newlines are corrected upstream. 🎉

Finally, there was a rescue that hid everything rather than just the expected timeout errors, so when any number or errors got thrown, it defaulted to that stupid no method reverse! for nil message because it was the first thing following that empty rescue all.

One concern I have is that we're making a awful lot of files on target for binary file appends- file_name.b64, file_name.tmp, and file_name and I am slightly concerned we might run into some collisions, especially with the *.tmp. I'd be curious to know what others think about that and if I'm just being overly paranoid.

Verification

List the steps needed to make sure this thing works

Old and busted

msf6 payload(windows/x64/shell_reverse_tcp) > [*] Started reverse TCP handler on 192.168.135.197:4444 
[*] Command shell session 1 opened (192.168.135.197:4444 -> 192.168.132.125:49676) at 2021-06-22 11:30:48 -0500

msf6 payload(windows/x64/shell_reverse_tcp) > loadpath test/modules/
Loaded 37 modules:
    14 auxiliary modules
    13 exploit modules
    10 post modules
msf6 payload(windows/x64/shell_reverse_tcp) > use post/test/file 
msf6 post(test/file) > set session 1
session => 1
msf6 post(test/file) > set verbose truye
[-] The following options failed to validate: Value 'truye' is not valid for option 'VERBOSE'.
verbose => false
msf6 post(test/file) > set verbose true
verbose => true
msf6 post(test/file) > run

[-] Post failed: NoMethodError undefined method `reverse!' for nil:NilClass
[-] Call stack:
[-]   /home/tmoose/rapid7/metasploit-framework/lib/msf/core/session/provider/single_command_shell.rb:136:in `shell_command_token_win32'
[-]   /home/tmoose/rapid7/metasploit-framework/lib/msf/core/session/provider/single_command_shell.rb:84:in `shell_command_token'
[-]   /home/tmoose/rapid7/metasploit-framework/lib/msf/core/post/file.rb:56:in `pwd'
[-]   /home/tmoose/rapid7/metasploit-framework/test/modules/post/test/file.rb:38:in `setup'
[*] Cleanup: changing working directory back to 
[-] Post failed: NoMethodError undefined method `reverse!' for nil:NilClass
[-] Call stack:
[-]   /home/tmoose/rapid7/metasploit-framework/lib/msf/core/session/provider/single_command_shell.rb:136:in `shell_command_token_win32'
[-]   /home/tmoose/rapid7/metasploit-framework/lib/msf/core/session/provider/single_command_shell.rb:84:in `shell_command_token'
[-]   /home/tmoose/rapid7/metasploit-framework/lib/msf/core/post/file.rb:38:in `cd'
[-]   /home/tmoose/rapid7/metasploit-framework/test/modules/post/test/file.rb:172:in `cleanup'
msf6 post(test/file) >

New Hotness

msf6 payload(windows/x64/shell_reverse_tcp) > 
[*] Started reverse TCP handler on 192.168.135.197:4444 
[*] Command shell session 1 opened (192.168.135.197:4444 -> 192.168.132.185:49165) at 2021-06-22 09:51:13 -0500

msf6 payload(windows/x64/shell_reverse_tcp) > loadpath test/modules/
Loaded 37 modules:
    14 auxiliary modules
    13 exploit modules
    10 post modules
msf6 payload(windows/x64/shell_reverse_tcp) > use post/test/file 
msf6 post(test/file) > set session 1
session => 1
msf6 post(test/file) > set verbose true
verbose => true
msf6 post(test/file) > run

[*] Setup: changing working directory to %TEMP%
[*] Running against session 1
[*] Session type is shell and platform is windows
[+] should test for file existence
[+] should test for directory existence
[+] should create text files
[+] should read the text we just wrote
[+] should append text files
[+] should delete text files
[+] should move files
[*] Writing 39256 bytes
[*] Finished in 0.265905096
[+] should write binary data
[*] Read 39256 bytes
[+] should read the binary data we just wrote
[+] should delete binary files
[+] should append binary data
[*] Testing complete in 1.197730979
[*] Passed: 11; Failed: 0
[*] Cleanup: changing working directory back to C:\Users\msfuser\Desktop

[*] Post module execution completed
msf6 post(test/file) > sessions

Also, I just realized I need to update some copy/pasta comments.....

…r RSPEC

bwatters-r7 · 2021-06-23T21:20:54Z

Testing

The good

The bad

While checking for regressions on the *nix side, it turns out it is all broken, but through testing, it was broken in the main branch before I changed anything.

msf6 payload(linux/x64/shell/reverse_tcp) > [*] Command shell session 1 opened (192.168.135.197:4444 -> 192.168.134.200:48734) at 2021-06-23 13:12:07 -0500
loadpath test/modules/
Loaded 37 modules:
    14 auxiliary modules
    13 exploit modules
    10 post modules
msf6 payload(linux/x64/shell/reverse_tcp) > use post/test/file 
msf6 post(test/file) > set session 1
session => 1
msf6 post(test/file) > run

[*] Running against session 1
[*] Session type is shell and platform is linux
[+] should test for file existence
[-] FAILED: should test for directory existence
[-] FAILED: should create text files
[-] Exception: RuntimeError : Can't find command on the victim for writing binary data
[-] Didn't read what we wrote, actual file on target: 
[-] FAILED: should read the text we just wrote
[-] FAILED: should append text files
[-] Exception: RuntimeError : Can't find command on the victim for writing binary data
[+] should delete text files
[+] should move files
[-] FAILED: should write binary data
[-] Exception: RuntimeError : Can't find command on the victim for writing binary data
[-] FAILED: should read the binary data we just wrote
[-] Exception: NoMethodError : undefined method `length' for nil:NilClass
[+] should delete binary files
[-] FAILED: should append binary data
[-] Exception: RuntimeError : Can't find command on the victim for writing binary data
[-] Passed: 9; Failed: 2
[*] Post module execution completed
msf6 post(test/file) >

lib/msf/core/post/file.rb

adfoster-r7 · 2021-06-24T13:52:16Z

lib/msf/core/post/file.rb

  # Platform-agnostic file write. Writes given object content to a remote file.
  #
-  # NOTE: *This is not binary-safe on Windows shell sessions!*
-  #


lib/msf/core/post/file.rb

adfoster-r7 · 2021-06-24T14:09:51Z

lib/msf/core/session/provider/single_command_shell.rb

            end
          end
        end
      end
-    rescue
-      # nothing, just continue


Bit on the fence about removing the rescues here, as we might have modules relying on nil being returned - rather than exploding. Even if exploding is better/expected behavior 🤔

Let's test this a bit more against linux targets too (not specifically file read/writes), and if it's good - we'll land it, and wait for the screaming 😱

lib/msf/core/post/file.rb

sempervictus · 2021-07-07T16:31:11Z

That's some epic windows shell-fu.
One thing i'd note here is that certutil invocations are often trapped-for in sysinternals collection, so using it naively for inline shell operations could create a lot of unintended noise. I guess you'd really expect folks to be upgrading session to meterp as soon as they could, but if the metserv itself is flagging and they dont know how to make their own evasive builds, then the environment is likely to be on its toes and have a higher chance of catching the LotL invocations. Not saying "this is bad" by any means, but if we have some other way to do the decoding, it might serve us better to use it for evasion purposes.

bwatters-r7 · 2021-07-12T14:02:43Z

@sempervictus I completely agree on certutil. I was hoping for something less obvious than invoking certutil, because it is the go-to for base64 interaction on the Windows side, and I figured it was likely to be monitored. Unfortunately, the reason it is the go-to is because it is one of the only ways covered in Windows 7-10 I've found to do base64 encode/decode.
The bright side is that if there's something I'm not aware of, we can switch it in pretty quickly. I briefly thought about using powershell as the base64 manipulator, but given the increased attention to powershell invocations, I was not sure it would be a much better than certutil. I also wanted to try and avoid powershell in the cmd shell libraries (though we use powershell shimmed in a lot of places, so I'd be interested in opinions on if leveraging powershell to do the base64 work makes more sense than certutil). It might not even be a bad idea to add a method for base64 decoding and encoding remote files into the powershell library somewhere (if it is not already there....) even if we don't use it here.

sjanusz-r7 · 2021-07-14T13:08:38Z

@agalway-r7 and myself have tested the above code and it seems to work great!

We did encounter one bug that we think is unrelated. When we get an initial shell with the windows/x64/shell/reverse_tcp payload using exploit/windows/smb/ms17_010_eternalblue, said shell becomes unresponsive after we upgrade it with post/multi/manage/shell_to_meterpreter.

We haven't tested this with other initial sessions. We wanted to check if this is a known issue before testing further.
Either way, this is almost certainly a different issue worth its own PR, so we are happy to land the above.

FYI, our environment:

Framework: 6.0.49-dev-2effd46bc6
Ruby: ruby 2.7.3p183 (2021-04-05 revision 6847ee089d) [x86_64-linux-gnu]
Install Root: /home/kali/Documents/metasploit-framework
Session Type: postgresql selected, no connection
Install Method: Git Clone

timwr

Looks good. This seems to fix #15311 and fixes the cmd_exec tests on both windows/x64/shell_reverse_tcp and cmd/windows/reverse_powershell sessions.

agalway-r7 · 2021-07-14T16:46:55Z

Double checked this hasn't broken linux 👍

agalway-r7 · 2021-07-15T09:26:08Z

Release Notes

Fixes a regression issue with post/multi/manage/shell_to_meterpreter, and other interactions with command shell based sessions

bwatters-r7 added 3 commits June 16, 2021 16:22

File tests work except for writing binary data; debugging commented out

8c09fc5

Fixed append and file write for windows

e24dc80

Burn all silent rescue alls to the ground

71db2fd

bwatters-r7 marked this pull request as draft June 22, 2021 16:39

bwatters-r7 added cmd_exec is broken again cmd_exec is broken again rn-fix release notes fix labels Jun 22, 2021

Update comments and refactor substring range to Ruby 2.5 standards fo…

a876500

…r RSPEC

bwatters-r7 marked this pull request as ready for review June 23, 2021 21:21

bwatters-r7 mentioned this pull request Jun 24, 2021

Fix dir method for windows shell sessions #15303

Merged