Expand environment variables on Linux

[smashery]

This PR resolves #14357. The expand_path function previously only worked on Windows; with this, it also works on Unix.

The approach I took was the one suggested by @timwr in the issue discussion: to do pattern-matching in the Framework itself, and then just request environment variables from Meterp, and slot them in.

The advantage of this approach is that the pattern-matching would have needed to be implemented somewhere for Mettle, since the built-in function to do it in Mettle's libc (musl) just launches sh, which isn't great from a forensics point of view. Doing it in Ruby rather than C lets us reduce code and leverage existing work in each of the other Meterps.

Implementing this parsing manually means there are some rarer use cases that I haven't covered compared to a regular implementation of wordexp, such as escaping dollar signs (e.g. an actual folder called $home), or resolving home directories of other users (e.g. ~root => /root); happy to discuss if it's felt these cases are important. (Now that I'm looking at it, I can't figure out a way to do the equivalent in Windows i.e. cding into a directory that is actually called %TEMP%).

Interestingly, I'd initially thought Python's os.path.expandvars would be an easy win for the Python meterpreter, but even that doesn't do dollar-escaping of backslashes or quotes... in short, if we wanted a completely consistent implementation, it seems like we'd need to manually implement parts of it for each of the meterp implementations anyway, so doing the pattern matching in one spot seemed much simpler overall.

Verification

• Start msfconsole
• Get a meterpreter session on Linux
• irb
• fs.file.expand_path('~')
• Verify this resolves to the user's home directory
• fs.file.expand_path('$HOME')
• Verify this resolves to the user's home directory
• Verify that using environment variables or tilde work in the commands cd, mkdir, rmdir, rm, mv, cp, chmod, ls, search, cat, checksum, download, upload, edit
• Verify that Windows meterpreter's use of path expansion (e.g. %TEMP%) still behaves

[smcintyre-r7]

Just a note for testers, we should determine what this will do when an environment variable can't be expanded because it doesn't exist in the target environment. We'll need to check if it'll be left in the resulting string or removed altogether.

[gwillcox-r7]

Overall not bad though I did have a few concerns and some suggestions.

[gwillcox-r7]

meterpreter > cd $rvm_bin_path
[-] stdapi_fs_chdir: Operation failed: 2
meterpreter > shell
Process 7637 created.
Channel 2 created.
env
USER=gwillcox
HOME=/home/gwillcox
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/system/bin:/system/sbin:/system/xbin
LANG=C
PWD=/home/gwillcox

pwd
/home/gwillcox
cd /
pwd
/
cd ${HOME}/gwillcox
/bin/sh: 6: cd: can't cd to /home/gwillcox/gwillcox
cd ${HOME}/   
pwd
/home/gwillcox

meterpreter > irb
[*] Starting IRB shell...
[*] You are in the "client" (session) object

>> fs.file.expand_path('~')
=> "/home/gwillcox"
>> fs.file.expand_path('$HOME')
=> "/home/gwillcox"
>> 

Seems to be working fine, the errors were mostly cause for some reason the Meterpreter on Linux isn't picking up about 90% of my environment variables. First I thought this might be cause I'm not using bash, but even that has environment variables that aren't showing up here.

[gwillcox-r7]

Oh and to be clear I'm not landing this yet as the issue that is still open needs to be resolved first but once that is fixed this should be good to go and land.

[gwillcox-r7]

Oh and Windows is still working (forgot to test earlier):

meterpreter > sysinfo
Computer        : WIN-86UKQGF31O1
OS              : Windows 2016+ (10.0 Build 14393).
Architecture    : x64
System Language : en_US
Domain          : WORKGROUP
Logged On Users : 1
Meterpreter     : x64/windows
meterpreter > pwd
C:\Users\Administrator\Desktop
meterpreter > cd %TEMP%
meterpreter > pwd
C:\Users\ADMINI~1\AppData\Local\Temp\2
meterpreter > 

[gwillcox-r7]

One potential oddity found though:

meterpreter > shell
Process 8506 created.
Channel 5 created.
env
USER=gwillcox
HOME=/home/gwillcox
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/system/bin:/system/sbin:/system/xbin
LANG=C
PWD=/home/gwillcox
exit
meterpreter > cat gwillcox
root:x:0:0:root:/root:/bin/bash
daemon:x:1:1:daemon:/usr/sbin:/usr/sbin/nologin
bin:x:2:2:bin:/bin:/usr/sbin/nologin
....*CUT FOR BREVITY*....
meterpreter > cat $USER
[-] stdapi_fs_stat: Operation failed: 1
meterpreter > 

[smashery]

@gwillcox-r7 - Yeah, a bunch of the commands didn't use the expand_path functionality; I've extended it to these ones now:

• search -d
• cat
• checksum
• download
• upload
• edit

[gwillcox-r7]

Thanks @smashery, appreciate your hard work on this! I'll check this more in depth tomorrow morning as soon as I can 👍

[gwillcox-r7]

All tests completed successfully. We are cleared for landing 👍

[gwillcox-r7]

Release Notes

Updates have been made to Linux Meterpreter libraries to support expanding environment variables in several different commands. This should provide users with a smoother experience when using environment variables in commands such as cd, ls, download, upload, mkdir and similar commands.