Auxiliary failed: ArgumentError invalid byte sequence in UTF-8

[acheong08]

Steps to reproduce

How'd you do it?

1. use admin/mssql/mssql_exec
2. set RHOSTS <vulnerable_target>
3. set CMD cmd.exe /c echo <some_chinese_characters>

Expected behavior

Return Chinese characters

Current behavior

[*] Running module against 172.26.0.24

[*] 172.26.0.24:1433 - SQL Query: EXEC master..xp_cmdshell 'cmd.exe /c echo 你好'
[-] 172.26.0.24:1433 - Auxiliary failed: ArgumentError invalid byte sequence in UTF-8
[-] 172.26.0.24:1433 - Call stack:
[-] 172.26.0.24:1433 -   /opt/metasploit-framework/embedded/lib/ruby/gems/2.7.0/gems/rex-text-0.2.34/lib/rex/text/wrapped_table.rb:413:in `split'
[-] 172.26.0.24:1433 -   /opt/metasploit-framework/embedded/lib/ruby/gems/2.7.0/gems/rex-text-0.2.34/lib/rex/text/wrapped_table.rb:413:in `block in chunk_values'
[-] 172.26.0.24:1433 -   /opt/metasploit-framework/embedded/lib/ruby/gems/2.7.0/gems/rex-text-0.2.34/lib/rex/text/wrapped_table.rb:410:in `each'
[-] 172.26.0.24:1433 -   /opt/metasploit-framework/embedded/lib/ruby/gems/2.7.0/gems/rex-text-0.2.34/lib/rex/text/wrapped_table.rb:410:in `each_with_index'
[-] 172.26.0.24:1433 -   /opt/metasploit-framework/embedded/lib/ruby/gems/2.7.0/gems/rex-text-0.2.34/lib/rex/text/wrapped_table.rb:410:in `each'
[-] 172.26.0.24:1433 -   /opt/metasploit-framework/embedded/lib/ruby/gems/2.7.0/gems/rex-text-0.2.34/lib/rex/text/wrapped_table.rb:410:in `map'
[-] 172.26.0.24:1433 -   /opt/metasploit-framework/embedded/lib/ruby/gems/2.7.0/gems/rex-text-0.2.34/lib/rex/text/wrapped_table.rb:410:in `chunk_values'
[-] 172.26.0.24:1433 -   /opt/metasploit-framework/embedded/lib/ruby/gems/2.7.0/gems/rex-text-0.2.34/lib/rex/text/wrapped_table.rb:395:in `row_to_s'
[-] 172.26.0.24:1433 -   /opt/metasploit-framework/embedded/lib/ruby/gems/2.7.0/gems/rex-text-0.2.34/lib/rex/text/wrapped_table.rb:125:in `block in to_s'
[-] 172.26.0.24:1433 -   /opt/metasploit-framework/embedded/lib/ruby/gems/2.7.0/gems/rex-text-0.2.34/lib/rex/text/wrapped_table.rb:121:in `each'
[-] 172.26.0.24:1433 -   /opt/metasploit-framework/embedded/lib/ruby/gems/2.7.0/gems/rex-text-0.2.34/lib/rex/text/wrapped_table.rb:121:in `to_s'
[-] 172.26.0.24:1433 -   /opt/metasploit-framework/embedded/framework/lib/msf/core/exploit/remote/mssql.rb:675:in `mssql_print_reply'
[-] 172.26.0.24:1433 -   /opt/metasploit-framework/embedded/framework/lib/msf/core/exploit/remote/mssql.rb:145:in `mssql_xpcmdshell'
[-] 172.26.0.24:1433 -   /opt/metasploit-framework/embedded/framework/modules/auxiliary/admin/mssql/mssql_exec.rb:31:in `run'
[*] Auxiliary module execution completed

Metasploit version

Framework Version: 6.0.40-dev-aaf27d7fa51390895dea63c58cb3b76e959d36f8

###Additional Information

It works with set CMD cmd.exe /c echo <English_Characters>

[acheong08]

@adfoster-r7 I understand there are a lot of ascii and utf-8 issues with other languages. Are there any workarounds or planned fixes soon?

[gwillcox-r7]

@acheong08 We have been making some progress towards fixing these types of errors however as it stands the current case is that some areas of the framework are hardcoded to use ASCII over UTF-8, others prefer UTF-8 vs ASCII, and some can accept either. Trying to fix this completely will likely take some time as there are lots of side effects to consider if we want a complete and accurate fix without introducing odd behavior.

For this particular case it would appear that either the rex-text gem needs to be updated. I've checked the code for the other lines and only saw this line:

metasploit-framework/lib/msf/core/exploit/remote/mssql.rb

Line 675 in 6ecaeaf

print_line(tbl.to_s)

Even then that line doesn't actually do anything r.e the encoding as we can see from the example below:

 ~/git/metasploit-framework │ upstream-master:master *1 ?16  irb                                              ✔ │ 4s │ 2.7.2 Ruby 
2.7.2 :001 > '你好'
 => "你好" 
2.7.2 :002 > '你好'.to_s
 => "你好" 
2.7.2 :003 > '你好'.encoding
 => #<Encoding:UTF-8> 
2.7.2 :004 > '你好'.to_s.encoding
 => #<Encoding:UTF-8> 
2.7.2 :005 > cmd = '你好'
 => "你好" 
2.7.2 :006 > "cmd.exe /c echo #{cmd}".encoding
 => #<Encoding:UTF-8> 
2.7.2 :007 > "cmd.exe /c echo #{cmd}".to_s.encoding
 => #<Encoding:UTF-8> 
2.7.2 :008 > 

Seems the issue is somewhere within the rex-text gem, specifically https://github.com/rapid7/rex-text/blob/df76ccf7008fe87bcb7a307b115bb37b7471d0d0/lib/rex/text/wrapped_table.rb as the error occurs when trying to execute https://github.com/rapid7/rex-text/blob/df76ccf7008fe87bcb7a307b115bb37b7471d0d0/lib/rex/text/wrapped_table.rb#L413

[adfoster-r7]

Thanks for taking a look Grant; Might be a regression in the rex-tables that should be fixed. I've created an internal ticket for this sprint 👍

[cgranleese-r7]

Hi @acheong08

Unfortunately I wasn't able to replicate this issue.

We'd appreciate if you would be able to confirm some information, could you provide more details on the host machine and terminal configuration you were using.

Could you please try this command and let send a screenshot of your output please, just so we can compare.

Could you also run systeminfo on the target, attached a screenshot of our output below.

If we could get these details, it may help us investigate this issue further.

Thanks

[acheong08]

[acheong08]

From wireshark capture:
���çþ���Ð�outputÑÿÿÑ:;N:g T: YCISCQ-CANTEENÑ�OS Tðy: Microsoft(R) Windows(R) Server 2003, Enterprise EditionÑdOS Hr,g: 5.2.3790 Service Pack 2 Build 3790ÑHOS 6R �FU: Microsoft CorporationÑ*OS M�n�: ìrËz g¡RhVÑBOS �göN{|�W: Multiprocessor FreeÑ$èl�Q�v@b gºN: ycis-1Ñ�èl�Q�vÄ~Ç~: ÑN§NÁT ID: 69813-640-2643913-45150Ñ@�RËY�[Å�åe�g: 2015-10-16, 12:37:49Ñ@û|ß~/T¨Röeô�: 23 )Y 16 �\öe 23 �R 46 ÒyÑ2û|ß~6R �FU: VMware, Inc.ÑJû|ß~�W÷S: VMware Virtual PlatformÑ4û|ß~{|�W: X86-based PCÑ4�Y�thV: �[Å��N 2 *N�Y�thV�0Ñ� [01]: x86 Family 6 Model 44 Stepping 2 GenuineIntel ~2132 MhzÑ� [02]: x86 Family 6 Model 44 Stepping 2 GenuineIntel ~2133 MhzÑ@BIOS Hr,g: INTEL - 6040000Ñ4Windows îvU_:

[ruant]

Just wanted to add that this happens in ls and download too when there is funny looking folder/filenames:

meterpreter > ls -la
[-] Error running command ls: ArgumentError invalid byte sequence in UTF-8
meterpreter > download -r ./
[-] Error running command download: ArgumentError invalid byte sequence in UTF-8

[acheong08]

Problem appears in non-English windows versions

[ruant]

For what it's worth: I encountered this on my Kali, while browsing an xubuntu machine.

[bcoles]

@ruant are you using the latest version of Metasploit? A similar issue was fixed in #15666 a week ago.

[skullcat2008]

I met the same problem in my kali system. How do you solve it ?

[ruant]

@bcoles @skullcat2008
Wow, must have missed both these comments. Sorry guys 🙏

It's been a while, but I can't remember managing to fix this issue.

[bcoles]

I met the same problem in my kali system. How do you solve it ?

Is your issue with the admin/mssql/mssql_exec module or something else?

[ruant]

@bcoles For me it was just listing files from a directory and recursively downloading that directory when in a session to the remote machine.
There was a file in that directory with weird chars/encoding IIRC.

So maybe not the exact same issue, but maybe there is some kinda common encoder used across the entire framework? That affects several parts?

[bcoles]

So maybe not the exact same issue, but maybe there is some kinda common encoder used across the entire framework? That affects several parts?

It is unlikely to be the same issue. There are many. ArgumentError invalid byte sequence in UTF-8 is a generic encoding error message.

The original issue in this thread describes as issue in admin/mssql/mssql_exec which hasn't been fixed.

You mentioned:

Just wanted to add that this happens in ls and download too when there is funny looking folder/filenames

I think those have been fixed. But there may be more.

[ruant]

@bcoles Ok. Then I'm sorry for hijacking this thread and will stop.
Thanks for replying.

[gwillcox-r7]

The crash should be fixed in the next release by #16729. If this is still an issue after using this release, which went out yesterday, then feel free to open this again.