database host info from post modules #10823

h00die · 2018-10-17T16:48:10Z

Requires: #10857

The lib/msf/core/post/*/system.rb modules pull hostname, os and kernel, but never bother to database them. Seems like wasted effort, it should definitely be saved once we see it!

This PR adds to the system.rb libraries calls to database information we gather from post modules about the host.

I DO NOT HAVE AN OSX BOX TO TEST!!!!!!!!!!!!

Verification

Start msfconsole
get a shell
use a post module which calls the get_sysinfo call (for linux, get_hostname as well)
Verify hosts now has more data populated

h00die · 2018-10-17T16:49:11Z

@bcoles mind giving this a look? I Know you have a few more solaris boxes to test against, and while I <3 ubuntu, you tend to have other flavors.

h00die · 2018-10-17T16:52:12Z

Pre patch:

msf5 exploit(solaris/local/test) > hosts

Hosts
=====

address       mac  name  os_name  os_flavor  os_sp  purpose  info  comments
-------       ---  ----  -------  ---------  -----  -------  ----  --------
1.1.1.1

Post patch (spacing off due to ip rewrite):

msf5 exploit(solaris/local/test) > run

[!] SESSION may not be compatible with this module.
[*] Started reverse TCP handler on 2.2.2.2:4444 
[*] get_sysinfo
[*] {:version=>"Oracle Solaris 11.3 X86", :kernel=>"SunOS solaris11.3 5.11 11.3 i86pc i386 i86pc", :hostname=>"solaris11.3"}

msf5 exploit(solaris/local/test) > hosts

Hosts
=====

address       mac  name         os_name  os_flavor  os_sp  purpose  info  comments
-------       ---  ----         -------  ---------  -----  -------  ----  --------
2.2.2.2       solaris11.3  Solaris  11.3

msf5 exploit(linux/local/test) > run

[!] SESSION may not be compatible with this module.
[*] Started reverse TCP handler on 1.1.1.1:4444 
[*] get_sysinfo
[*] {:kernel=>"Linux ubuntu1604 4.4.0-137-generic #163-Ubuntu SMP Mon Sep 24 13:14:43 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux", :distro=>"ubuntu", :version=>"Ubuntu 16.04.4 LTS  "}
[*] get_hostname
[*] #<Mdm::Host:0x00005598c8268110>
[*] Exploit completed, but no session was created.
msf5 exploit(linux/local/test) > hosts

Hosts
=====

address        mac  name         os_name  os_flavor           os_sp  purpose  info  comments
-------        ---  ----         -------  ---------           -----  -------  ----  --------
3.3.3.3       ubuntu1604   ubuntu   Ubuntu 16.04.4 LTS

sempervictus

Does report host gracefully deal with no db? Looks sane and useful to me.

h00die · 2018-10-24T20:19:37Z

This should be ready to test now that the blocker has landed.

bcoles · 2018-11-02T01:17:49Z

I ran some tests on Linux and Solaris. I haven't had a chance to test OSX yet.

I also tried a couple quick tests without a database connected and encountered no issues.

While not directly relevant to this PR, on a somewhat related note, CentOS is incorrectly identified as amazon flavor.

    # Amazon
    elsif etc_files.include?("system-release")
      version = read_file("/etc/system-release").gsub(/\n|\\n|\\l/,'')
      system_data[:distro] = "amazon"
      system_data[:version] = version

[user@centos-7-1708 ~]$ cat /etc/system-release
CentOS Linux release 7.4.1708 (Core)

Linux

Hosts
=====

address         mac  name                       os_name  os_flavor                             os_sp  purpose  info  comments
-------         ---  ----                       -------  ---------                             -----  -------  ----  --------
123.45.67.139        manjaro-gnome-17-1-0       arch     Manjaro Linux                                               
123.45.67.141        centos-7-1708.localdomain  amazon   CentOS Linux release 7.4.1708 (Core)                         
123.45.67.188        kali                       debian   Kali GNU/Linux 1.0.6

Solaris

Hosts
=====

address         mac  name     os_name  os_flavor  os_sp  purpose  info  comments
-------         ---  ----     -------  ---------  -----  -------  ----  --------
123.45.67.221        solaris  Solaris  11.3                             
123.45.67.148        unknown  Solaris

It looks like the version detection could be improved. Here's the current version check:

    if system_data[:version] =~ /([\d]?\d\.\d)/
      host_info[:os_flavor] = $1
    end

Note that it only matches \d?\d\.\d which isn't a safe assumption prior to Solaris 11.x.

Here's the reported version from the system lib for a Solaris 9 box: :version=>"Solaris 9 9/04 s9x_u7wos_09 x86".

The host flavor reporting portion of the method should probably trust the system lib. Same as the Linux and OSX flavor reporting.

Misc

I did encounter this error, twice, but I'm not sure what caused it, and wasn't able to reproduce it.

msf5 exploit(solaris/local/test) > check

[!] SESSION may not be compatible with this module.
[*] get_sysinfo: {:version=>"Solaris 9 9/04 s9x_u7wos_09 x86", :kernel=>"SunOS unknown 5.9 Generic_117172-07 i86pc i386 i86pc", :hostname=>"unknown"}
[*] get_hostname: unknown
[+]  The target is vulnerable.
msf5 exploit(solaris/local/test) > hosts
[-] Error while running command hosts: Problem reporting event: PG::ConnectionBad: PQconsumeInput() SSL error: BN lib
: BEGIN. See log for more details.

Call stack:
/pentest/exploit/metasploit-framework/lib/metasploit/framework/data_service/proxy/core.rb:174:in `log_error'
/pentest/exploit/metasploit-framework/lib/metasploit/framework/data_service/proxy/event_data_proxy.rb:9:in `rescue in report_event'
/pentest/exploit/metasploit-framework/lib/metasploit/framework/data_service/proxy/event_data_proxy.rb:4:in `report_event'
/pentest/exploit/metasploit-framework/lib/msf/core/framework.rb:300:in `report_event'
/pentest/exploit/metasploit-framework/lib/msf/core/framework.rb:349:in `on_ui_command'
/pentest/exploit/metasploit-framework/lib/msf/core/event_dispatcher.rb:183:in `block in method_missing'
/pentest/exploit/metasploit-framework/lib/msf/core/event_dispatcher.rb:181:in `each'
/pentest/exploit/metasploit-framework/lib/msf/core/event_dispatcher.rb:181:in `method_missing'
/pentest/exploit/metasploit-framework/lib/msf/ui/console/command_dispatcher.rb:24:in `block in initialize'
/pentest/exploit/metasploit-framework/lib/rex/ui/text/dispatcher_shell.rb:452:in `block in run_single'
/pentest/exploit/metasploit-framework/lib/rex/ui/text/dispatcher_shell.rb:447:in `each'
/pentest/exploit/metasploit-framework/lib/rex/ui/text/dispatcher_shell.rb:447:in `run_single'
/pentest/exploit/metasploit-framework/lib/rex/ui/text/shell.rb:151:in `run'
/pentest/exploit/metasploit-framework/lib/metasploit/framework/command/console.rb:48:in `start'
/pentest/exploit/metasploit-framework/lib/metasploit/framework/command/base.rb:82:in `start'
./msfconsole:49:in `<main>'
msf5 exploit(solaris/local/test) >

lib/msf/core/post/osx/system.rb

bcoles · 2018-11-02T04:14:31Z

Tested on OSX. There's a typo that needs to be fixed, as per above. Apart from that, it worked as described.

msf5 exploit(osx/local/test) > check

[!] SESSION may not be compatible with this module.
[*] get_sysinfo: {"ProductName"=>"Mac OS X", "ProductVersion"=>"10.7.4", "BuildVersion"=>"11E53", "Kernel"=>"Darwin my-MacBook.local 11.4.0 Darwin Kernel Version 11.4.0: Mon Apr  9 19:33:05 PDT 2012; root:xnu-1699.26.8~1/RELEASE_I386 i386", "Hostname"=>"my-MacBook.local"}
[*]  Cannot reliably check exploitability.
msf5 exploit(osx/local/test) > hosts

Hosts
=====

address      mac  name               os_name  os_flavor                                             os_sp  purpose  info  comments
-------      ---  ----               -------  ---------                                             -----  -------  ----  --------
12.34.56.78       my-MacBook.local   osx      Darwin my-MacBook.local 11.4.0 Darwin Kernel Version

h00die · 2018-11-02T15:38:13Z

The OSX os_flavor is particularly ugly. It may be worth stripping out the hostname from that, but would need to check if any modules would be effected or not (note to self).

bcoles · 2018-11-03T01:03:27Z

Agreed.

The OSX libraries have been neglected. I only have a Darwin system to test.

The os_flavor could be improved, but the existing parsing to split by space may be problematic. This may or may not cause problems with systems such as Snow Leopard, Mountain Lion, etc.

h00die · 2018-11-03T14:44:34Z

@bcoles
For the solaris version, current nomenclature is x.x, previously it was x/x. I'm thinking when its changed to standardize to x.x so 9/04 will be 9.04. Thoughts?
Of note, this data isn't used anywhere, so there will be no regression impacts

bcoles · 2018-11-03T15:22:38Z

@h00die 9/04 is the update release date in mm/yy format. It's just a coincidence that it happens to also be a released date for a version of Solaris 9.

Prior to 11.x the typical naming pattern was in the form of <VERSION>u<UPDATE> (mm/yy). For example, 9u7 (09/04) denotes Solaris 9 Update 7 released September 2004.

Using <VERSION>u<UPDATE> or <VERSION>.<UPDATE> would probably make more sense. Or alternatively, using the SunOS version. ie, 5.9 for Solaris 9.

… into hosts_data

h00die · 2018-11-03T21:14:40Z

resource (solaris.rb)> hosts

Hosts
=====

address        mac  name         os_name  os_flavor  os_sp  purpose  info  comments
-------        ---  ----         -------  ---------  -----  -------  ----  --------
1.1.1.1        solaris11.3  Solaris  11.3                             
2.2.2.2       solaris10u2  Solaris  10.2

@bcoles i always hated the 5.x solaris numbering scheme. I adjusted the regex so that it should catch <11 now and make it consistent. I tested w/ all my solaris boxes, let me know how the <11 testing goes!

bcoles · 2018-11-04T03:06:08Z

@h00die How far down the rabbit hole do you want to go? See #10437 for a bunch of example system_info hashes.

x86 / x64 / Intel Architecture

The regex will run into issues with Solaris Nevada (10u3) and Solaris 8:

{:version=>"Solaris Nevada snv_50 X86", :kernel=>"SunOS unknown 5.11 snv_50 i86pc i386 i86pc", :hostname=>"unknown"}

{:version=>"Solaris 8 1/01 s28x_u3wos_08 INTEL", :kernel=>"SunOS unknown 5.8 Generic_108529-05 i86pc i386 i86pc", :hostname=>"unknown"}

SPARC

The regex won't work for SPARC, which makes use of s rather than x in the uname string.

The versioning also gets a whole lot trickier when taking into account rebuilds and kernel patches which also modify the uname output.

bcoles · 2018-11-04T03:08:19Z

There's also the issue of the hardcoded Solaris string for os_name, which arguably should reflect the variety of Solaris/SunOS; ie, OpenSolaris, OpenIndiana, ...

Edit: It appears OpenSolaris does not reveal its presence in the uname -a output. OpenIndiana uses illumos in the uname string. Without parsing /etc/release, hardcoded Solaris will have to do.

Issue with OSX lib has been resolved

h00die · 2018-11-05T01:42:54Z

@bcoles Ok, I'd like to not make a 90+ character regex to catch it all the solaris pokemon, so I broke it down in to 3. I like the idea of hardcoding Solaris as I think it makes life easier for GUI developers, but meh.

test strings:

Solaris 9 9/04 s9x_u7wos_09 x86
Solaris 10 /0 s10x_u2wos_09a x86
Oracle Solaris 11.3 X86
Oracle Solaris 11.4 X86
Oracle Solaris 11.1 X86
Oracle Solaris 10 1/13 s10x_u11wos_24a X86
Solaris 10 8/07 s10x_u4wos_12b X86
Solaris 10 6/06 s10x_u2wos_09a X86
Solaris 10 1/06 s10x_u1wos_19a X86
Solaris Nevada snv_50 X86
Solaris 9 9/04 s9x_u7wos_09 x86
Solaris 8 1/01 s28x_u3wos_08 INTEL
OpenSolaris 2009.06 snv_111b X86
OpenIndiana Hipster 2018.04 (powered by illumos)

Regexes which I believe catch everything correctly.

/(?<OS>(?<!Open|Oracle )Solaris).+s2?(?<major>\d?\d)[x|s]?(_u)(?<minor>\d?\d)/
/(?<OS>Oracle Solaris) (?<major>\d\d)\.(?<minor>\d?\d)/
/(?<OS>OpenSolaris|OpenIndiana Hipster) (?<major>\d\d\d\d)\.(?<minor>\d\d)/

If these look good to you, i'll implement them, and this can be landed.

Also keep in mind, if something isn't caught properly by a regex, it'll skip the DBing which is more or less the current behavior, so no big loss.

PS, the more you teach me about solaris and its derivatives, the more I don't like it.

bcoles · 2018-11-05T03:00:01Z

Looks reasonable.

Hipster is the release name, so probably shouldn't be hardcoded in the regex.

In the long term, it might be better to implement parsing of /etc/release.

Edit: I realize this PR was intended to simply catch and report information that would have been ignored otherwise. It doesn't break anything, and I agree with the version.update formatting.

Edit 2: There's also no existing convention for reporting host details. It's taken years for the database to become popular. Many modules still don't report hosts, services or vulns. I'm keen to get this landed and move on. We can circle back to improving the Solaris libs later.

PS, the more you teach me about solaris and its derivatives, the more I don't like it.

It's a matter of perspective. The path to Solaris enlightenment starts with throwing preconceived ideas out the window. Solaris cares not for IBM / MS propaganda.

h00die · 2018-11-06T00:26:59Z

@bcoles should be good to land 😉

bcoles · 2018-11-06T02:36:50Z

Test cases look good. WFM.

Hosts
=====

address         mac  name     os_name  os_flavor  os_sp  purpose  info  comments
-------         ---  ----     -------  ---------  -----  -------  ----  --------
12.34.56.147         unknown  Solaris  8.3                              
12.34.56.148         unknown  Solaris  9.7

bcoles · 2018-11-06T02:47:14Z

Release Notes

This updates the post/system libraries for Linux, OS X, and Solaris to store retrieved host system information in the database.

database host info from post modules

e78b760

h00die added the blocked Blocked by one or more additional tasks label Oct 17, 2018

sempervictus approved these changes Oct 19, 2018

View reviewed changes

h00die removed the blocked Blocked by one or more additional tasks label Oct 24, 2018

bcoles added library enhancement labels Oct 28, 2018

bcoles previously requested changes Nov 2, 2018

View reviewed changes

lib/msf/core/post/osx/system.rb Outdated Show resolved Hide resolved

h00die added 2 commits November 3, 2018 17:05

enhancements to solaris host info db regex

ca0249c

Merge branch 'master' of https://github.com/rapid7/metasploit-framework…

7326453

… into hosts_data

h00die force-pushed the hosts_data branch from d718487 to 7326453 Compare November 3, 2018 21:11

additional solaris-ish regexes

e7f5c0c

bcoles added the rn-enhancement release notes enhancement label Nov 6, 2018

bcoles merged commit e7f5c0c into rapid7:master Nov 6, 2018

bcoles added a commit that referenced this pull request Nov 6, 2018

Land #10823, store host system data from post/system libs in database

a8ff9b2

msjenkins-r7 pushed a commit that referenced this pull request Nov 6, 2018

Land #10823, store host system data from post/system libs in database

8381372

h00die deleted the hosts_data branch November 15, 2018 22:04

bcoles mentioned this pull request Dec 1, 2018

return hostname #11046

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

database host info from post modules #10823

database host info from post modules #10823

h00die commented Oct 17, 2018 •

edited

Loading

h00die commented Oct 17, 2018

h00die commented Oct 17, 2018

sempervictus left a comment

h00die commented Oct 24, 2018

bcoles commented Nov 2, 2018

bcoles commented Nov 2, 2018

h00die commented Nov 2, 2018 •

edited

Loading

bcoles commented Nov 3, 2018

h00die commented Nov 3, 2018 •

edited

Loading

bcoles commented Nov 3, 2018

h00die commented Nov 3, 2018

bcoles commented Nov 4, 2018

bcoles commented Nov 4, 2018 •

edited

Loading

h00die commented Nov 5, 2018 •

edited

Loading

bcoles commented Nov 5, 2018 •

edited

Loading

h00die commented Nov 6, 2018

bcoles commented Nov 6, 2018

bcoles commented Nov 6, 2018 •

edited by gdavidson-r7

Loading

database host info from post modules #10823

database host info from post modules #10823

Conversation

h00die commented Oct 17, 2018 • edited Loading

Verification

h00die commented Oct 17, 2018

h00die commented Oct 17, 2018

Pre patch:

Post patch (spacing off due to ip rewrite):

sempervictus left a comment

Choose a reason for hiding this comment

h00die commented Oct 24, 2018

bcoles commented Nov 2, 2018

Linux

Solaris

Misc

bcoles commented Nov 2, 2018

h00die commented Nov 2, 2018 • edited Loading

bcoles commented Nov 3, 2018

h00die commented Nov 3, 2018 • edited Loading

bcoles commented Nov 3, 2018

h00die commented Nov 3, 2018

bcoles commented Nov 4, 2018

x86 / x64 / Intel Architecture

SPARC

bcoles commented Nov 4, 2018 • edited Loading

h00die commented Nov 5, 2018 • edited Loading

bcoles commented Nov 5, 2018 • edited Loading

h00die commented Nov 6, 2018

bcoles commented Nov 6, 2018

bcoles commented Nov 6, 2018 • edited by gdavidson-r7 Loading

Release Notes

h00die commented Oct 17, 2018 •

edited

Loading

h00die commented Nov 2, 2018 •

edited

Loading

h00die commented Nov 3, 2018 •

edited

Loading

bcoles commented Nov 4, 2018 •

edited

Loading

h00die commented Nov 5, 2018 •

edited

Loading

bcoles commented Nov 5, 2018 •

edited

Loading

bcoles commented Nov 6, 2018 •

edited by gdavidson-r7

Loading