Packrat: a post module to gather artifacts from a multitude of applications #11719

ghost · 2019-04-12T15:28:12Z

… it can gather from applications on end user systems.

Artifacts include: chat logins and logs, browser logins and history and
cookies, email logins and emails sent and received and deleted, contacts
and many others. These artifacts are collected from applications
including: 12 browsers, 13 chat/IM/IRC applications, 6 email clients and
1 game.

These artifacts are then scraped for credentials,

The use case for this post-exploitation module is to specify the types
of artefacts you are interesed in, to gather the relevent files
depending on your aims.

Tell us what this change does. If you're fixing a bug, please mention
the github issue number.

Please ensure you are submitting from a unique branch in your repository to master in Rapid7's.

Verification

List the steps needed to make sure this thing works

Start msfconsole
use exploit/windows/smb/ms08_067_netapi
...
Verify the thing does what it should
Verify the thing does not do what it should not
Document the thing and how it works (Example)

… it can gather from applications on end user systems. Artifacts include: chat logins and logs, browser logins and history and cookies, email logins and emails sent and received and deleted, contacts and many others. These artifacts are collected from applications including: 12 browsers, 13 chat/IM/IRC applications, 6 email clients and 1 game. These artifacts are then scraped for credentials, The use case for this post-exploitation module is to specify the types of artefacts you are interesed in, to gather the relevent files depending on your aims.

modules/post/windows/gather/credentials/packrat_credentials.rb

wvu

Please review and merge https://github.com/danhallsworth1/metasploit-framework/pull/1.

You will also want to run RuboCop against this module with our .rubocop.yml and clean up the most reasonable reported issues. You should write a module doc as well.

You can find our contributing guide here. We more or less follow the Ruby style guide.

Thanks!

This merge makes the changes on pull request 11719. Module has been tested and still works and also passes Robocop

ghost · 2019-04-19T13:45:31Z

Hi I forgot to add the data/packrat/artifacts.json directory in this pull request, I think that's why the checks have failed. I just put the 'artifact.json' file in the 'metasploit/data' directory. Do you need me to submit it again in the correct directory @wvu-r7

wvu · 2019-04-19T15:40:48Z

@danhallsworth1: You shouldn't need to do that. The merge took care of it. Check out the diff: https://github.com/rapid7/metasploit-framework/pull/11719/files. :)

The RSpec failures look unrelated. Do we know anything about this, @jmartin-r7?

ghost · 2019-04-19T15:46:05Z

ah amazing so is there anything I else I need to do now then? @wvu-r7

wvu · 2019-04-19T15:52:04Z

Please review and merge danhallsworth1#1.

You will also want to run RuboCop against this module with our .rubocop.yml and clean up the most reasonable reported issues. You should write a module doc as well.

You can find our contributing guide here. We more or less follow the Ruby style guide.

Thanks!

I would use a consistent style in the code. Now that the large hash is well-formed JSON and out of the way, we can focus on improving the functional code within the module. I suggest leveraging RuboCop.

Note that better code, especially when documented (as requested), leads to more maintainable code. And we will have to maintain it. :-)

Let me know if you need any assistance. Thanks!

wvu · 2019-04-19T15:54:36Z

modules/post/windows/gather/credentials/packrat_credentials.rb

+
+    #used to grab files for each user on the remote host.
+    grab_user_profiles.each do |userprofile|
+      APPLICATION_ARRAY.each {|app_loop|


As an example of what I mean by consistent, you are using do for the outside block but { for the inside block, and both blocks are multiline. Please use do as per https://github.com/rubocop-hq/ruby-style-guide#single-line-blocks.

I don't care so much about what the "right" style is so long as your code is consistent and clean.

Good coding is language-agnostic.

ghost · 2019-04-19T15:54:45Z

Okay thank you I’ll get that sorted then Get Outlook for iOS<https://aka.ms/o0ukef>

…

________________________________ From: n̸̺̳̮̖͚̺̺͍̪̰͎̰͇̯̙̋ͣͣ̆̇ͥ̕n̷̑͐̑ͩ̾͗ͣ͐̏҉̣̞̖͖͕̗̞̳̯͚̖n̶̵̫̣͓͈͓̥̣̬̠̗̤͔͖̘̫̪̱͇̤̏ͯ̿ͫͥ͐̍͂͛͊̓͜͢n̨̡̙̼̗̳̿̂ͤ̑͆ͧ́͂̈n̷̸̛͙̘̙̪͔͕͖̻̹͇̮̰̖̣͓̖̫ͪͥ̓̑͒͟͠ņͤͤͯ̌̒̽́̎ͦ̃̈͆̿͒̊ͣ͏̡͎̪̗̻̱̭̲̤͈̞̦͇͍̤̪̥̳̙͘ͅn̨͇̦̭̞̹̗̞̖̪̤̗ͩ́̋͛͂̂̿ͭͭ̈͢ʌ̸̵͉̱͍̤͍̫̰̘͚̲̝̩̤̙͗̑ͥ̃̈́͌̔̿̉͒ͦ̿̈̋́̚̚̕͢ͅʍ̸̭̰̬̖͚͓̠̮͙͓̝̥̞̥͉̏̈ͩͥͣͣ̆ <notifications@github.com> Sent: Friday, April 19, 2019 4:53:28 PM To: rapid7/metasploit-framework Cc: danhallsworth1; Mention Subject: Re: [rapid7/metasploit-framework] Add post module to gather artifacts from a multitude of applications (#11719) Please review and merge danhallsworth1#1<https://github.com/danhallsworth1/metasploit-framework/pull/1>. You will also want to run RuboCop<https://github.com/rubocop-hq/rubocop> against this module with our .rubocop.yml<https://github.com/rapid7/metasploit-framework/blob/master/.rubocop.yml> and clean up the most reasonable reported issues. You should write a module doc<https://github.com/rapid7/metasploit-framework/wiki/Writing-Module-Documentation> as well. You can find our contributing guide here<https://github.com/rapid7/metasploit-framework/blob/master/CONTRIBUTING.md>. We more or less follow the Ruby style guide<https://github.com/rubocop-hq/ruby-style-guide>. Thanks! I would use a consistent style in the code. Now that the large hash is well-formed JSON and out of the way, we can focus on improving the functional code within the module. I suggest leveraging RuboCop. Note that better code, especially when documented (as requested), leads to more maintainable code. And we will have to maintain it. :-) Let me know if you need any assistance. Thanks! — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<#11719 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AI4APQUHDVZAJRHBFZR7EWTPRHTHRANCNFSM4HFRQQZQ>.

wvu · 2019-04-19T15:55:13Z

modules/post/windows/gather/credentials/packrat_credentials.rb

+    grab_user_profiles.each do |userprofile|
+      APPLICATION_ARRAY.each {|app_loop|
+        download(userprofile, app_loop)
+


As an example of what I mean by clean, this blank line should be removed.

Hi i've made some of these changes and ran Robocop against it fixing the main issues its highlighted. I've got 2 uni exams coming up soon so wont be able to make the changes to this as soon as id like, but i'll still be working on it hope that's okay @wvu-r7

Totally fine with that. Thanks for taking the time to contribute. Thanks to your collaborators, too! And good luck on the exams. :)

ghost · 2019-05-22T11:17:56Z

Hi I’ve made these changes now and have made the documentation, ready to submit. What’s are the commands I need to do to submit these changes?

timwr · 2019-05-22T11:21:24Z

git add 
git commit
git push

?

jrobles-r7 · 2019-05-22T11:53:49Z

To update this PR push to your branch danhallsworth1:packrat_post_module.

wvu · 2019-05-22T15:35:41Z

No, you don't want to submit a new PR. I'll close that so you can add the commits here.

wvu · 2019-05-22T15:38:36Z

I do something like this:

git add -p
git commit
git push origin packrat_post_module

throughout the code e.g. loops, and made the changes suggested through robocop

ghost · 2019-05-22T18:55:18Z

I do something like this:

git add -p
git commit
git push origin packrat_post_module

Hi thanks those commands helped a lot, got everything submitted now thanks for the help

wvu

You have two modules in this PR. Please consolidate your changes into one.

wvu · 2019-05-22T19:03:30Z

documentation/modules/post/windows/gather/packrat_credentials.md

+
+Users can enter APPLICATIONS to extract from example output shown below for email service incredimail
+
+msf post(windows/gather/credentials/packrat_credentials) > set SESSION 1


You'll want to put code in code blocks.

cbrnrd

Pretty solid overall. What I commented is mostly small things (verbose context, code style, grammar, etc.) Thanks for taking the time to write this!

cbrnrd · 2019-06-20T21:09:28Z

modules/post/windows/gather/packrat_credentials.rb

+      'Description'  => %q{
+        This module extracts artifcats from a large list of applications
+        and can extract credentials storing content in loot. Full list in
+        module documentation.


~~Full list in module documentation.~~ -> The full list of supported applications is in the module documentation.

cbrnrd · 2019-06-20T21:12:11Z

modules/post/windows/gather/credentials/packrat_credentials.rb

+
+    register_options(
+      [
+        OptRegexp.new('REGEX', [false, 'Match a regular expression', '^password']),


Clarify what's being matched by the regex

cbrnrd · 2019-06-20T21:12:31Z

modules/post/windows/gather/credentials/packrat_credentials.rb

+        OptRegexp.new('REGEX', [false, 'Match a regular expression', '^password']),
+        OptBool.new('STORE_LOOT', [false, 'Store artifacts into loot database (otherwise, only download)', 'true']),
+        # enumerates the options based on the artifacts that are defined below
+        OptEnum.new('APPCATEGORY', [false, 'Category of applications to gather from', 'All', APPLICATION_ARRAY.map {|x| x[:category]}.uniq.unshift('All')]),


Suggested change

OptEnum.new('APPCATEGORY', [false, 'Category of applications to gather from', 'All', APPLICATION_ARRAY.map {|x| x[:category]}.uniq.unshift('All')]),

OptEnum.new('APP_CATEGORY', [false, 'Category of applications to gather from', 'All', APPLICATION_ARRAY.map {|x| x[:category]}.uniq.unshift('All')]),

and refactor references

cbrnrd · 2019-06-20T21:14:28Z

modules/post/windows/gather/credentials/packrat_credentials.rb

+
+  # Check to see if the artifact exists on the remote system.
+  def location(profile, opts = {})
+


delete empty line for consistency

cbrnrd · 2019-06-20T21:15:49Z

modules/post/windows/gather/credentials/packrat_credentials.rb

+      artifact_child[:xml_search].each do |xml_split|
+        xml_split[:xml].each do |xml_string|
+          xml_file.xpath("#{xml_string}").each do |xml_match|
+            vprint_status("#{xml_split[:extraction_description]}")


It may be a little hard to debug this module if you're just printing the variable data with no context. For this (and most other) vprint_*() calls, try to add some context to what you're printing

cbrnrd · 2019-06-20T21:23:29Z

modules/post/windows/gather/credentials/packrat_credentials.rb

+      end
+
+      sql_credential_path = store_loot("#{artifact}#{cred}", "", session, "#{database_string}", local_loc) #saves neatened up database file
+      print_status "File with credentials saved:  #{sql_credential_path}"


Suggested change

print_status "File with credentials saved: #{sql_credential_path}"

print_status "File with credentials saved: #{sql_credential_path}"

cbrnrd · 2019-06-20T21:24:21Z

modules/post/windows/gather/credentials/packrat_credentials.rb

+
+      child_json_query.each do |split|
+        children = eval("json_parse#{parent_json_query}")
+        children.each {|child_node|


Use the same block styling for inline blocks:

children.each do |child_node| ....... end

cbrnrd · 2019-06-20T21:25:30Z

modules/post/windows/gather/credentials/packrat_credentials.rb

+      # filter based on options
+      if (category != datastore['APPCATEGORY'] && datastore['APPCATEGORY'] != 'All') || (application != datastore['APPLICATION'] && datastore['APPLICATION'] != 'All') || (file_type != datastore['ARTIFACTS'] && datastore['ARTIFACTS'] != 'All')
+        # doesn't match search criteria, skip this artifact
+        next


Suggested change

next

next if (category != datastore['APPCATEGORY'] && datastore['APPCATEGORY'] != 'All') || (application != datastore['APPLICATION'] && datastore['APPLICATION'] != 'All') || (file_type != datastore['ARTIFACTS'] && datastore['ARTIFACTS'] != 'All')

and delete end statement on the next line

cbrnrd · 2019-06-20T21:27:47Z

modules/post/windows/gather/credentials/packrat_credentials.rb

+
+        if credential_type == 'sqlite'
+          extract_sqlite(saving_path, artifact_child, artifact, local_loc)
+        end


These can all be compressed into one line each:

extract_xml(saving_path, artifact_child, artifact, local_loc) if credential_type == 'xml' extract_json(saving_path, artifact_child, artifact, local_loc) if credential_type == 'json' extract_regex(saving_path, artifact_child, artifact, local_loc) if credential_type == 'text' extract_sqlite(saving_path, artifact_child, artifact, local_loc) if credential_type == 'sqlite'

Or you could turn it into a case/switch, up to you.

cbrnrd · 2019-06-20T21:28:47Z

documentation/modules/post/windows/gather/packrat_credentials.md

+1 Game:
+Xfire
+
+13 Browser:


Suggested change

13 Browser:

13 Browsers:

wvu · 2019-06-20T23:24:07Z

@cbrnrd: I'm not sure @danhallsworth1 is coming back, so any review you leave will be more work for me. :-)

wvu · 2019-06-20T23:28:18Z

@cbrnrd: How about I close this PR and open up a new one you can PR to? Then I can address the changes directly.

h00die · 2019-06-20T23:29:15Z

My only real issue with this module is that it's a huge pile of modules. In theory it would be broken down into parts, each functional item, then packrat could be a post script that runs them all.
This makes it easier to maintain, test, and doesn't lose functionality

h00die · 2019-06-20T23:29:55Z

Not to mention, and I didn't check, there's possible overlap in code and functionality between this and other modules that already exist

wvu · 2019-06-20T23:34:28Z

~~I have already written code for this PR, and I am unable to contribute further at the moment. Someone else will have to.~~ Eh, opening a new PR. Let's find a way to carry this across the finish line.

wvu · 2019-06-21T00:22:03Z

See me over in #11998. Thanks.

wvu changed the title ~~This post exploitation module has an extensive list of artefacts that…~~ Add post module to gather artifacts from a multitude of applications Apr 12, 2019

acammack-r7 mentioned this pull request Apr 17, 2019

packrat_credentials Post-Exploitation Module #11700

Closed

wvu reviewed Apr 17, 2019

View reviewed changes

modules/post/windows/gather/credentials/packrat_credentials.rb Outdated Show resolved Hide resolved

wvu added feature module needs-docs labels Apr 18, 2019

wvu added 5 commits April 18, 2019 18:49

Add splat

6b1a8e2

Remove requires

f960d89

Fix author

60dc61b

Fix typo

d8d168f

Use an external JSON file for artifacts

1da5bc2

wvu suggested changes Apr 18, 2019

View reviewed changes

Merge branch 'wvu-r7-pr/11719' into packrat_post_module

976c388

This merge makes the changes on pull request 11719. Module has been tested and still works and also passes Robocop

wvu reviewed Apr 19, 2019

View reviewed changes

wvu changed the title ~~Add post module to gather artifacts from a multitude of applications~~ Packrat: a post module to gather artifacts from a multitude of applications Apr 19, 2019

wvu self-assigned this Apr 24, 2019

ghost mentioned this pull request May 22, 2019

Packrat post exploitation 'gather' module #11867

Closed

These changes add documentation, changed the coding style to be the same

4d0ba88

throughout the code e.g. loops, and made the changes suggested through robocop

danhallsworth1 added 2 commits May 22, 2019 14:49

Module documentation

87d5e96

Changes made with RoboCop and styling changes

448ab71

wvu suggested changes May 22, 2019

View reviewed changes

cbrnrd reviewed Jun 20, 2019

View reviewed changes

wvu removed the needs-docs label Jun 21, 2019

wvu mentioned this pull request Jun 21, 2019

Packrat: a post module to gather artifacts from a multitude of applications #11998

Closed

wvu closed this Jun 21, 2019

h00die mentioned this pull request Apr 18, 2020

Divide Up Packrat #11998 #13284

Closed

33 tasks

KazuCyber mentioned this pull request May 14, 2021

PackRat post exploitation modules based on a common mixin #15200

Merged

4 tasks

This pull request was closed.


		Users can enter APPLICATIONS to extract from example output shown below for email service incredimail

		msf post(windows/gather/credentials/packrat_credentials) > set SESSION 1

	OptEnum.new('APPCATEGORY', [false, 'Category of applications to gather from', 'All', APPLICATION_ARRAY.map {\|x\| x[:category]}.uniq.unshift('All')]),
	OptEnum.new('APP_CATEGORY', [false, 'Category of applications to gather from', 'All', APPLICATION_ARRAY.map {\|x\| x[:category]}.uniq.unshift('All')]),


		# Check to see if the artifact exists on the remote system.
		def location(profile, opts = {})

	print_status "File with credentials saved: #{sql_credential_path}"
	print_status "File with credentials saved: #{sql_credential_path}"

	next
	next if (category != datastore['APPCATEGORY'] && datastore['APPCATEGORY'] != 'All') \|\| (application != datastore['APPLICATION'] && datastore['APPLICATION'] != 'All') \|\| (file_type != datastore['ARTIFACTS'] && datastore['ARTIFACTS'] != 'All')

Packrat: a post module to gather artifacts from a multitude of applications #11719

Packrat: a post module to gather artifacts from a multitude of applications #11719

Conversation

ghost commented Apr 12, 2019

Verification

wvu left a comment • edited Loading

Choose a reason for hiding this comment

ghost commented Apr 19, 2019

wvu commented Apr 19, 2019 • edited Loading

ghost commented Apr 19, 2019

wvu commented Apr 19, 2019

wvu Apr 19, 2019 • edited Loading

Choose a reason for hiding this comment

ghost commented Apr 19, 2019 via email

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ghost commented May 22, 2019

timwr commented May 22, 2019

jrobles-r7 commented May 22, 2019

wvu commented May 22, 2019

wvu commented May 22, 2019

ghost commented May 22, 2019

wvu left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cbrnrd left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cbrnrd Jun 20, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

wvu commented Jun 20, 2019

wvu commented Jun 20, 2019

h00die commented Jun 20, 2019

h00die commented Jun 20, 2019

wvu commented Jun 20, 2019 • edited Loading

wvu commented Jun 21, 2019

wvu left a comment •

edited

Loading

wvu commented Apr 19, 2019 •

edited

Loading

wvu Apr 19, 2019 •

edited

Loading

cbrnrd left a comment •

edited

Loading

cbrnrd Jun 20, 2019 •

edited

Loading

wvu commented Jun 20, 2019 •

edited

Loading