Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Password Cracker Overhaul (ie hashcat) #11695

Merged
merged 7 commits into from Nov 8, 2019
Merged

Conversation

@h00die
Copy link
Contributor

h00die commented Apr 7, 2019

This PR is a complete transformation of the cracking system. Main Features:

  1. adds hashcat as a cracker type
  2. combines the database crackers into one file
  3. expands the granularity of what hashes to crack (ie crack_linux you can unset md5 if you didn't want it)
  4. adds debugging by printing the version of the cracker used
  5. prints the cracked passwords in a table including how we cracked it
  6. adds a show_command option to output the exact command being run for debugging by a user
  7. added crack_osx
  8. added jtr 'normal' mode (no mode specified) which does single, wordlist, incremental. While we call these individually, at times this cracks things the other modes don't for some reason.
  9. changes the behavior of creds to truncate anything over sha512 length (looking at you oracle) unless -v is added.
  10. thanks to @jmartin-r7 for a block of code that now we don't crack already cracked password hashes.
  • Convert jtr_aix to crack_aix
  • Hashcat addition to crack_aix
  • Deprecate jtr_aix
  • Write documentation for crack_aix
  • Convert jtr_linux to crack_linux
  • Hashcat addition to crack_linux
  • Deprecate jtr_linux
  • Write documentation for crack_linux
  • Convert jtr_windows to crack_windows
  • Hashcat addition to crack_windows
  • Deprecate jtr_windows
  • Write documentation for crack_windows
  • Convert jtr_mssql|mysql|oracle|postgres to crack_databases
  • Hashcat addition to crack_databases
  • Deprecate jtr_mssql|mysql|oracle|postgres
  • Write documentation for crack_databases
  • Write crack_osx
  • Write documentation for crack_osx

@bcook-r7 is this more what you had in mind? Note, i haven't tested hashcat in here yet, just made the cracker an action, and made everything decision branches to cut down on the amount of duplicate code. Also went ahead and redesigned the code structure so that every other module will replicate this just with (more or less) changing out the hashes_regex appropriately.

new output:

msf5 auxiliary(analyze/crack_aix) > run

[*] Hashes Written out to /tmp/hashes_tmp20190406-9867-h22npd
[*] Wordlist file written out to /tmp/jtrtmp20190406-9867-gghzok
[*] Cracking descrypt hashes in normal wordlist mode...
Using default input encoding: UTF-8
Will run 8 OpenMP threads
Press 'q' or Ctrl-C to abort, almost any other key for status
1g 0:00:00:00 DONE (2019-04-06 21:50) 50.00g/s 1638Kp/s 1638Kc/s 1638KC/s 1qwerty..grandaun
Use the "--show" option to display all of the cracked passwords reliably
Session completed
[*] Cracking descrypt hashes in single mode...
Using default input encoding: UTF-8
[*] Cracking descrypt hashes in incremental mode (Digits)...
Using default input encoding: UTF-8
[+] Cracked Hashes
==============

 DB ID  Hash Type  Username      Cracked Password  Method
 -----  ---------  --------      ----------------  ------
 16     descrypt   des_password  password          Wordlist

old output:

msf5 auxiliary(analyze/jtr_aix) > run

[*] Hashes Written out to /tmp/hashes_tmp20190406-10581-vbb9a5
[*] Wordlist file written out to /tmp/jtrtmp20190406-10581-iv7r22
[*] Cracking descrypt hashes in normal wordlist mode...
Using default input encoding: UTF-8
[*] Cracking descrypt hashes in single mode...
Using default input encoding: UTF-8
[*] Cracking descrypt hashes in incremental mode (Digits)...
Using default input encoding: UTF-8
[*] Cracked Passwords this run:
[+] des_password:password
[*] Auxiliary module execution completed
@h00die h00die added the delayed label Apr 7, 2019
@h00die

This comment has been minimized.

Copy link
Contributor Author

h00die commented Apr 12, 2019

Ping @bcook-r7 is this way better or worse than the other hashcat implementation

@busterb

This comment has been minimized.

Copy link
Member

busterb commented Apr 16, 2019

I like this better! Now a question is how to we fix the muscle memory of everyone looking for jtr_aix?

@h00die

This comment has been minimized.

Copy link
Contributor Author

h00die commented Apr 17, 2019

Since this requires a substantial library re-write, which more or less will break compatibility, i think I'll leave a module stub with deprecated and failwith. That should give a lot of info to the user that they need to update their behavior

@h00die

This comment has been minimized.

Copy link
Contributor Author

h00die commented May 11, 2019

back at this. aix, and linux more or less work.
Decided to combine mssql, mysql, oracle, postgres into crack_databases. i set options to manually select which types of database hashes you want to do.
I think this will be better so that we can have crack_webapps in the future, instead of a wordpress, drupal, etc.
Also, much of the logic is the same when doing these, so it should save a lot of lines of code.

@h00die

This comment has been minimized.

Copy link
Contributor Author

h00die commented May 13, 2019

Note to self:
1 need to check hash casing, sometimes it matters, some times it doesn't
Need to check if multiple users with same hash is handled correctly

@h00die

This comment has been minimized.

Copy link
Contributor Author

h00die commented May 21, 2019

I believe I more or less have this working. It ran through all my tests successfully.

Since I wrote all the crack modules in a similar fashion (for consistency), can someone look at crack_linux for optimizations? @bcoles you are always awesome at helping optimize my python styled ruby, if you have some time would you mind taking a look? Any changes i'll replicate to the other modules as well.

@h00die

This comment has been minimized.

Copy link
Contributor Author

h00die commented May 25, 2019

Anyone at R7, if you get a little bit of free time do you mind giving crack_linux a look over? Any changes I can push to the other modules, but since this such a major re-write i'd like to get it really looked at well.

@h00die

This comment has been minimized.

Copy link
Contributor Author

h00die commented May 29, 2019

@timwr I just added crack_osx, which also makes a slight adjustment to the osx hashdumper (sha1 vs xsha).
I have some confusion on 10.7, is it its own hash, is it the xsha or the pbkdf2? hashcat seems to say its unique, but i can't confirm.

@timwr if you have some more sample hashes for various OSX versions, please pass them my way!

@h00die h00die removed the delayed label May 29, 2019
@h00die h00die changed the title WIP: hashcat Password Cracker Overhaul (ie hashcat) May 29, 2019
debuggin hashcat aix

remove word normal

get hashcat working on aix

add deprecated jtr_aix

prettying up crack_aix

custom wordlists should include the words themselves

make format transparent to user

aix cleanup, linux working

linux working, database in progress

crack databases working

crack windows working

spaces at eol

spec updates

spec updates

spec working

add version detection

crack_aix fixes and docs

refactoring crack modules

fix syntax error

docs for crackers

markup touchups

osx cracker

jenkins

fix jenkins

remove crypt fix osx for 10.7

doc fixes and osx sha512
@h00die h00die force-pushed the h00die:jtr_hcat_combine branch from e3ad634 to 2cccd50 May 31, 2019
@h00die

This comment has been minimized.

Copy link
Contributor Author

h00die commented May 31, 2019

TODO:

  1. datastore options to set diff modes to run (aka bypass incremental or wordlist)
  2. detect if john is not jumbopatch
  3. offload the mode settting to the libs
@h00die h00die added the a2k19 label May 31, 2019
@h00die

This comment has been minimized.

Copy link
Contributor Author

h00die commented Jun 1, 2019

This PR is ready for review, good luck to the poor intern who gets it!

@h00die

This comment has been minimized.

Copy link
Contributor Author

h00die commented Jun 4, 2019

Updated main list of items, but here it is:

  • changes the behavior of creds to truncate anything over sha512 length (looking at you oracle) unless -v is added.
  • thanks to @jmartin-r7 for a block of code that now we don't crack already cracked password hashes.
Copy link
Contributor

bcoles left a comment

@bcoles you are always awesome at helping optimize my python styled ruby, if you have some time would you mind taking a look?

modules/auxiliary/analyze/crack_aix.rb Outdated Show resolved Hide resolved
modules/auxiliary/analyze/crack_aix.rb Outdated Show resolved Hide resolved
modules/auxiliary/analyze/crack_aix.rb Outdated Show resolved Hide resolved
lib/metasploit/framework/password_crackers/cracker.rb Outdated Show resolved Hide resolved
@h00die

This comment has been minimized.

Copy link
Contributor Author

h00die commented Jul 15, 2019

Thanks for all the review/comments @bcoles . I've addressed them all!

@h00die

This comment has been minimized.

Copy link
Contributor Author

h00die commented Aug 7, 2019

Any more input on this? I know it's a large PR with a complete overhaul of the password cracking system, so it's pretty daunting. Just want to get it in the framework and start seeing real world use and see other's input on it!

@h00die

This comment has been minimized.

Copy link
Contributor Author

h00die commented Sep 8, 2019

@jmartin-r7 jmartin-r7 self-assigned this Sep 8, 2019
@jmartin-r7 jmartin-r7 mentioned this pull request Oct 7, 2019
3 of 3 tasks complete
@busterb

This comment has been minimized.

Copy link
Member

busterb commented Oct 8, 2019

Probably the last thing to add would be any module aliases that might be needed to make sure existing scripts don't break immediately if we can avoid it. OTOH I just noticed reading Metasploit Unleashed that it refers to jtr_crack_fast that doesn't exist already.

@h00die

This comment has been minimized.

Copy link
Contributor Author

h00die commented Oct 8, 2019

@h00die

This comment has been minimized.

Copy link
Contributor Author

h00die commented Oct 9, 2019

Looks like this is up for a rebase, which isn't a surprise after 6 months.
W/o @wvu-r7 there by my side, I'm honestly not confident enough to do it on this branch w/o messing it up, and there are tooooooo many files to redo it for a 5th time.

@jmartin-r7

This comment has been minimized.

Copy link
Contributor

jmartin-r7 commented Oct 9, 2019

The rebase is actually due to the PR I put up that mentions this, I will resolve the conflicts as I land.

@h00die

This comment has been minimized.

Copy link
Contributor Author

h00die commented Oct 25, 2019

Need any help on this @jmartin-r7 ?

@h00die h00die mentioned this pull request Oct 27, 2019
@h00die h00die added the attic label Nov 7, 2019
@h00die

This comment has been minimized.

Copy link
Contributor Author

h00die commented Nov 7, 2019

Added attic since this has been waiting for a 2nd review or land for 4 months :(

@wvu-r7

This comment has been minimized.

Copy link
Contributor

wvu-r7 commented Nov 7, 2019

Are you still landing this, @jmartin-r7?

@jmartin-r7

This comment has been minimized.

Copy link
Contributor

jmartin-r7 commented Nov 7, 2019

Yes, this is still in my todo list.

Copy link
Contributor

jmartin-r7 left a comment

Just a couple details to share, I be landing this shortly as is.

jmartin-r7 added a commit that referenced this pull request Nov 8, 2019
@jmartin-r7 jmartin-r7 merged commit 3ca4fa1 into rapid7:master Nov 8, 2019
1 of 2 checks passed
1 of 2 checks passed
Metasploit Automation - Test Execution Failed to pass tests.
Details
Metasploit Automation - Sanity Test Execution Successfully completed all tests.
Details
@jmartin-r7

This comment has been minimized.

Copy link
Contributor

jmartin-r7 commented Nov 8, 2019

Release Notes

A complete transformation of the cracking system, adding support for additional applications and hash types to be utilized during reversing of stored credential details. JtR has been migrated and Hashcat has been added using this pattern.

@jmartin-r7 jmartin-r7 removed the attic label Nov 8, 2019
@h00die h00die deleted the h00die:jtr_hcat_combine branch Nov 9, 2019
@jmartin-r7 jmartin-r7 mentioned this pull request Dec 4, 2019
1 of 2 tasks complete
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
5 participants
You can’t perform that action at this time.