Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

jtr modernizations (again again again) #11351

Merged
merged 12 commits into from Feb 13, 2019
Merged

jtr modernizations (again again again) #11351

merged 12 commits into from Feb 13, 2019

Conversation

@h00die
Copy link
Contributor

h00die commented Feb 3, 2019

Fixes #11260

This PR modernizes the jtr_* modules. It requires #11263 #11261 .

Global Updates

  1. Add docs to all jtr_ modules
  2. When working with jtr_postgres_fast, postgres format is covered by dynamic_1034, but that didn't exist. Why? Well, john.conf is >6yrs old (has 2011 in the file, magnumripper/JohnTheRipper@f4467dd is when they change it from 2012 to 2013). I'm not sure if there were custom changes to john.conf, but i've overwritten it and added all the included files.
  3. If no hashes are written to the output file, bail
  4. generate the hashes first so we can bail early from above which also prevents writing hash file and wordlist for no reason.
  5. the hashes file and wordlist file were left on disk after execution. I can only imagine on a pro server that gets rebooted every once in a blue moon how many files were left (wordlist by default is almost 2MB and generated each execution)
  6. added max_length to the cracker object to deal with the Digits incremental issue described in jtr_windows_fast
  7. The creds command now shows the jtr_format #11260
  8. The creds add command now takes jtr: to set the jtr format #11260
  9. The creds add command now takes postgres: to properly set a postgres hash.

Module Specific Changes

  1. jtr_aix: des changed to descrypt about 5yrs ago in JtR so this module hasn't worked in a long while. The linux cracker knew this already:
    formats = [ 'md5crypt', 'descrypt', 'bsdicrypt']
  2. jtr_crack_fast: lanman is 7len, so generate a 2nd wordlist optimized for that length
  3. jtr_crack_fast: final output showed the user id which isn't needed so we optimize from lm_password:password:4 to lm_password:password
  4. jtr_crack_fast: john's --show command includes the user_id, which wasn't being stripped off properly, thus making this module not work correctly. the password password was being 'correctly' saved as password:4. I'm sure john's --show syntax changed at some point.
  5. jtr_linux: cracked passwords showed a passwd type format on output, now they just show un:pass to standardize w/ everything else
  6. jtr_linux: said it cracked blowfish, but didn't (most likely a change on jtr's end), added back in
  7. jtr_crack_fast is now jtr_windows_fast because its more descriptive and more standardized with every other jtr naming convention.
  8. jtr_m[sy]sql_fast: used --incremental=Digits without removing rules and wordlist which caused Invalid options combination or duplicate option: "--incremental=Digits". Not sure if this was a warning or error, or how john handles that, but its fixed now (the other modules had it running correctly)
  9. jtr_postgres_fast: previously used raw-md5 as the format, now using the modernized dynamic_1034
  10. jtr_postgres_fast: previously only pulled postgresmd5 des passwords, which was none since that isn't how the postgres hashdumper module saves them. Therefore, this module didn't work at all, but it now properly grabs postgres or raw-md5.
  11. jtr_oracle_fast: added oracle12c support (even though oracle_hashdump will need to be updated to handle this, which should be minor changes)
  12. jtr_oracle_fast: added support for oracle11/oracle12c H format (JtR dynamic_1506) which is MD5 based, so should be much faster than pbkdf2 :)
  13. jtr_oracle_fast: was doing a split on : and taking the 2nd field, this was meant to pick the oracle11 format hash, however it was running on oracle format as well, therefore nothing really worked at all.
  14. jtr_windows_fast: Digits was increased from 8 to 20 length 6yrs ago. This seems to be no big deal for most hashing algorithms, however NTLM was taking a LOOOOONG time on my i7, so we'll push it back down to 8. magnumripper/JohnTheRipper@f4467dd#diff-c499d11af6e80a995563b547db7ce022R341
  15. jtr_windows_fast: all4 was removed, and id think most everyone doesn't have a 4char password, but most should be covered in the default wordlist anyways.
  16. apply_pot: a new module that will take a pot file and run it through all the hashes in the db to see if we can instantly crack any that were previously cracked. Scenario this helps to solve: on your pro server you have a BIG pot file from a few years of cracking passwords. You get to a new gig, and pop an AD or NIS so you get a lot of hashes. OLD WAY you'd have to run the modules manually, it would go through lots of diff crack attempts only to finally just do the pot trick. This could take a while. NEW WAY you run this module, it instacracks a lot, and hopefully you're off to the races.

How To Test This PR

  • Load up a sample of test data:
creds add user:des_password hash:rEK1ecacw.7.c jtr:des
creds add user:md5_password hash:$1$O3JMY.Tw$AdLnLjQ/5jXF9.MTp3gHv/ jtr:md5
creds add user:bsdi_password hash:_J9..K0AyUubDrfOgO4s jtr:bsdi
creds add user:sha256_password hash:$5$MnfsQ4iN$ZMTppKN16y/tIsUYs/obHlhdP.Os80yXhTurpBMUbA5 jtr:sha256,crypt
creds add user:sha512_password hash:$6$zWwwXKNj$gLAOoZCjcr8p/.VgV/FkGC3NX7BsXys3KHYePfuIGMNjY83dVxugPYlxVg/evpcVEJLT/rSwZcDMlVVf/bhf.1 jtr:sha512,crypt
creds add user:blowfish_password hash:$2a$05$bvIG6Nmid91Mu9RcmmWZfO5HJIMCT8riNW0hEp8f6/FuA2/mHZFpe jtr:bf
creds add user:lm_password ntlm:E52CAC67419A9A224A3B108F3FA6CB6D:8846F7EAEE8FB117AD06BDD830B7586C jtr:lm
creds add user:nt_password ntlm:AAD3B435B51404EEAAD3B435B51404EE:8846F7EAEE8FB117AD06BDD830B7586C jtr:nt
creds add user:mssql05_toto hash:0x01004086CEB6BF932BC4151A1AF1F13CD17301D70816A8886908 jtr:mssql05
creds add user:mssql_foo hash:0x0100A607BA7C54A24D17B565C59F1743776A10250F581D482DA8B6D6261460D3F53B279CC6913CE747006A2E3254 jtr:mssql
creds add user:mssql12_Password1! hash:0x0200F733058A07892C5CACE899768F89965F6BD1DED7955FE89E1C9A10E27849B0B213B5CE92CC9347ECCB34C3EFADAF2FD99BFFECD8D9150DD6AACB5D409A9D2652A4E0AF16 jtr:mssql12
creds add user:mysql_probe hash:445ff82636a7ba59 jtr:mysql
creds add user:mysql-sha1_tere hash:*5AD8F88516BD021DD43F171E2C785C69F8E54ADB jtr:mysql-sha1
## oracle (10) uses usernames in the hashing, so we can't overide that here
creds add user:simon hash:4F8BC1809CB2AF77 jtr:des,oracle
creds add user:SYSTEM hash:9EEDFA0AD26C6D52 jtr:des,oracle
## oracle 11/12 H value, username is used
creds add user:DEMO hash:'S:8F2D65FB5547B71C8DA3760F10960428CD307B1C6271691FC55C1F56554A;H:DC9894A01797D91D92ECA1DA66242209;T:23D1F8CAC9001F69630ED2DD8DF67DD3BE5C470B5EA97B622F757FE102D8BF14BEDC94A3CC046D10858D885DB656DC0CBF899A79CD8C76B788744844CADE54EEEB4FDEC478FB7C7CBFBBAC57BA3EF22C' jtr:raw-sha1,oracle
## oracle 11/12 uses a LONG format, see lib/msf/core/auxiliary/jtr.rb
creds add user:oracle11_epsilon hash:'S:8F2D65FB5547B71C8DA3760F10960428CD307B1C6271691FC55C1F56554A;H:DC9894A01797D91D92ECA1DA66242209;T:23D1F8CAC9001F69630ED2DD8DF67DD3BE5C470B5EA97B622F757FE102D8BF14BEDC94A3CC046D10858D885DB656DC0CBF899A79CD8C76B788744844CADE54EEEB4FDEC478FB7C7CBFBBAC57BA3EF22C' jtr:raw-sha1,oracle
creds add user:oracle12c_epsilon hash:'H:DC9894A01797D91D92ECA1DA66242209;T:E3243B98974159CC24FD2C9A8B30BA62E0E83B6CA2FC7C55177C3A7F82602E3BDD17CEB9B9091CF9DAD672B8BE961A9EAC4D344BDBA878EDC5DCB5899F689EBD8DD1BE3F67BFF9813A464382381AB36B' jtr:pbkdf2,oracle12c
##postgres uses username, so we can't overide that here
creds add user:example postgres:md5be86a79bf2043622d58d5453c47d4860
  • Run through each module and see the hashes cracked:
use auxiliary/analyze/jtr_aix
run
use auxiliary/analyze/jtr_linux
set crypt true
run
use auxiliary/analyze/jtr_mssql_fast
run
use auxiliary/analyze/jtr_mysql_fast
run
use auxiliary/analyze/jtr_oracle_fast
run
use auxiliary/analyze/jtr_postgres_fast
run
use auxiliary/analyze/jtr_windows_fast
run
creds
  • Now drop your creds:
creds -d
  • Now re-add your creds from above and try apply_pot to see everything crack in ~1sec (i7 2600):
creds add user:des_password hash:rEK1ecacw.7.c jtr:des
creds add user:md5_password hash:$1$O3JMY.Tw$AdLnLjQ/5jXF9.MTp3gHv/ jtr:md5
creds add user:bsdi_password hash:_J9..K0AyUubDrfOgO4s jtr:bsdi
creds add user:sha256_password hash:$5$MnfsQ4iN$ZMTppKN16y/tIsUYs/obHlhdP.Os80yXhTurpBMUbA5 jtr:sha256,crypt
creds add user:sha512_password hash:$6$zWwwXKNj$gLAOoZCjcr8p/.VgV/FkGC3NX7BsXys3KHYePfuIGMNjY83dVxugPYlxVg/evpcVEJLT/rSwZcDMlVVf/bhf.1 jtr:sha512,crypt
creds add user:blowfish_password hash:$2a$05$bvIG6Nmid91Mu9RcmmWZfO5HJIMCT8riNW0hEp8f6/FuA2/mHZFpe jtr:bf
creds add user:lm_password ntlm:E52CAC67419A9A224A3B108F3FA6CB6D:8846F7EAEE8FB117AD06BDD830B7586C jtr:lm
creds add user:nt_password ntlm:AAD3B435B51404EEAAD3B435B51404EE:8846F7EAEE8FB117AD06BDD830B7586C jtr:nt
creds add user:mssql05_toto hash:0x01004086CEB6BF932BC4151A1AF1F13CD17301D70816A8886908 jtr:mssql05
creds add user:mssql_foo hash:0x0100A607BA7C54A24D17B565C59F1743776A10250F581D482DA8B6D6261460D3F53B279CC6913CE747006A2E3254 jtr:mssql
creds add user:mssql12_Password1! hash:0x0200F733058A07892C5CACE899768F89965F6BD1DED7955FE89E1C9A10E27849B0B213B5CE92CC9347ECCB34C3EFADAF2FD99BFFECD8D9150DD6AACB5D409A9D2652A4E0AF16 jtr:mssql12
creds add user:mysql_probe hash:445ff82636a7ba59 jtr:mysql
creds add user:mysql-sha1_tere hash:*5AD8F88516BD021DD43F171E2C785C69F8E54ADB jtr:mysql-sha1
## oracle (10) uses usernames in the hashing, so we can't overide that here
creds add user:simon hash:4F8BC1809CB2AF77 jtr:des,oracle
creds add user:SYSTEM hash:9EEDFA0AD26C6D52 jtr:des,oracle
## oracle 11/12 H value, username is used
creds add user:DEMO hash:'S:8F2D65FB5547B71C8DA3760F10960428CD307B1C6271691FC55C1F56554A;H:DC9894A01797D91D92ECA1DA66242209;T:23D1F8CAC9001F69630ED2DD8DF67DD3BE5C470B5EA97B622F757FE102D8BF14BEDC94A3CC046D10858D885DB656DC0CBF899A79CD8C76B788744844CADE54EEEB4FDEC478FB7C7CBFBBAC57BA3EF22C' jtr:raw-sha1,oracle
## oracle 11/12 uses a LONG format, see lib/msf/core/auxiliary/jtr.rb
creds add user:oracle11_epsilon hash:'S:8F2D65FB5547B71C8DA3760F10960428CD307B1C6271691FC55C1F56554A;H:DC9894A01797D91D92ECA1DA66242209;T:23D1F8CAC9001F69630ED2DD8DF67DD3BE5C470B5EA97B622F757FE102D8BF14BEDC94A3CC046D10858D885DB656DC0CBF899A79CD8C76B788744844CADE54EEEB4FDEC478FB7C7CBFBBAC57BA3EF22C' jtr:raw-sha1,oracle
creds add user:oracle12c_epsilon hash:'H:DC9894A01797D91D92ECA1DA66242209;T:E3243B98974159CC24FD2C9A8B30BA62E0E83B6CA2FC7C55177C3A7F82602E3BDD17CEB9B9091CF9DAD672B8BE961A9EAC4D344BDBA878EDC5DCB5899F689EBD8DD1BE3F67BFF9813A464382381AB36B' jtr:pbkdf2,oracle12c
##postgres uses username, so we can't overide that here
creds add user:example postgres:md5be86a79bf2043622d58d5453c47d4860

use auxiliary/analyze/apply_pot
run
creds
@h00die

This comment has been minimized.

Copy link
Contributor Author

h00die commented Feb 3, 2019

Just a heads up that r7 should review this to make sure changes aren't required to PRO.
The creds page has a table that may be 1 column short now.
The creds entry page does not let you set a jtr format (maybe add that?)

  • It may not be the worst idea to have a list of jtr hash types we use for a drop down, however I didn't code that in because that makes it static and less flexible in the future.

Any other 'automated' push to a jtr module may need to be adjusted if i changed the module name.

@wvu-r7

This comment has been minimized.

Copy link
Contributor

wvu-r7 commented Feb 6, 2019

You've been doing an incredible job with these PRs, @h00die. :)

@h00die

This comment has been minimized.

Copy link
Contributor Author

h00die commented Feb 12, 2019

Updated the docs to reflect adding jtr_format to the tables, as well as the ability to use creds add for all these hashes

@busterb busterb self-assigned this Feb 12, 2019
@busterb

This comment has been minimized.

Copy link
Member

busterb commented Feb 12, 2019

Hi @h00die it seems this adds a lot of files to data/jtr, but it is unclear what the license is. Could you please add an annotation to LICENSE so we know where these files came from and their license?

@busterb

This comment has been minimized.

Copy link
Member

busterb commented Feb 12, 2019

@h00die

This comment has been minimized.

Copy link
Contributor Author

h00die commented Feb 12, 2019

Good suggestion, for the copyright portion I took it from https://www.openwall.com/john/doc/LICENSE.shtml

@busterb

This comment has been minimized.

Copy link
Member

busterb commented Feb 13, 2019

Nice results, thanks!

msf5 auxiliary(analyze/apply_pot) > creds
Credentials
===========

host  origin  service  public              private                                                                                                                                                                                                                                                               realm  private_type        JtR Format
----  ------  -------  ------              -------                                                                                                                                                                                                                                                               -----  ------------        ----------
                       des_password        rEK1ecacw.7.c                                                                                                                                                                                                                                                                Nonreplayable hash  des
                       md5_password        $1$O3JMY.Tw$AdLnLjQ/5jXF9.MTp3gHv/                                                                                                                                                                                                                                           Nonreplayable hash  md5
                       bsdi_password       _J9..K0AyUubDrfOgO4s                                                                                                                                                                                                                                                         Nonreplayable hash  bsdi
                       sha256_password     $5$MnfsQ4iN$ZMTppKN16y/tIsUYs/obHlhdP.Os80yXhTurpBMUbA5                                                                                                                                                                                                                      Nonreplayable hash  sha256,crypt
                       sha512_password     $6$zWwwXKNj$gLAOoZCjcr8p/.VgV/FkGC3NX7BsXys3KHYePfuIGMNjY83dVxugPYlxVg/evpcVEJLT/rSwZcDMlVVf/bhf.1                                                                                                                                                                           Nonreplayable hash  sha512,crypt
                       blowfish_password   $2a$05$bvIG6Nmid91Mu9RcmmWZfO5HJIMCT8riNW0hEp8f6/FuA2/mHZFpe                                                                                                                                                                                                                 Nonreplayable hash  bf
                       lm_password         e52cac67419a9a224a3b108f3fa6cb6d:8846f7eaee8fb117ad06bdd830b7586c                                                                                                                                                                                                            NTLM hash           nt,lm
                       nt_password         aad3b435b51404eeaad3b435b51404ee:8846f7eaee8fb117ad06bdd830b7586c                                                                                                                                                                                                            NTLM hash           nt,lm
                       mssql05_toto        0x01004086CEB6BF932BC4151A1AF1F13CD17301D70816A8886908                                                                                                                                                                                                                       Nonreplayable hash  mssql05
                       mssql_foo           0x0100A607BA7C54A24D17B565C59F1743776A10250F581D482DA8B6D6261460D3F53B279CC6913CE747006A2E3254                                                                                                                                                                               Nonreplayable hash  mssql
                       mssql12_Password1!  0x0200F733058A07892C5CACE899768F89965F6BD1DED7955FE89E1C9A10E27849B0B213B5CE92CC9347ECCB34C3EFADAF2FD99BFFECD8D9150DD6AACB5D409A9D2652A4E0AF16                                                                                                                               Nonreplayable hash  mssql12
                       mysql_probe         445ff82636a7ba59                                                                                                                                                                                                                                                             Nonreplayable hash  mysql
                       mysql-sha1_tere     *5AD8F88516BD021DD43F171E2C785C69F8E54ADB                                                                                                                                                                                                                                    Nonreplayable hash  mysql-sha1
                       simon               4F8BC1809CB2AF77                                                                                                                                                                                                                                                             Nonreplayable hash  des,oracle
                       SYSTEM              9EEDFA0AD26C6D52                                                                                                                                                                                                                                                             Nonreplayable hash  des,oracle
                       DEMO                S:8F2D65FB5547B71C8DA3760F10960428CD307B1C6271691FC55C1F56554A;H:DC9894A01797D91D92ECA1DA66242209;T:23D1F8CAC9001F69630ED2DD8DF67DD3BE5C470B5EA97B622F757FE102D8BF14BEDC94A3CC046D10858D885DB656DC0CBF899A79CD8C76B788744844CADE54EEEB4FDEC478FB7C7CBFBBAC57BA3EF22C         Nonreplayable hash  raw-sha1,oracle
                       oracle11_epsilon    S:8F2D65FB5547B71C8DA3760F10960428CD307B1C6271691FC55C1F56554A;H:DC9894A01797D91D92ECA1DA66242209;T:23D1F8CAC9001F69630ED2DD8DF67DD3BE5C470B5EA97B622F757FE102D8BF14BEDC94A3CC046D10858D885DB656DC0CBF899A79CD8C76B788744844CADE54EEEB4FDEC478FB7C7CBFBBAC57BA3EF22C         Nonreplayable hash  raw-sha1,oracle
                       oracle12c_epsilon   H:DC9894A01797D91D92ECA1DA66242209;T:E3243B98974159CC24FD2C9A8B30BA62E0E83B6CA2FC7C55177C3A7F82602E3BDD17CEB9B9091CF9DAD672B8BE961A9EAC4D344BDBA878EDC5DCB5899F689EBD8DD1BE3F67BFF9813A464382381AB36B                                                                        Nonreplayable hash  pbkdf2,oracle12c
                       example             md5be86a79bf2043622d58d5453c47d4860                                                                                                                                                                                                                                          Postgres md5        raw-md5,postgres
@busterb busterb added the msf5 label Feb 13, 2019
@busterb busterb merged commit f4f495e into rapid7:master Feb 13, 2019
3 checks passed
3 checks passed
Metasploit Automation - Sanity Test Execution Successfully completed all tests.
Details
Metasploit Automation - Test Execution Successfully completed all tests.
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
busterb added a commit that referenced this pull request Feb 13, 2019
@busterb

This comment has been minimized.

Copy link
Member

busterb commented Feb 13, 2019

Release Notes

This modernizes Metasploit's interaction with the John the Ripper password cracker by updating configurations, adding current hash types, removing obsolete decryption modes, and fixing numerous bugs. It also adds a new module, auxiliary/analyze/apply_pot, which uses a John the Ripper 'pot' file to accelerate processing of previously-cracked password hashes.

@h00die h00die deleted the h00die:jtr_3rd_try branch Feb 13, 2019
@h00die

This comment has been minimized.

Copy link
Contributor Author

h00die commented Feb 13, 2019

Thanks @busterb

@@ -0,0 +1,180 @@
## Vulnerable Application

This module applys a john the ripper (or hashcat) style .pot file to hashes in the database.

This comment has been minimized.

Copy link
@wvu-r7

wvu-r7 Feb 14, 2019

Contributor

applys

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.