diff --git a/doc/Regen-Lost-Salts.txt b/doc/Regen-Lost-Salts.txt index 5e2c5004321..541b8b31d0e 100644 --- a/doc/Regen-Lost-Salts.txt +++ b/doc/Regen-Lost-Salts.txt @@ -1,42 +1,43 @@ -*************************************************************************** -* JtR --regen-lost-salts code, written by JimF, 2012-2015, for use -* within the salted dynamic formats. -* -* No copyright is claimed, and the software is hereby -* placed in the public domain. In case this attempt to disclaim -* copyright and place the software in the public domain is deemed -* null and void, then the software is Copyright (c) 2012-2015 JimF -* and it is hereby released to the general public under the following -* terms: -* -* This software may be modified, redistributed, and used for any -* purpose, in source and binary forms, with or without modification. -*************************************************************************** - - ---regen-lost-salts=type:hash_sz:mask - -This option will allow certain types (with short salts), to be found, when -they are present in a set of raw hashes, but where we have lost the salt value. -Normally, without the salt, JtR would not be able to do anything with the hash, -and could not find the password. In this mode, JtR will load the hashes, giving -each a 'fake' salt (but all get the same salt). JtR then builds all of the -possible salt values for this format, and associates every hash with each one -of these salts. So, JtR will now run, and find passwords using all possible -salts for each of the hashes, thus recreating the salt (and finding the passwords). - -This function has recently been re-written. The prior way of usage is still supported -(limited), and should be considered depricated. See the end of this document for -the *** NOTE section. +Salt brute-force feature: --regen-lost-salts + +This option will allow hashes to be cracked even where the salt, normally +stored together with the hash, was somehow lost. Normally a cracker would not +be able to attack such hashes. Using this feature, only supported for dynamic +formats (including "dynamic compiler" ones), JtR will load the hashes and +brute-force all possible salts for each. + +Obviously this comes at a price - but not always: If you attack a single hash +with 1000 possible salts, it will end up with same same p/s as an attack on +1000 uniquly salted hashes. If on the other hand the number of hashes is +larger than 1000, you'll end up with no penalty (well, until you've cracked so +many that there are less than 1000 left to crack) and for very short salt +sizes and/or large number of hashes, normal "same-salt boosts" may even apply. + +The feature should only be used with bare hashes input, meaning the ciphertext +should be, for example, raw hex such as d3d9446802a44259755d38e6d163e820 without +any tag such as the "$dynamic_xx$" prefix and obviously without the "$babe" salt +suffix. Other fields such as login/gecos are optionally supported though, just +like they always are: + + user:d3d9446802a44259755d38e6d163e820:101:100:Joe Random Luser + +All salts should be missing in the input file; Mixing missing-salt hashes with +correctly salted ones in one same session is not supported. Usage: +--regen-lost-salts=:: ---regen-lost-salts=dynamic_9:32:?d?d?d- --format=dynamic_9 +Showing cracked hashes: +Use --show as usual, but also include the full --regen-lost-salts parameters +used while cracking them. This is required in order to match the pot file +entries (which have a format tag and the correct salt appended) to the bare +hashes input. -Note, regen has been updated to work with dynamic generic expression compiler also. -Usage for same media-wiki as above is: ---regen-lost-salts='@dynamic=md5($s.md5($p))@:32:?d?d?d-' --format='dynamic=md5($s.md5($p))' +Detailed example: +--regen-lost-salts=dynamic_9:32:?d?d?d- +or, using dynamic compiler mode: +--regen-lost-salts='@dynamic=md5($s.md5($p))@:32:?d?d?d-' The above command will run on a set of 32 byte 'raw' hashes, but which are known to contain media-wiki hashes. Media-wiki is of the format md5(salt.-.md5(pass)) @@ -70,16 +71,19 @@ So that 'parts' of the --regen-lost-salts are: salt code will NOT work for salts this large. Here are more details about the mask value: +- Note that these mask placeholders often differ from what is used by mask mode + and other parts of John. For backward compatibility reasons, we are reluctant + to change this. - mask can contain static bytes. These will simply always be output. The example above did this with the '-' character in the last spot. - the mask can use different 'character class' bytes. These are similar to the character types within JtR rules. To set a 'class' for a specific byte location, simply use 2 characters, a question mark, and the class character. Here are the classes. - ?? - becomes a single ? char (2 question marks mean a literal question mark) + ?? - becomes a single ? char (2 question marks mean a literal question mark) ?d - decimal digits [0-9] ** very common salt ** ?l - lower case letters [a-z] (only ANSI lower case, does not take encodings - into account) + into account) ?u - upper case letters [A-Z] ?a - upper / lower case letters [a-zA-Z] ?h - hex lower case [0-9a-f] ** common salt ** @@ -111,21 +115,11 @@ now, use this command line for john: --regen-lost-salts=dynamic_9:32:?1?d?d- --format=dynamic_9 This will use the user class-1 for the first byte, then digits for the other 2. -This means our salt will start from 100- and go to 999- which skips 10% of the -salts, and speeds things up by 10%. This is due to the user-class-1 not having -the '0' byte in it. -At this time, only a fixed length salt is handled. Doing a variable sized salt -regen, simply adds too much complexity, and does slow things down, just a touch. -This is different than the older --regen-lost-salts=3 This 'used to do 0- to 9- -then 00- to 99- then 000- to 999- The 'depricated' --regen-lost-salts=3 now only -does 000- to 999- and would require running 2 more regen-salt runs to cover the -entire salt range. +An ancient version of the code used a single 'type' digit to specify what format +to attack. This still works but is deprecated: -****************************************************************************** - -*** NOTE, depricated method (still works, but should not be used) There are only a few types supported. To properly run, you MUST use one of the proper types in the -format=type command, AND use the -regen-lost-salts=N with N being set properly. The valid N's are: @@ -147,7 +141,3 @@ find them using 'normal' methods. This new functionality was added to handle leaked hashes where the original salt has been lost. It is VERY slow to run in this manner, due to JtR having to check every possible salt, but it does allow these hashes to be cracked 'properly'. - -NOTE, the dyna-generic expression compiler is now supported, so dummy dynamic -formats do not have to be hand coded simply to use the regen-salts logic on -hashes where the salts were lost. diff --git a/src/cracker.c b/src/cracker.c index 0da3d95dcb2..7245b67aea8 100644 --- a/src/cracker.c +++ b/src/cracker.c @@ -374,8 +374,7 @@ static void crk_remove_hash(struct db_salt *salt, struct db_password *pw) } /* Negative index is not counted/reported (got it from pot sync) */ -static int crk_process_guess(struct db_salt *salt, struct db_password *pw, - int index) +static int crk_process_guess(struct db_salt *salt, struct db_password *pw, int index) { char utf8buf_key[PLAINTEXT_BUFFER_SIZE + 1]; char utf8login[PLAINTEXT_BUFFER_SIZE + 1]; @@ -478,6 +477,23 @@ static int crk_process_guess(struct db_salt *salt, struct db_password *pw, if (!(crk_params->flags & FMT_NOT_EXACT)) crk_remove_hash(salt, pw); + if (options.regen_lost_salts) { + /* + * salt->list pointer was copied to all salts so if the first + * entry was removed, we need to fixup all other salts. If OTOH + * the last hash was removed, we need to drop all salts. + */ + struct db_salt *next, *s = crk_db->salts; + + do { + next = s->next; + if (!crk_db->password_count) + crk_remove_salt(s); + else if (s->list && s->list->binary == NULL) + s->list = s->list->next; + } while ((s = next)); + } + if (!crk_db->salts) return 1; @@ -891,8 +907,7 @@ static int crk_password_loop(struct db_salt *salt) if (crk_methods.cmp_all(pw->binary, match)) for (index = 0; index < match; index++) if (crk_methods.cmp_one(pw->binary, index)) - if (crk_methods.cmp_exact(crk_methods.source( - pw->source, pw->binary), index)) { + if (crk_methods.cmp_exact(crk_methods.source(pw->source, pw->binary), index)) { if (crk_process_guess(salt, pw, index)) return 1; else { diff --git a/src/dynamic_fmt.c b/src/dynamic_fmt.c index db1d20deccf..10193b0f95e 100644 --- a/src/dynamic_fmt.c +++ b/src/dynamic_fmt.c @@ -1011,16 +1011,16 @@ static char *prepare(char *split_fields[10], struct fmt_main *pFmt) // at this point max length is still < 512. 491 + strlen($dynamic_xxxxx$) is 506 - if (strncmp(cpBuilding, "$dynamic_", 9)) { - // ok, here we add the 'generic' regen salt code - if (options.regen_lost_salts && !strchr(cpBuilding, '$')) { - char *cp = load_regen_lost_salt_Prepare(cpBuilding); - if (cp) - return cp; - } - return split_fields[1]; + // If --regen-lost-salts and salt is missing, add the first possible salt + if (options.regen_lost_salts && !strchr(cpBuilding + strlen(pFmt->params.label) + 2, '$')) { + char *cp = load_regen_lost_salt_Prepare(cpBuilding); + if (cp) + return cp; } + if (strncmp(cpBuilding, "$dynamic_", 9)) + return split_fields[1]; + if ( (pPriv->pSetup->flags&MGF_SALTED) == 0) return cpBuilding; @@ -1101,22 +1101,20 @@ static char *prepare(char *split_fields[10], struct fmt_main *pFmt) static char *split(char *ciphertext, int index, struct fmt_main *pFmt) { static char out[1024]; - char search_char = '$'; private_subformat_data *pPriv = pFmt->private.data; if (strnlen(ciphertext, 951) > 950) return ciphertext; - if (!strncmp(ciphertext, "@dynamic=", 9)) - search_char = '@'; - // mime. We want to strip off ALL trailing '=' characters to 'normalize' them if (pPriv->dynamic_base64_inout == 3 && (!strncmp(ciphertext, "$dynamic_", 9) || !strncmp(ciphertext, "@dynamic=", 9))) { static char ct[496]; unsigned int len; + char search_char = (!strncmp(ciphertext, "@dynamic=", 9)) ? '@' : '$'; char *cp = strchr(&ciphertext[9], search_char), *cp2; + if (cp) { ++cp; len = base64_valid_length(cp, e_b64_mime, flg_Base64_MIME_TRAIL_EQ_CNT, 0); diff --git a/src/fake_salts.c b/src/fake_salts.c index e8d7279db15..fdf9c052693 100644 --- a/src/fake_salts.c +++ b/src/fake_salts.c @@ -1,24 +1,14 @@ /* - * This software was written by JimF jfoug AT cox dot net - * in 2012-2013. No copyright is claimed, and the software is hereby - * placed in the public domain. In case this attempt to disclaim - * copyright and place the software in the public domain is deemed - * null and void, then the software is Copyright (c) 2012-2013 JimF - * and it is hereby released to the general public under the following - * terms: - * - * This software may be modified, redistributed, and used for any - * purpose, in source and binary forms, with or without modification. - * - * Salt finder. This will allow JtR to process a few salted type - * hashes, if the original salt has been lost. The only 2 types - * done at this time, are PHPS (VB), and osCommerce. PHPS is dynamic_6 - * which is md5(md5($p).$s) with a 3 byte salt. osCommerce is dynamic_4 - * of md5($s.$p) with a 2 type salt. + * This software is Copyright (c) 2012-2013 JimF + * and Copyright (c) 2021 magnum, + * and is hereby released to the general public under the following terms: + * Redistribution and use in source and binary forms, with or without + * modifications, are permitted. * + * Salt brute-force. This will allow JtR to process dynamic type hashes + * if the original salt has been lost. */ - #include #include "misc.h" // error() #include "config.h" @@ -27,56 +17,56 @@ #include "options.h" #include "fake_salts.h" -// global data (Options loading uses this variable). char *regen_salts_options; +int regen_salts_count; static char *regen_schema, DynaType[2048], FirstSalt[11]; static int hash_len, DynaTypeLen; -static int salt_len, total_regen_salts_count; -static int loc_cnt[10] = {0}; /* how many chars are used for each location */ -static int cur_cnt[10] = {0}; /* this is a complex number, we use to permute all values. */ -static char *candi[10] = {0}; /* This is the valid chars, for each salt character position. */ - -static void bailout(const char *str) { - if (john_main_process) - fprintf(stderr, "%s\n", str); - error(); -} +static int salt_len; +static int loc_cnt[10] = { 0 }; /* How many chars are used for each location */ +static int cur_cnt[10] = { 0 }; /* This is a complex number, we use to permute all values. */ +static char *candi[10] = { 0 }; /* This is the valid chars, for each salt character position. */ + +#define REGEN_OPTION "--regen-lost-salts: " +#define REGEN_DOCS "See doc/Regen-Lost-Salts.txt\n" -/* this has been made 'generic', and not built for a set of specific formats */ -void build_fake_salts_for_regen_lost(struct db_salt *salts) { - struct db_salt *sp, *fake_salts; +void build_fake_salts_for_regen_lost(struct db_main *database) +{ + struct db_salt *sp, *fake_salts, *salts = database->salts; int i; char dyna_salt_header[7], *buf, *cp; - unsigned long *ptr; + size_t *ptr; - fake_salts = mem_calloc_tiny(sizeof(struct db_salt) * (total_regen_salts_count+1), MEM_ALIGN_WORD); + fake_salts = mem_calloc_tiny(sizeof(struct db_salt) * regen_salts_count, MEM_ALIGN_WORD); - // Find the 'real' salt. We loaded ALL of the file into 1 salt. - // we then take THAT salt record, and build a list pointing to these fake salts, - // AND build 'proper' dynamic salts for all of our data. + /* + * Find the orignal salt. We loaded all of the hashes into the first possible salt (eg. "0" or "aa"). + * We now duplicate that salt record, pointing them to all other possible salts. + */ sp = salts; - while (sp->next) { - sp = sp->next; - } - // a dynamic salt is 0x0000[salt_len-byte-salt] for ALL of the salts. - buf = mem_alloc_tiny(total_regen_salts_count*(6+salt_len)+1, MEM_ALIGN_NONE); + if (sp->next) + error_msg(REGEN_OPTION "More salts loaded than expected; Something's not right!\n"); + + if (memcmp(*((char**)sp->salt) + 6, FirstSalt, salt_len)) + error_msg(REGEN_OPTION "Database not looking like expected; Something's not right!\n" + "Please only use this option with bare hashes. " REGEN_DOCS); + + /* Salt is : 0N0000REAL_SALT_STRING (real salt string is N bytes long). */ + buf = mem_alloc_tiny(regen_salts_count * (6 + salt_len) + 1, MEM_ALIGN_NONE); cp = buf; sprintf(dyna_salt_header, "%02d0000", salt_len); - // we start from salt 1, (all hashes should already be salted to salt0) - for (i = 1; i < total_regen_salts_count; ++i) { + /* We start from salt 1. Salt 0 was already written by dynamic prepare function. */ + for (i = 1; i < regen_salts_count; ++i) { int j = 0; char *cp2 = cp; - // Now compute next salt NOTE, we start from salt1 not salt0, so we first need - // to increment out 'complex' number, prior to using it. - cur_cnt[j=0]++; + cur_cnt[j = 0]++; while (cur_cnt[j] >= loc_cnt[j]) { cur_cnt[j++] = 0; - if (j==10) - break; // done, but we should never get here (i should be == total_regen_salts_count before we hit this) + if (j == 10) + break; /* Safety, but we should never get here */ ++cur_cnt[j]; } @@ -84,29 +74,30 @@ void build_fake_salts_for_regen_lost(struct db_salt *salts) { cp2 += 6; for (j = 0; j < salt_len; ++j) *cp2++ = candi[j][cur_cnt[j]]; - // now link in this salt struct + database->salt_count++; sp->next = &fake_salts[i]; + + /* Copy the whole salt as-is, then change the few members that differs */ + fake_salts[i] = *sp; /* Note this is a full struct copy, akin to memcpy */ fake_salts[i].next = NULL; - fake_salts[i].count = sp->count; - fake_salts[i].hash = sp->hash; - fake_salts[i].hash_size = sp->hash_size; - fake_salts[i].index = sp->index; - fake_salts[i].keys = sp->keys; - fake_salts[i].list = sp->list; - fake_salts[i].bitmap = sp->bitmap; // 'bug' fix when we went to bitmap. Old code was not copying this. - ptr=mem_alloc_tiny(sizeof(char*), MEM_ALIGN_WORD); - *ptr = (size_t) (buf + (cp-buf)); + ptr = mem_alloc_tiny(sizeof(char*), MEM_ALIGN_WORD); + *ptr = (size_t) (buf + (cp - buf)); fake_salts[i].salt = ptr; + fake_salts[i].sequential_id++; + cp = cp2; sp = sp->next; } } -/* this is called from dynamic prepare() function, whenever we have a 'raw' hash, and when we are in re-gen salts mode */ -char *load_regen_lost_salt_Prepare(char *split_fields1) { +/* This is called from dynamic prepare() function when we are in regen salts mode */ +char *load_regen_lost_salt_Prepare(char *split_fields1) +{ char *cp = split_fields1, *Prep, *pPrep; int len; + if (options.flags & FLG_SHOW_CHK) + return split_fields1; if (cp && *cp == '$' && !strncmp(cp, DynaType, DynaTypeLen)) cp += DynaTypeLen; if (!cp) @@ -118,44 +109,55 @@ char *load_regen_lost_salt_Prepare(char *split_fields1) { Prep = mem_alloc_tiny(DynaTypeLen+hash_len+1+salt_len+1, MEM_ALIGN_NONE); pPrep = Prep; - // add a the 'first' salt that is the proper. So, once loaded, there will - // be only ONE salt, but ALL candidates will use this salt. We then load - // all possible salt structures, and link all input candidates to ALL of them. + /* + * Add a the first possible salt. So, once loaded, there will be only one salt, + * but all candidates will use this salt. We then load all remaining possible + * salts and link all input candidates to all of them. + */ pPrep += sprintf(Prep, "%s%s$%s", DynaType, cp, FirstSalt); return Prep; } -/* this is called from cracker.c crk_process_guess() function, and here we FIX the crack with the */ -/* proper found salt. When the data was loaded, it was assigned the 'first' salt, but likely we */ -/* cracked this hash with a different salt (the real one). Here we fix it before writing to .pot */ -void crk_guess_fixup_salt(char *source, char *salt) { - // salt is : 0N0000REAL_SALT_STRING (real salt string is N bytes long). - int real_salt_len = salt[1]-'0'; - memcpy(source+DynaTypeLen+hash_len+1, &salt[6], real_salt_len); - source[DynaTypeLen+hash_len+1+real_salt_len] = 0; +/* + * This is called from crk_process_guess(), and here we fixup the crack with the found working + * salt. When the data was loaded, it was assigned the 'first' salt, but likely we cracked + * this hash with a different salt. Here we fix it before writing to .pot + * Salt is : 0N0000REAL_SALT_STRING (real salt string is N bytes long). + */ +void crk_guess_fixup_salt(char *source, char *salt) +{ + int real_salt_len = salt[1] - '0'; + + memcpy(source + DynaTypeLen + hash_len + 1, &salt[6], real_salt_len); + source[DynaTypeLen + hash_len + 1 + real_salt_len] = 0; } -/* this is called from loader.c ldr_load_pot_line() function. During loading of the .pot file, */ -/* the prepare has lost the proper salt. We now need to fix that to the 'correct' salt. */ -void ldr_pot_possible_fixup_salt(char *source, char *ciphertext) { +/* + * This is called from ldr_load_pot_line(). During loading of the .pot file, the prepare + * has lost the proper salt. We now need to fix that to the 'correct' salt. + */ +void ldr_pot_possible_fixup_salt(char *source, char *ciphertext) +{ if (!strncmp(source, DynaType, DynaTypeLen)) { - memcpy(&(source[DynaTypeLen+hash_len+1]), &(ciphertext[DynaTypeLen+hash_len+1]), salt_len); + memcpy(&(source[DynaTypeLen + hash_len + 1]), &(ciphertext[DynaTypeLen + hash_len + 1]), salt_len); } } -/* Load a custom user class string from john.pot from the [Regen_Salts_UserClasses] section. */ -/* NOTE, we have to init the config file, since this will be called within option loading */ -static char *LoadUserClass(char which, int i) { - // Load user-defined character classes ?0 .. ?9 from john.conf +/* + * Load user-defined character classes ?0 .. ?9 from john.conf + */ +static char *LoadUserClass(char which, int i) +{ char user_class_num[2]; const char *user_class; static int loaded=0; user_class_num[0] = which; user_class_num[1] = 0; - // The config has not been loaded, so we have to load it now, if we want to 'check' - // and show any user set md5-generic functions. + /* + * The config has not yet been loaded, so we have to load it now + */ if (!loaded) { #if JOHN_SYSTEMWIDE cfg_init(CFG_PRIVATE_FULL_NAME, 1); @@ -171,59 +173,70 @@ static char *LoadUserClass(char which, int i) { return NULL; } -/* at the end of options loading, this function is called. It will return 0 or 1. If the user did NOT */ -/* use the --regen-lost-salts option or not. If the user used it, but it was not valid, then we abort with */ -/* an error message. Once this function is done, and returns a 1, then all data should be setup and ready */ -int regen_lost_salt_parse_options() { +/* + * At the end of options loading, this function is called. It will return 0 if the user didn't use + * the --regen-lost-salts option at all. If the user used it but it wasn't valid, we abort with a + * message. Once this function is done and returns a 1, then all data should be setup and ready. + */ +int regen_lost_salt_parse_options() +{ char *cp; int i, regen_salts_dyna_num; - if (regen_salts_options==NULL) return 0; - if (!strcmp(regen_salts_options, "1")) regen_salts_options="dynamic_6:32:?y?y?y"; - else if (!strcmp(regen_salts_options, "2")) regen_salts_options="dynamic_4:32:?y?y"; - // NOTE mediawiki 3 here is not doing ?d- or ?d?d- I am not implementing 'variable' length regen - // var length would add a LOT of complexity and may reduce speed a little. - else if (!strcmp(regen_salts_options, "3")) regen_salts_options="dynamic_9:32:?d?d?d-"; - else if (!strcmp(regen_salts_options, "4")) regen_salts_options="dynamic_9:32:?d?d?d?d-"; - else if (!strcmp(regen_salts_options, "5")) regen_salts_options="dynamic_9:32:?d?d?d?d?d-"; - else if (!strcmp(regen_salts_options, "6")) regen_salts_options="dynamic_61:64:?d?d"; + if (regen_salts_options == NULL) + return 0; + + if (!strcmp(regen_salts_options, "1")) regen_salts_options = "dynamic_6:32:?y?y?y"; + else if (!strcmp(regen_salts_options, "2")) regen_salts_options = "dynamic_4:32:?y?y"; + /* + * mediawiki 3 here is not doing ?d- or ?d?d- I am not implementing 'variable' length regen + * var length would add a lot of complexity and may reduce speed a little. + */ + else if (!strcmp(regen_salts_options, "3")) regen_salts_options = "dynamic_9:32:?d?d?d-"; + else if (!strcmp(regen_salts_options, "4")) regen_salts_options = "dynamic_9:32:?d?d?d?d-"; + else if (!strcmp(regen_salts_options, "5")) regen_salts_options = "dynamic_9:32:?d?d?d?d?d-"; + else if (!strcmp(regen_salts_options, "6")) regen_salts_options = "dynamic_61:64:?d?d"; if (!strncmp(regen_salts_options, "@dynamic=", 9)) { char *cp = strrchr(regen_salts_options, '@'); int len; if (!cp) - bailout("Error, invalid @dynamic= signature in the -salt-regen section"); + error_msg(REGEN_OPTION "Invalid @dynamic= signature in parameter"); ++cp; len = cp-regen_salts_options; if (len > sizeof(DynaType) - 1) len = sizeof(DynaType) - 1; - regen_salts_dyna_num=6000; + regen_salts_dyna_num = 6000; if (sscanf(cp, ":%d:", &hash_len) != 1) - bailout("Error, invalid regen-lost-salts argument. Must start with @dynamic=EXPR:hash_len: value\nSee docs/REGEN-LOST-SALTS document"); - // at this point in the JtR loading, we do not know if $dynamic_`regen_salts_dyna_num`$ is valid. We have to check later. + error_msg(REGEN_OPTION "Invalid argument. Must start with @dynamic=EXPR:hash_len: value.\n" REGEN_DOCS); + /* At this point we don't know if $dynamic_`regen_salts_dyna_num`$ is valid. We have to check later. */ sprintf(DynaType, "%*.*s", len, len, regen_salts_options); } else { if (strncmp(regen_salts_options, "dynamic_", 8)) - bailout("Error, invalid regen-lost-salts argument. Must start with dynamic_# value\nSee docs/REGEN-LOST-SALTS document"); + error_msg(REGEN_OPTION "Invalid argument. Must start with dynamic_# value.\n" REGEN_DOCS); if (sscanf(regen_salts_options, "dynamic_%d:%d:", ®en_salts_dyna_num, &hash_len) != 2) - bailout("Error, invalid regen-lost-salts argument. Must start with dynamic_#:hash_len: value\nSee docs/REGEN-LOST-SALTS document"); - // at this point in the JtR loading, we do not know if $dynamic_`regen_salts_dyna_num`$ is valid. We have to check later. + error_msg(REGEN_OPTION "Invalid argument. Must start with dynamic_#:hash_len: value.\n" REGEN_DOCS); + /* At this point we don't know if $dynamic_`regen_salts_dyna_num`$ is valid. We have to check later. */ sprintf(DynaType, "$dynamic_%d$", regen_salts_dyna_num); } DynaTypeLen = strlen(DynaType); - // do 'some' sanity checking on input length. Only known valid input lengths for raw hashes are: - // (we are only handling hex at this time) 2 times: 16, 20, 24, 28, 32, 40, 48, 64 bytes. + /* + * Do some sanity checking on input length. Only known valid input lengths for raw hashes are: + * (we are only handling hex at this time) 2 times: 16, 20, 24, 28, 32, 40, 48, 64 bytes. + */ switch(hash_len) { case 32: case 40: case 48: case 56: case 64: case 80: case 96: case 128: break; default: - bailout("Error, invalid regen-lost-salts argument. Hash_length not valid\nSee docs/REGEN-LOST-SALTS document"); + error_msg(REGEN_OPTION "Invalid hash length\n" REGEN_DOCS); } - // Ok, now parse the string, making sure it is valid. NOTE, since the sscanf gave us a 2 above, - // we know the string has 2 : in it, so we do not need NULL pointer checking here. + /* + * Now parse the string, making sure it is valid. Since the sscanf gave us a 2 above, + * we know the string has 2 : in it, so we don't need NULL pointer checking here. + */ cp = strchr(regen_salts_options, ':'); cp = strchr(&cp[1], ':'); ++cp; @@ -231,85 +244,86 @@ int regen_lost_salt_parse_options() { i = 0; while (*cp) { - // d=dec h=hex H=HEX x=hHeExX b=binary o=oct a=a-zA-Z l=a-z u=A-Z n=a-zA-Z0-9 y=' '-~ (95 chars) + /* d=dec h=hex H=HEX x=hHeExX b=binary o=oct a=a-zA-Z l=a-z u=A-Z n=a-zA-Z0-9 y=' '-~ (95 chars) */ if (*cp == '?') { switch(cp[1]) { - case 'd': - loc_cnt[i] = 10; - candi[i] = "0123456789"; - break; - case 'h': - candi[i] = "0123456789abcdef"; - loc_cnt[i] = 16; - break; - case 'H': - candi[i] = "0123456789ABCDEF"; - loc_cnt[i] = 16; - break; - case 'x': - candi[i] = "0123456789abcdefABCDEF"; - loc_cnt[i] = 22; - break; - case 'b': - candi[i] = "01"; - loc_cnt[i] = 2; - break; - case 'o': - candi[i] = "012345678"; - loc_cnt[i] = 8; - break; - case 'a': - candi[i] = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ"; - loc_cnt[i] = 52; - break; - case 'l': - candi[i] = "abcdefghijklmnopqrstuvwxyz"; - loc_cnt[i] = 26; - break; - case 'u': - candi[i] = "ABCDEFGHIJKLMNOPQRSTUVWXYZ"; - loc_cnt[i] = 26; - break; - case 'n': - candi[i] = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789"; - loc_cnt[i] = 62; - break; - case 'y': - candi[i] = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789 `~!@#$%^&*()_-+=[{]};:,<.>/?|\\'\""; - loc_cnt[i] = 95; - break; - case '?': - loc_cnt[i] = 1; - candi[i] = mem_alloc_tiny(2,1); - candi[i][0] = cp[1]; - candi[i][1] = 0; - break; - // need to handle user types also. - case '0': - case '1': - case '2': - case '3': - case '4': - case '5': - case '6': - case '7': - case '8': - case '9': - { - char *classData = LoadUserClass(cp[1], i); - if (!classData) - bailout("Error, invalid regen-lost-salts argument. Invalid class character in salt schema\nSee docs/REGEN-LOST-SALTS document"); - candi[i] = classData; - loc_cnt[i] = strlen(classData); - break; - } - default: - bailout("Error, invalid regen-lost-salts argument. Invalid class character in salt schema\nSee docs/REGEN-LOST-SALTS document"); + case 'd': + loc_cnt[i] = 10; + candi[i] = "0123456789"; + break; + case 'h': + candi[i] = "0123456789abcdef"; + loc_cnt[i] = 16; + break; + case 'H': + candi[i] = "0123456789ABCDEF"; + loc_cnt[i] = 16; + break; + case 'x': + candi[i] = "0123456789abcdefABCDEF"; + loc_cnt[i] = 22; + break; + case 'b': + candi[i] = "01"; + loc_cnt[i] = 2; + break; + case 'o': + candi[i] = "012345678"; + loc_cnt[i] = 8; + break; + case 'a': + candi[i] = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ"; + loc_cnt[i] = 52; + break; + case 'l': + candi[i] = "abcdefghijklmnopqrstuvwxyz"; + loc_cnt[i] = 26; + break; + case 'u': + candi[i] = "ABCDEFGHIJKLMNOPQRSTUVWXYZ"; + loc_cnt[i] = 26; + break; + case 'n': + candi[i] = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789"; + loc_cnt[i] = 62; + break; + case 'y': + candi[i] = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789" + " `~!@#$%^&*()_-+=[{]};:,<.>/?|\\'\""; + loc_cnt[i] = 95; + break; + case '?': + loc_cnt[i] = 1; + candi[i] = mem_alloc_tiny(2, MEM_ALIGN_NONE); + candi[i][0] = cp[1]; + candi[i][1] = 0; + break; + /* User types */ + case '0': + case '1': + case '2': + case '3': + case '4': + case '5': + case '6': + case '7': + case '8': + case '9': + { + char *classData = LoadUserClass(cp[1], i); + if (!classData) + error_msg(REGEN_OPTION "Invalid argument. Invalid class character in salt schema.\n" REGEN_DOCS); + candi[i] = classData; + loc_cnt[i] = strlen(classData); + break; + } + default: + error_msg(REGEN_OPTION "Invalid argument. Invalid class character in salt schema.\n" REGEN_DOCS); } ++cp; } else { loc_cnt[i] = 1; - candi[i] = mem_alloc_tiny(2,1); + candi[i] = mem_alloc_tiny(2, MEM_ALIGN_NONE); candi[i][0] = cp[0]; candi[i][1] = 0; } @@ -320,13 +334,13 @@ int regen_lost_salt_parse_options() { } FirstSalt[i] = 0; if (salt_len > 9) { - bailout("Error, invalid regen-lost-salts argument. The max length salt can only be 99 bytes long\nSee docs/REGEN-LOST-SALTS document"); + error_msg(REGEN_OPTION "Invalid argument. The max length salt can only be 99 bytes long.\n" REGEN_DOCS); } - total_regen_salts_count = 1; + regen_salts_count = 1; for (i = 0; i < salt_len; ++i) { - if (total_regen_salts_count * loc_cnt[i] < total_regen_salts_count) - bailout("too many re-gen salt values requested to be able to allocate them\n"); - total_regen_salts_count *= loc_cnt[i]; + if (regen_salts_count * loc_cnt[i] < regen_salts_count) + error_msg(REGEN_OPTION "Unsupported number of salt values\n"); + regen_salts_count *= loc_cnt[i]; } return 1; } diff --git a/src/fake_salts.h b/src/fake_salts.h index d0f0b8fbc60..5a38080bd35 100644 --- a/src/fake_salts.h +++ b/src/fake_salts.h @@ -52,9 +52,10 @@ From jtr rules. Using ! instead of ? means 'optional' character. So ?d?d?d?d- */ extern char *regen_salts_options; +extern int regen_salts_count; extern int regen_lost_salt_parse_options(); extern char *load_regen_lost_salt_Prepare(char *split_fields1); extern void crk_guess_fixup_salt(char *source, char *salt); extern void ldr_pot_possible_fixup_salt(char *source, char *ciphertext); -extern void build_fake_salts_for_regen_lost(struct db_salt *); +extern void build_fake_salts_for_regen_lost(struct db_main *); diff --git a/src/john.c b/src/john.c index 589b56bd5a8..be7664f2580 100644 --- a/src/john.c +++ b/src/john.c @@ -731,19 +731,36 @@ char *john_loaded_counts(struct db_main *db, char *prelude) if (db->password_count == 0) return "No remaining hashes"; - if (db->password_count == 1) { + if (db->password_count == 1 && !options.regen_lost_salts) { sprintf(buf, "%s 1 password hash", prelude); return buf; } - int p = sprintf(buf, "%s %d password hashes with %s different salts", - prelude, db->password_count, db->salt_count > 1 ? - jtr_itoa(db->salt_count, nbuf, 24, 10) : "no"); + int salt_count = db->salt_count; - int b = (10 * db->password_count + (db->salt_count / 2)) / db->salt_count; + /* At the time we say "Loaded xx hashes", the regen code hasn't yet updated the salt_count */ + if (options.regen_lost_salts && salt_count == 1) + salt_count = regen_salts_count; - if (p >= 0 && db->salt_count > 1 && b > 10) - sprintf(buf + p, " (%d.%dx same-salt boost)", b / 10, b % 10); + int p = sprintf(buf, "%s %d password hash%s with %s %s salts", prelude, db->password_count, + db->password_count > 1 ? "es" : "", + salt_count > 1 ? jtr_itoa(salt_count, nbuf, 24, 10) : "no", + (salt_count > 1 && options.regen_lost_salts) ? "possible" : "different"); + + if (p >= 0) { + if (options.regen_lost_salts && db->password_count < salt_count) { + int bf_penalty = 10 * salt_count / db->password_count; + + if (bf_penalty > 10) + sprintf(buf + p, " (%d.%dx salt BF penalty)", bf_penalty / 10, bf_penalty % 10); + } else { + int bf_penalty = options.regen_lost_salts ? regen_salts_count : 1; + int boost = (10 * bf_penalty * db->password_count + (salt_count / 2)) / salt_count; + + if (salt_count > 1 && boost > 10) + sprintf(buf + p, " (%d.%dx same-salt boost)", boost / 10, boost % 10); + } + } return buf; } @@ -1212,6 +1229,9 @@ static void john_load(void) ldr_fix_database(&database); + if (database.password_count && options.regen_lost_salts) + build_fake_salts_for_regen_lost(&database); + if (!database.password_count) { log_discard(); if (john_main_process) @@ -1249,10 +1269,9 @@ static void john_load(void) database.min_cost[i]); } } - if ((options.flags & FLG_PWD_REQ) && !database.salts) exit(0); - if (options.regen_lost_salts) - build_fake_salts_for_regen_lost(database.salts); + if ((options.flags & FLG_PWD_REQ) && !database.salts) + exit(0); } /* diff --git a/src/loader.c b/src/loader.c index 9041f0233bf..ad89e2014dc 100644 --- a/src/loader.c +++ b/src/loader.c @@ -2168,6 +2168,17 @@ static int ldr_cracked_hash(char *ciphertext) return hash; } +static int drop_regen_salt(char *line) +{ + char *sdl; + + if ((sdl= strrchr(line, '$'))) { + *sdl = 0; + return 1; + } + return 0; +} + static void ldr_show_pot_line(struct db_main *db, char *line) { char *ciphertext, *pos; @@ -2228,6 +2239,31 @@ static void ldr_show_pot_line(struct db_main *db, char *line) return; } + /* + * Bodge for --show to work w/ --regen-lost-salts + * + * This requires you to supply the same --regen-lost-salts parameters with --show + * as what what used during cracking (actually just the total length need to match) + */ + if (options.regen_lost_salts) { + char *p; + + if (!strncmp(ciphertext, "$dynamic_", 9)) { + p = ciphertext + 10; + if ((p = strchr(p, '$'))) + p++; + if (drop_regen_salt(p)) + ciphertext = p; + } else + if (!strncmp(ciphertext, "@dynamic=", 9)) { + p = ciphertext + 10; + if ((p = strchr(p, '@'))) + p++; + if (drop_regen_salt(p)) + ciphertext = p; + } + } + if (format) ciphertext = format->methods.split(ciphertext, 0, format); diff --git a/src/options.c b/src/options.c index abb5e6f332d..72b82317c92 100644 --- a/src/options.c +++ b/src/options.c @@ -180,7 +180,7 @@ static struct opt_entry opt_list[] = { {"max-candidates", FLG_ONCE, 0, FLG_CRACKING_CHK, USUAL_REQ_CLR | OPT_REQ_PARAM, "%lld", &options.max_cands}, {"max-run-time", FLG_ONCE, 0, FLG_CRACKING_CHK, USUAL_REQ_CLR | OPT_REQ_PARAM, "%d", &options.max_run_time}, {"progress-every", FLG_ONCE, 0, FLG_CRACKING_CHK, USUAL_REQ_CLR | OPT_REQ_PARAM, "%u", &options.status_interval}, - {"regen-lost-salts", FLG_ONCE, 0, FLG_CRACKING_CHK, USUAL_REQ_CLR | OPT_REQ_PARAM, OPT_FMT_STR_ALLOC, ®en_salts_options}, + {"regen-lost-salts", FLG_ONCE, 0, FLG_PWD_REQ, USUAL_REQ_CLR | OPT_REQ_PARAM, OPT_FMT_STR_ALLOC, ®en_salts_options}, {"bare-always-valid", FLG_ONCE, 0, FLG_PWD_REQ, OPT_REQ_PARAM, "%c", &options.dynamic_bare_hashes_always_valid}, {"reject-printable", FLG_REJECT_PRINTABLE, FLG_REJECT_PRINTABLE}, {"verbosity", FLG_VERBOSITY, FLG_VERBOSITY, 0, OPT_REQ_PARAM, "%u", &options.verbosity}, @@ -1010,6 +1010,28 @@ void opt_init(char *name, int argc, char **argv) options.regen_lost_salts = regen_lost_salt_parse_options(); + /* + * The format should never have been a parameter to --regen-lost-salts but now that we have to live with it: + * If --regen-lost-salts=TYPE:hash_sz:mask and no --format option was given, infer --format=TYPE. + * If on the other hand --format=TYPE *was* given, require that they actually match. + */ + if (options.regen_lost_salts) { + char *s = str_alloc_copy(regen_salts_options); + char *e = strchr(s + 1, ':'); + + if (e > s + 8) { + if (*s == '@') { + s++; + e--; + } + *e = 0; + if (!options.format) + options.format = s; + else if (strcmp(options.format, s)) + error_msg("Error: --regen-lost-salts parameter not matching --format option\n"); + } + } + if (field_sep_char_str) { // Literal tab or TAB will mean 0x09 tab character if (!strcasecmp(field_sep_char_str, "tab")) diff --git a/src/options.h b/src/options.h index 139c5843cf8..5a684555f55 100644 --- a/src/options.h +++ b/src/options.h @@ -366,14 +366,7 @@ struct options_main { /* Stacked rules applied within cracker.c for any mode */ char *rule_stack; -/* This is a 'special' flag. It causes john to add 'extra' code to search for - * some salted types, when we have only the hashes. The only type supported is - * PHPS (at this time.). So PHPS will set this to a 1. OTherwise it will - * always be zero. LIKELY we will add the same type logic for the OSC - * (mscommerse) type, which has only a 2 byte salt. That will set this field - * to be a 2. If we add other types, then we will have other values which can - * be assigned to this variable. This var is set by the undocummented - * --regen_lost_salts=# */ +/* Salt brute-force */ int regen_lost_salts; /* Requested max_keys_per_crypt (for testing purposes) */