Adding parser and basic kernels for -m 13500 #311

Merged
merged 15 commits into from Apr 26, 2016

Conversation

Projects
None yet
2 participants
@fgaudreault
Contributor

fgaudreault commented Apr 22, 2016

Adding the parser for PS_TOKEN long salt, and copy of the -m 140 kernels to start with.

See Issue #306

@jsteube

This comment has been minimized.

Show comment
Hide comment
@jsteube

jsteube Apr 22, 2016

Member

It's a good start, but we have to do some changes:

  • First I'd like to move the hashmode from 134 to 13500 (I'll reserver that for your hash-mode).
  • Then I've seen that you've used parse_and_store_salt() to store the salt. This is invalid in your case, as the salt is so long. Also note that the salt_buf in salt_t is limited to 64 byte, so you've propably already overwrote it.
  • To make this correctly, you first need to define a new esalt (external salt) and give it enough space to handle it. Like 512 byte. Note that you have to work with 32 bit integers so you need to create something like u32 buf[128].
  • After that you need to create the space for the salt, see other algorithms like 13400 in how to do that.
  • Next you need to store the salt into this esalt manually and without parse_and_store_salt. There's also plenty of examples in other parser functions.
  • Finally I'd suggest to copy at least 16 byte of that salt into the original salt_t salt_buf because based on that it will find if a hash is unique in some internal sorting routine

After this is done we can start with the kernel code

Member

jsteube commented Apr 22, 2016

It's a good start, but we have to do some changes:

  • First I'd like to move the hashmode from 134 to 13500 (I'll reserver that for your hash-mode).
  • Then I've seen that you've used parse_and_store_salt() to store the salt. This is invalid in your case, as the salt is so long. Also note that the salt_buf in salt_t is limited to 64 byte, so you've propably already overwrote it.
  • To make this correctly, you first need to define a new esalt (external salt) and give it enough space to handle it. Like 512 byte. Note that you have to work with 32 bit integers so you need to create something like u32 buf[128].
  • After that you need to create the space for the salt, see other algorithms like 13400 in how to do that.
  • Next you need to store the salt into this esalt manually and without parse_and_store_salt. There's also plenty of examples in other parser functions.
  • Finally I'd suggest to copy at least 16 byte of that salt into the original salt_t salt_buf because based on that it will find if a hash is unique in some internal sorting routine

After this is done we can start with the kernel code

@fgaudreault

This comment has been minimized.

Show comment
Hide comment
@fgaudreault

fgaudreault Apr 22, 2016

Contributor

We changed the mode to 13500, and added an esalt structure. I believe the buffers are populating the data according to your recommendations. So what are the next steps? :)

Contributor

fgaudreault commented Apr 22, 2016

We changed the mode to 13500, and added an esalt structure. I believe the buffers are populating the data according to your recommendations. So what are the next steps? :)

@jsteube

This comment has been minimized.

Show comment
Hide comment
@jsteube

jsteube Apr 22, 2016

Member

That looks much better. Following things are missing:

  • Code in ascii_digest(). This function is responsible for generating the output string. It's very important to generate 1:1 the same output string as the input string is.
  • Add unit testing code in test.pl and test.sh.
  • You use memcpy for the salt buffer, but you'd actually have to decode the hex data first. something like this:
    for (i = 0; i < salt_len / 2; i++)
    {
    out[i] = hex_to_u8 ((const u8 *) &salt_pos[i * 2]);
    }
  • You marked the hash with the option OPTS_TYPE_ST_ADD80 but using OPTS_TYPE_ST_ADD80 makes sense only for schemes like sha1($pass.$salt) where the $salt comes after the $pass. I think what you want to use is OPTS_TYPE_PT_ADD80. But note this will work on -a3 kernel only automatically. For -a0 and -a1 you have to do it in the kernel.
Member

jsteube commented Apr 22, 2016

That looks much better. Following things are missing:

  • Code in ascii_digest(). This function is responsible for generating the output string. It's very important to generate 1:1 the same output string as the input string is.
  • Add unit testing code in test.pl and test.sh.
  • You use memcpy for the salt buffer, but you'd actually have to decode the hex data first. something like this:
    for (i = 0; i < salt_len / 2; i++)
    {
    out[i] = hex_to_u8 ((const u8 *) &salt_pos[i * 2]);
    }
  • You marked the hash with the option OPTS_TYPE_ST_ADD80 but using OPTS_TYPE_ST_ADD80 makes sense only for schemes like sha1($pass.$salt) where the $salt comes after the $pass. I think what you want to use is OPTS_TYPE_PT_ADD80. But note this will work on -a3 kernel only automatically. For -a0 and -a1 you have to do it in the kernel.
@fgaudreault

This comment has been minimized.

Show comment
Hide comment
@fgaudreault

fgaudreault Apr 22, 2016

Contributor

The tests are still missing, as well as the appropriate kernel code. But for the rest, we should have the base in place. More tests will be required afterward.

Contributor

fgaudreault commented Apr 22, 2016

The tests are still missing, as well as the appropriate kernel code. But for the rest, we should have the base in place. More tests will be required afterward.

@jsteube

This comment has been minimized.

Show comment
Hide comment
@jsteube

jsteube Apr 24, 2016

Member

If you want I can merge and finish the kernel

Member

jsteube commented Apr 24, 2016

If you want I can merge and finish the kernel

@fgaudreault

This comment has been minimized.

Show comment
Hide comment
@fgaudreault

fgaudreault Apr 24, 2016

Contributor

I think that would be great. I'll just make sure we have the tests in place, and remove the warning on compiling. Should be done in the next commit. Meanwhile, I will also fix the test.pl file for the new Digest::ECB versions, the constants are no longer exported.

Contributor

fgaudreault commented Apr 24, 2016

I think that would be great. I'll just make sure we have the tests in place, and remove the warning on compiling. Should be done in the next commit. Meanwhile, I will also fix the test.pl file for the new Digest::ECB versions, the constants are no longer exported.

@jsteube

This comment has been minimized.

Show comment
Hide comment
@jsteube

jsteube Apr 24, 2016

Member

OK, waiting for your "go"

Member

jsteube commented Apr 24, 2016

OK, waiting for your "go"

@fgaudreault

This comment has been minimized.

Show comment
Hide comment
@fgaudreault

fgaudreault Apr 24, 2016

Contributor

I think we are good now. The test.pl results have been cross-checked using JtR, and it cracks properly. Unfortunately, I had to use a constant salt, but that should do the trick.

Contributor

fgaudreault commented Apr 24, 2016

I think we are good now. The test.pl results have been cross-checked using JtR, and it cracks properly. Unfortunately, I had to use a constant salt, but that should do the trick.

@fgaudreault fgaudreault changed the title from Adding parser and basic kernels for -m 134 to Adding parser and basic kernels for -m 13500 Apr 24, 2016

@jsteube

This comment has been minimized.

Show comment
Hide comment
@jsteube

jsteube Apr 25, 2016

Member

Please don't use * here

#define DISPLAY_LEN_MIN_13500 40 + 1 + 16 * 2
#define DISPLAY_LEN_MAX_13500 40 + 1 + 512 * 2

Simply do:

#define DISPLAY_LEN_MIN_13500 40 + 1 + 32
#define DISPLAY_LEN_MAX_13500 40 + 1 + 1024

Sorting of this:

13400,
13500,

Should be:

  133,
13500,

You're still using an invalid option for the salt which you need to remove:

| OPTS_TYPE_ST_ADDBITS15;

The following optimization is not possible since our data buffer is > length 55, you need to remove it:

| OPTI_TYPE_PRECOMPUTE_MERKLE

This code:

memset(pstoken_tmp, 0, mysalt_len + 1);

Is wrong in two ways. First you should use sizeof():

memset (pstoken_tmp, 0, sizeof (pstoken_tmp));

Another problem is that you're using a value which is unknown at compile time to declare a number of elements in an array.

Instead of:

u8 pstoken_tmp[mysalt_len + 1];

You should do:

u8 pstoken_tmp[512 + 1];

More problems with snprintf():

snprintf((char *)(pstoken_tmp + i), (size_t)2, "%02x", pstoken->salt_buf[i]);

Problem 1, the "%02x" does not match salt_buf[] which is 32 bit, not 8 bit. You propably need to something like this befoire you can use it:

u8 *salt_buf_ptr = (u8 *) pstoken->salt_buf:

... and then use salt_buf_ptr[i] in the snprintf()

Problem 2, the casting of pstoken_tmp from u8 * -> char * is not necessary.

Just declare pstoken_tmp as char *.

Suggestion:

The use of snprintf() is slow, take a look bin_to_hex_lower (). This is not always useful but sometimes it is.

Inside the parser you do:

u8 pstoken_tmp[DISPLAY_LEN_MAX_13500 - 40 - 1];

memset(pstoken_tmp, 0, DISPLAY_LEN_MAX_13500 - 40 - 1);

I'd recommend to use the same declaration as in ascii_digest instead:

u8 pstoken_tmp[512 + 1];

memset (pstoken_tmp, 0, sizeof (pstoken_tmp));

This optimization does not work here, because we need a 2nd call to sha1_transform because of the length > 55. You need to remove it:

digest[0] -= SHA1M_A;
digest[1] -= SHA1M_B;
digest[2] -= SHA1M_C;
digest[3] -= SHA1M_D;
digest[4] -= SHA1M_E;

That was easy stuff so fow, but now in test.pl it's getting real problematic. From your original issue you said:

Would love to see a new mode added for that. There is one for the application password, but if we could get a sha1($salt.utf16le($pass)) to break the SHA1 signature of the token, that would be great. The salt length is very long, 130-150bytes, so we can't really use any existing mode.

What you totaly did not mention is the use of DES inside the algorithm. Now I'm very confused. Please explain in words how DES is involved in the process. It's important to write a proper kernel.

Member

jsteube commented Apr 25, 2016

Please don't use * here

#define DISPLAY_LEN_MIN_13500 40 + 1 + 16 * 2
#define DISPLAY_LEN_MAX_13500 40 + 1 + 512 * 2

Simply do:

#define DISPLAY_LEN_MIN_13500 40 + 1 + 32
#define DISPLAY_LEN_MAX_13500 40 + 1 + 1024

Sorting of this:

13400,
13500,

Should be:

  133,
13500,

You're still using an invalid option for the salt which you need to remove:

| OPTS_TYPE_ST_ADDBITS15;

The following optimization is not possible since our data buffer is > length 55, you need to remove it:

| OPTI_TYPE_PRECOMPUTE_MERKLE

This code:

memset(pstoken_tmp, 0, mysalt_len + 1);

Is wrong in two ways. First you should use sizeof():

memset (pstoken_tmp, 0, sizeof (pstoken_tmp));

Another problem is that you're using a value which is unknown at compile time to declare a number of elements in an array.

Instead of:

u8 pstoken_tmp[mysalt_len + 1];

You should do:

u8 pstoken_tmp[512 + 1];

More problems with snprintf():

snprintf((char *)(pstoken_tmp + i), (size_t)2, "%02x", pstoken->salt_buf[i]);

Problem 1, the "%02x" does not match salt_buf[] which is 32 bit, not 8 bit. You propably need to something like this befoire you can use it:

u8 *salt_buf_ptr = (u8 *) pstoken->salt_buf:

... and then use salt_buf_ptr[i] in the snprintf()

Problem 2, the casting of pstoken_tmp from u8 * -> char * is not necessary.

Just declare pstoken_tmp as char *.

Suggestion:

The use of snprintf() is slow, take a look bin_to_hex_lower (). This is not always useful but sometimes it is.

Inside the parser you do:

u8 pstoken_tmp[DISPLAY_LEN_MAX_13500 - 40 - 1];

memset(pstoken_tmp, 0, DISPLAY_LEN_MAX_13500 - 40 - 1);

I'd recommend to use the same declaration as in ascii_digest instead:

u8 pstoken_tmp[512 + 1];

memset (pstoken_tmp, 0, sizeof (pstoken_tmp));

This optimization does not work here, because we need a 2nd call to sha1_transform because of the length > 55. You need to remove it:

digest[0] -= SHA1M_A;
digest[1] -= SHA1M_B;
digest[2] -= SHA1M_C;
digest[3] -= SHA1M_D;
digest[4] -= SHA1M_E;

That was easy stuff so fow, but now in test.pl it's getting real problematic. From your original issue you said:

Would love to see a new mode added for that. There is one for the application password, but if we could get a sha1($salt.utf16le($pass)) to break the SHA1 signature of the token, that would be great. The salt length is very long, 130-150bytes, so we can't really use any existing mode.

What you totaly did not mention is the use of DES inside the algorithm. Now I'm very confused. Please explain in words how DES is involved in the process. It's important to write a proper kernel.

@fgaudreault

This comment has been minimized.

Show comment
Hide comment
@fgaudreault

fgaudreault Apr 25, 2016

Contributor

There is no DES in the algorithm. I fixed the use Crypt::ECB module only because the newer version is no longer using PADDING_NONE and PADDING_AUTO, you have to push the padding in the encrypt sub. That being said, we will fix the problems.

Contributor

fgaudreault commented Apr 25, 2016

There is no DES in the algorithm. I fixed the use Crypt::ECB module only because the newer version is no longer using PADDING_NONE and PADDING_AUTO, you have to push the padding in the encrypt sub. That being said, we will fix the problems.

@fgaudreault

This comment has been minimized.

Show comment
Hide comment
@fgaudreault

fgaudreault Apr 25, 2016

Contributor

Aight. So at this point, we think we would need the kernels in order to test the full loop on our own and fix the remaining issues. Because it's the first time we touch the code base, having working kernels would help, and if you can help us with that, it would go much faster. Thanks for your patience btw ;)

Contributor

fgaudreault commented Apr 25, 2016

Aight. So at this point, we think we would need the kernels in order to test the full loop on our own and fix the remaining issues. Because it's the first time we touch the code base, having working kernels would help, and if you can help us with that, it would go much faster. Thanks for your patience btw ;)

@jsteube jsteube merged commit 9294aac into hashcat:master Apr 26, 2016

@jsteube

This comment has been minimized.

Show comment
Hide comment
@jsteube

jsteube Apr 26, 2016

Member

I'll now rewrite the kernel and inform you when I've pushed a working version

Member

jsteube commented Apr 26, 2016

I'll now rewrite the kernel and inform you when I've pushed a working version

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment