New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for cracking DPAPI masterkey files from XP to Win10 #2521

Merged
merged 1 commit into from Apr 25, 2017

Conversation

Projects
None yet
4 participants
@Fist0urs
Contributor

Fist0urs commented Apr 24, 2017

Summary

tl;dr: DPAPI is an API offered to Windows developpers by Microsoft to protect some data (password, certificates, data, etc.). DPAPI data protection relies on the currently logged on user's password. In order to encrypt/decrypt data, Windows derivates the user's password to unprotect "masterkey files", then extract a symetric master key from them and uses it to proceed further encryption/decryption.
DPAPI is massively used (Chrome, Skype, Dropbox, EFS, Credman, IE, etc.).

Cracking DPAPI masterkey file can be useful in different scenarii:

  • phishing attacks, where all you have is an authenticated access but not the actual credentials

  • if the workstation is harden and no mscash are stored, trying to break DPAPI masterkeys is an alternative (when a user interactively logs on, his roaming profil is created and his masterkeys are imported on the target workstation)

The support of this algorithm is splitted in 2 separate formats, dpapimkv1 and dpapimkv2 as pbkdf2 routines do not rely on the same hashing algorithm, thus a bit annoying for sse optimization.
Also provided is DPAPImk2john that extract the hash from the masterkey file.

Concerning the commit:

  • EFS support and efs2john.py were moved to unused/ as EFS relies on DPAPI
  • @kholia work was incorporated in dpapimkv1_fmt_plug.c and domain credentials case was added
  • dpapimkv2_fmt_plug.c was fully implemented
  • DPAPImk2john.py was added to run/
  • doc/DPAPImk-Auditing-HOWTO.md will be there soon

Finally, the format of input hash was defined for possible other algorithm.

This addresses #898

I'll implement these algorithms in hashcat as soon as I have time, using the same input hashing form.

Other Information

Tests have been done using ASAN, compiler was gcc version 6.3.0 and version 5.4.0

Concerning benchmarks:

./john --format=mscash2 --test && ./john --format=dpapimkv1 --test && ./john --format=dpapimkv2 --test
Will run 8 OpenMP threads
Benchmarking: mscash2, MS Cache Hash 2 (DCC2) [PBKDF2-SHA1 256/256 AVX2 8x]... (8xOMP) DONE
Warning: "Many salts" test limited: 18/256
Many salts:	9216 c/s real, 1178 c/s virtual
Only one salt:	8031 c/s real, 1104 c/s virtual

Will run 8 OpenMP threads
Benchmarking: DPAPImkv1, DPAPI masterkey file v1 [SHA1/NTLM PBKDF2-SHA1-DPAPI-variant 3DES 256/256 AVX2 8x]... (8xOMP) DONE
Speed for cost 1 (iteration count) of 24000
Raw:	1845 c/s real, 256 c/s virtual

Will run 8 OpenMP threads
Benchmarking: DPAPImkv2, DPAPI masterkey file v2 [SHA1/NTLM PBKDF2-SHA512-DPAPI-variant AES256 256/256 AVX2 8x]... (8xOMP) DONE
Speed for cost 1 (iteration count) of 8000 and 17000
Raw:	1044 c/s real, 141 c/s virtual

which is better than I thought regarding the high number of iterations (it can actually change from one operating system version to another and from a workstation to a server version).

Cheers!

@Fist0urs Fist0urs force-pushed the Fist0urs:DPAPI branch from b307b7f to 7dee89a Apr 24, 2017

@kholia

This comment has been minimized.

Collaborator

kholia commented Apr 24, 2017

Some feedback,

  1. There was a lot of useful testing code in efs2john.py which is gone now. That code was and is very useful for future maintenance, debugging and testing work. That code treated DPAPIck as the upstream project and made minimal changes to it (ensuring reuse and less future work for us). Can't we easily modify efs2john.py to output the new hash format(s)?

  2. dpapimkv1_fmt_plug.c and dpapimkv2_fmt_plug.c should be combined into a single file dpapimk_fmt_plug.c ideally. There is lot of common code between the two formats, so it makes sense to combine them. The combined format label could be DPAPImk. The combined format tag could be $DPAPImk$. This will reduce future maintenance work.

  3. There are some minor whitespace errors in the new files. Please remove them.

  4. How does the W = '\033[0m' color hack work on Windows? Do we really need fancy output?

  5. The first * character in $DPAPImkv2$*1*... is my mistake from the past. Instead the hash should look like $DPAPImk$1*....

  6. Please retain the old test vectors. The corresponding DPAPI MK blobs can be downloaded from http://openwall.info/wiki/john/sample-non-hashes page. It would be nice to annotate the new test vectors with some extra information, like Windows versions etc.

Overall, the new code is looking great 👍

@kholia

This comment has been minimized.

Collaborator

kholia commented Apr 24, 2017

  1. Please nuke the unused files instead of moving them to the unused folder.
@Fist0urs

This comment has been minimized.

Contributor

Fist0urs commented Apr 24, 2017

  1. Actually I have coded also a dpapi_decrypt.py and a dpapi_generate.py in order to generate multiple unit tests, that also contain debugging informations. That's why I'm not relying on "real" files (like the efs samples) neither on dpapick for the .py (I tried to implement the code as minimalistic as possible for other people if they want to debug). But if you really want it, yes I could modify efs2john.py to fit the format (and add the "Preferred" stuff from my actual DPAPImk2john)
  2. I thought about that, but I couldn't see an easy way to handle SSE cases because of SHA1 and SHA512 (MAX_KEYS_PER_CRYPT)
  3. Consider done
  4. I'll remove it
  5. Not exactly as I make a difference between local credentials (1) and domain credentials (2), I can add an extra field then (when 2. is done)
  6. As I told you by mail, I'm a bit annoyed by samples tests of type "EFS" as this encryption scheme is not specific to EFS only. Many softwares use DPAPI, EFS being one of them, so we should name it "DPAPI masterkeys" maybe? And there would lack test cases concerning domain credentials maybe?
  7. Will do it

Thanks for the review!

@kholia

This comment has been minimized.

Collaborator

kholia commented Apr 24, 2017

  1. It would be great if original efs2john.py functionality can be retained without too much effort. If you can add your new DPAPI blob generation and testing features to it, then that would be even better.

  2. Take a look at itunes_fmt_plug.c. It defines MAX_KEYS_PER_CRYPT as (SSE_GROUP_SZ_SHA1 * SSE_GROUP_SZ_SHA256). You will need to do something similar.

  3. 👍

  4. 👍

  5. $DPAPImkv2$*1*... should be $DPAPImkv2$1*.... No extra * character at the start (before the 1). Hope this is clear now.

  6. Supporting exisiting test vectors ensures that there are no regressions. You don't have to copy-paste and modify them by hand! Just use the original EFS DPAPI blobs from the http://openwall.info/wiki/john/sample-non-hashes page with the new hash extraction tool. Of course, feel free to add more test vectors of varied types.

#define OMP_SCALE 64
#endif
#endif
#include "memdbg.h"

This comment has been minimized.

@magnumripper

magnumripper Apr 24, 2017

Owner

nitpick: Please include all system headers, then a blank line, the all local ones.

This comment has been minimized.

@Fist0urs

Fist0urs Apr 24, 2017

Contributor

👍

#ifdef SIMD_COEF_32
#define ALGORITHM_NAME "SHA1/NTLM PBKDF2-SHA1-DPAPI-variant 3DES " SHA1_ALGORITHM_NAME
#else
#define ALGORITHM_NAME "SHA1/NTLM PBKDF2-SHA1-DPAPI-variant 3DES 32/" ARCH_BITS_STR

This comment has been minimized.

@magnumripper

magnumripper Apr 24, 2017

Owner

We're supposed to list primitives here, so I believe NTLM should be replaced with MD4.

This comment has been minimized.

@magnumripper

magnumripper Apr 24, 2017

Owner

Also, what exactly does "-DPAPI-variant" mean here? It seems to me a perfectly normal PBKDF2-HMAC-SHA1 is one of the primitives, perhaps a correct string would be just "SHA1/MD4 PBKDF2-SHA1 3DES" (plus the arch stuff)?

This comment has been minimized.

@Fist0urs

Fist0urs Apr 24, 2017

Contributor

You're right concerning the NTLM part, I'll change that

Concerning the "-DPAPI-variant", actually there is a tweak in MS implementation of PBKDF2 (instead of re-using the same salt during the XORing scheme in the loop as in the RFC, they use the result of the XORing of each iteration... Dunno if it is on purpose or just a misunderstanding of them)

This comment has been minimized.

@magnumripper

magnumripper Apr 24, 2017

Owner

Oh, I think I see now: That's what happens when DPAPI_CRAP_LOGIC is defined, right? In that case, "PBKDF2-SHA1-DPAPI-variant" is probably the better description.

This comment has been minimized.

@Fist0urs

Fist0urs Apr 24, 2017

Contributor

Yes exactly!

{
/* Convert key to UTF-16LE (--encoding aware) */
enc_to_utf16(saved_key[index], PLAINTEXT_LENGTH, (UTF8*)key, strlen(key));
}

This comment has been minimized.

@magnumripper

magnumripper Apr 24, 2017

Owner

I'm delighted to see you implement proper encoding support 👍

This comment has been minimized.

@Fist0urs

Fist0urs Apr 24, 2017

Contributor

Haha, actually all the credit goes to @kholia as he did this 👍

This comment has been minimized.

@magnumripper

magnumripper Apr 24, 2017

Owner

Oh, so the credit bounces back to me because I brute forced that into his head over several years 😆

This comment has been minimized.

@Fist0urs

Fist0urs Apr 24, 2017

Contributor

Oh, I see what you did there: smart move to get credits back 😆 😜

@Fist0urs

This comment has been minimized.

Contributor

Fist0urs commented Apr 24, 2017

  1. Okay, let's do it this way then: I'll add the extra features my implementation provides, including the "Preferred" stuff, and the generation of samples hashes for every possible format.
    I'll also adapt the hash format extraction to fit my implementation
  2. Great thanks 👍
  3. 👍
  4. 👍
@@ -108,8 +108,12 @@ static void _pbkdf2_sha512(const unsigned char *S, int SL, int R, uint64_t *out,
SHA512_Update(&ctx, tmp_hash, SHA512_DIGEST_LENGTH);
SHA512_Final(tmp_hash, &ctx);

for (j = 0; j < SHA512_DIGEST_LENGTH/sizeof(uint64_t); j++)
for (j = 0; j < SHA512_DIGEST_LENGTH/sizeof(uint64_t); j++){

This comment has been minimized.

@magnumripper

magnumripper Apr 24, 2017

Owner

Please add one space before the curly bracket (twice in this file)

@Fist0urs

This comment has been minimized.

Contributor

Fist0urs commented Apr 24, 2017

Ok, now the main remarks should be addressed. Please check, I may have forgotten some things.

Remaining before merge:

  • Adapt efs2john.py so that it includes all features of DPAPImk2john.py
  • Adapt output hash format of efs2john.py
  • Rename efs2john.py to DPAPImk2john.py
  • Delete efs2john.py
  • Provide extra samples (masterkey files) to http://openwall.info/wiki/john/sample-non-hashes (may need a bit of time, as I need to have masterkeys of computers belonging to a domain and from various operating systems)
  • Write doc/DPAPImk-Auditing-HOWTO.md

Am I missing something?

@Fist0urs Fist0urs force-pushed the Fist0urs:DPAPI branch from 79b6b46 to 9ec90bd Apr 24, 2017


#include <string.h>
#include <assert.h>
#include <errno.h>

This comment has been minimized.

@kholia

kholia Apr 25, 2017

Collaborator

Both <assert.h> and <errno.h> includes can be removed.

#define KEY_LEN1 24
#define IV_LEN1 8
#define DIGEST_LEN1 20

This comment has been minimized.

@kholia

kholia Apr 25, 2017

Collaborator

Nitpick: The values of these define statements can be aligned to match the existing alignment. It might look better :-)

};



This comment has been minimized.

@kholia

kholia Apr 25, 2017

Collaborator

Tighten line spacing (i.e. remove extra newlines).

static char *ptrSID;

memset(&cs, 0, sizeof(cs));
ctcopy += FORMAT_TAG_LEN; /* skip over "$DPAPImkv2$*" */

This comment has been minimized.

@kholia

kholia Apr 25, 2017

Collaborator

Comment needs to be updated to reflect the new format tag.

return (void *)&cs;
}


This comment has been minimized.

@kholia

kholia Apr 25, 2017

Collaborator

Tighten line spacing (i.e. remove extra newlines).

SHA_CTX ctx;
MD4_CTX ctx2;

int i;

This comment has been minimized.

@kholia

kholia Apr 25, 2017

Collaborator

Tighten line spacing. Various variable declarations can be clubbed together without newlines.

This comment has been minimized.

@Fist0urs

Fist0urs Apr 25, 2017

Contributor

All done and merged in previous commit 👍

@Fist0urs Fist0urs force-pushed the Fist0urs:DPAPI branch from 9ec90bd to 6353410 Apr 25, 2017

@Fist0urs

This comment has been minimized.

Contributor

Fist0urs commented Apr 25, 2017

Ok, now everything should be ready for merge, can you take a look please?

Remaining:

  • Provide extra samples (masterkey files) to http://openwall.info/wiki/john/sample-non-hashes (may need a bit of time, as I need to have masterkeys of computers belonging to a domain and from various operating systems)
  • Write doc/DPAPImk-Auditing-HOWTO.md

Let me know when it is okay for you, I'll merge all commits in the first one in order to keep history clean.

@kholia

This comment has been minimized.

Collaborator

kholia commented Apr 25, 2017

@Fist0urs This looks ready to merge now. Go ahead with the squashing process. Once done, I will happily merge this PR 👍

@kholia

This comment has been minimized.

Collaborator

kholia commented Apr 25, 2017

The wiki and documentation stuff can be done afterwards for sure.

@kholia

kholia approved these changes Apr 25, 2017

@Fist0urs Fist0urs force-pushed the Fist0urs:DPAPI branch from a885920 to 4be0413 Apr 25, 2017

@Fist0urs

This comment has been minimized.

Contributor

Fist0urs commented Apr 25, 2017

Squash done, thank you very much for your time guys! 👍

@kholia kholia merged commit 58b4eb4 into magnumripper:bleeding-jumbo Apr 25, 2017

1 check passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details
@kholia

This comment has been minimized.

Collaborator

kholia commented Apr 25, 2017

Thanks 👍

@frank-dittrich

This comment has been minimized.

Collaborator

frank-dittrich commented Sep 10, 2017

Unfortunately, this broke backwards compatibility to 1.8.0-jumbo-1 release.

Looks like --format=efs is gone, and hashes like these are not considered valid anymore:

/* Windows XP, openwall.efs */
{"$efs$0$S-1-5-21-1482476501-1659004503-725345543-1003$b3d62a0b06cecc236fe3200460426a13$4000$d3841257348221cd92caf4427a59d785ed1474cab3d0101fc8d37137dbb598ff1fd2455826128b2594b846934c073528f8648d750d3c8e6621e6f706d79b18c22f172c0930d9a934de73ea2eb63b7b44810d332f7d03f14d1c153de16070a5cab9324da87405c1c0", "openwall"},
/* Windows XP, openwall.efs.2 */
{"$efs$0$S-1-5-21-1482476501-1659004503-725345543-1005$c9cbd491f78ea6d512276b33f025bce8$4000$091a13443cfc2ddb16dcf256ab2a6707a27aa22b49a9a9011ebf3bb778d0088c2896de31de67241d91df75306e56f835337c89cfb2f9afa940b4e7e019ead2737145032fac0bb34587a707d42da7e00b72601a730f5c848094d54c47c622e2f8c8d204c80ad061be", "JtRisthebest"},

Is it possible to consider the old hashes as valid, and just convert the hashes in split()?

Do we need a new document describing format name changes, and how to convert the old hashes?

Are there other formats affected as well? I didn't really check.

@Fist0urs

This comment has been minimized.

Contributor

Fist0urs commented Sep 16, 2017

Hi @frank-dittrich,

I'll add the backward compatibility in order to support older versions of this hashformat.

Cheers!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment