Support for cracking DPAPI masterkey files from XP to Win10 #2521

Fist0urs · 2017-04-24T14:13:14Z

Summary

tl;dr: DPAPI is an API offered to Windows developpers by Microsoft to protect some data (password, certificates, data, etc.). DPAPI data protection relies on the currently logged on user's password. In order to encrypt/decrypt data, Windows derivates the user's password to unprotect "masterkey files", then extract a symetric master key from them and uses it to proceed further encryption/decryption.
DPAPI is massively used (Chrome, Skype, Dropbox, EFS, Credman, IE, etc.).

Cracking DPAPI masterkey file can be useful in different scenarii:

phishing attacks, where all you have is an authenticated access but not the actual credentials
if the workstation is harden and no mscash are stored, trying to break DPAPI masterkeys is an alternative (when a user interactively logs on, his roaming profil is created and his masterkeys are imported on the target workstation)

The support of this algorithm is splitted in 2 separate formats, dpapimkv1 and dpapimkv2 as pbkdf2 routines do not rely on the same hashing algorithm, thus a bit annoying for sse optimization.
Also provided is DPAPImk2john that extract the hash from the masterkey file.

Concerning the commit:

EFS support and efs2john.py were moved to unused/ as EFS relies on DPAPI
@kholia work was incorporated in dpapimkv1_fmt_plug.c and domain credentials case was added
dpapimkv2_fmt_plug.c was fully implemented
DPAPImk2john.py was added to run/
doc/DPAPImk-Auditing-HOWTO.md will be there soon

Finally, the format of input hash was defined for possible other algorithm.

This addresses #898

I'll implement these algorithms in hashcat as soon as I have time, using the same input hashing form.

Other Information

Tests have been done using ASAN, compiler was gcc version 6.3.0 and version 5.4.0

Concerning benchmarks:

./john --format=mscash2 --test && ./john --format=dpapimkv1 --test && ./john --format=dpapimkv2 --test
Will run 8 OpenMP threads
Benchmarking: mscash2, MS Cache Hash 2 (DCC2) [PBKDF2-SHA1 256/256 AVX2 8x]... (8xOMP) DONE
Warning: "Many salts" test limited: 18/256
Many salts:	9216 c/s real, 1178 c/s virtual
Only one salt:	8031 c/s real, 1104 c/s virtual

Will run 8 OpenMP threads
Benchmarking: DPAPImkv1, DPAPI masterkey file v1 [SHA1/NTLM PBKDF2-SHA1-DPAPI-variant 3DES 256/256 AVX2 8x]... (8xOMP) DONE
Speed for cost 1 (iteration count) of 24000
Raw:	1845 c/s real, 256 c/s virtual

Will run 8 OpenMP threads
Benchmarking: DPAPImkv2, DPAPI masterkey file v2 [SHA1/NTLM PBKDF2-SHA512-DPAPI-variant AES256 256/256 AVX2 8x]... (8xOMP) DONE
Speed for cost 1 (iteration count) of 8000 and 17000
Raw:	1044 c/s real, 141 c/s virtual

which is better than I thought regarding the high number of iterations (it can actually change from one operating system version to another and from a workstation to a server version).

Cheers!

kholia · 2017-04-24T15:24:26Z

Some feedback,

There was a lot of useful testing code in efs2john.py which is gone now. That code was and is very useful for future maintenance, debugging and testing work. That code treated DPAPIck as the upstream project and made minimal changes to it (ensuring reuse and less future work for us). Can't we easily modify efs2john.py to output the new hash format(s)?
dpapimkv1_fmt_plug.c and dpapimkv2_fmt_plug.c should be combined into a single file dpapimk_fmt_plug.c ideally. There is lot of common code between the two formats, so it makes sense to combine them. The combined format label could be DPAPImk. The combined format tag could be $DPAPImk$ . This will reduce future maintenance work.
There are some minor whitespace errors in the new files. Please remove them.
How does the W = '\033[0m' color hack work on Windows? Do we really need fancy output?
The first * character in $DPAPImkv2$*1*... is my mistake from the past. Instead the hash should look like $DPAPImk$1*....
Please retain the old test vectors. The corresponding DPAPI MK blobs can be downloaded from http://openwall.info/wiki/john/sample-non-hashes page. It would be nice to annotate the new test vectors with some extra information, like Windows versions etc.

Overall, the new code is looking great 👍

kholia · 2017-04-24T15:25:37Z

Please nuke the unused files instead of moving them to the unused folder.

Fist0urs · 2017-04-24T15:39:38Z

Actually I have coded also a dpapi_decrypt.py and a dpapi_generate.py in order to generate multiple unit tests, that also contain debugging informations. That's why I'm not relying on "real" files (like the efs samples) neither on dpapick for the .py (I tried to implement the code as minimalistic as possible for other people if they want to debug). But if you really want it, yes I could modify efs2john.py to fit the format (and add the "Preferred" stuff from my actual DPAPImk2john)
I thought about that, but I couldn't see an easy way to handle SSE cases because of SHA1 and SHA512 (MAX_KEYS_PER_CRYPT)
Consider done
I'll remove it
Not exactly as I make a difference between local credentials (1) and domain credentials (2), I can add an extra field then (when 2. is done)
As I told you by mail, I'm a bit annoyed by samples tests of type "EFS" as this encryption scheme is not specific to EFS only. Many softwares use DPAPI, EFS being one of them, so we should name it "DPAPI masterkeys" maybe? And there would lack test cases concerning domain credentials maybe?
Will do it

Thanks for the review!

kholia · 2017-04-24T16:45:38Z

It would be great if original efs2john.py functionality can be retained without too much effort. If you can add your new DPAPI blob generation and testing features to it, then that would be even better.
Take a look at itunes_fmt_plug.c. It defines MAX_KEYS_PER_CRYPT as (SSE_GROUP_SZ_SHA1 * SSE_GROUP_SZ_SHA256). You will need to do something similar.
👍
👍
$DPAPImkv2$*1*... should be $DPAPImkv2$1*.... No extra * character at the start (before the 1). Hope this is clear now.
Supporting exisiting test vectors ensures that there are no regressions. You don't have to copy-paste and modify them by hand! Just use the original EFS DPAPI blobs from the http://openwall.info/wiki/john/sample-non-hashes page with the new hash extraction tool. Of course, feel free to add more test vectors of varied types.

magnumripper

Dry review

magnumripper · 2017-04-24T19:20:24Z

src/dpapimkv1_fmt_plug.c

+#define OMP_SCALE               64
+#endif
+#endif
+#include "memdbg.h"


nitpick: Please include all system headers, then a blank line, the all local ones.

magnumripper · 2017-04-24T19:21:45Z

src/dpapimkv1_fmt_plug.c

+#ifdef SIMD_COEF_32
+#define ALGORITHM_NAME          "SHA1/NTLM PBKDF2-SHA1-DPAPI-variant 3DES " SHA1_ALGORITHM_NAME
+#else
+#define ALGORITHM_NAME          "SHA1/NTLM PBKDF2-SHA1-DPAPI-variant 3DES 32/" ARCH_BITS_STR


We're supposed to list primitives here, so I believe NTLM should be replaced with MD4.

Also, what exactly does "-DPAPI-variant" mean here? It seems to me a perfectly normal PBKDF2-HMAC-SHA1 is one of the primitives, perhaps a correct string would be just "SHA1/MD4 PBKDF2-SHA1 3DES" (plus the arch stuff)?

You're right concerning the NTLM part, I'll change that

Concerning the "-DPAPI-variant", actually there is a tweak in MS implementation of PBKDF2 (instead of re-using the same salt during the XORing scheme in the loop as in the RFC, they use the result of the XORing of each iteration... Dunno if it is on purpose or just a misunderstanding of them)

Oh, I think I see now: That's what happens when DPAPI_CRAP_LOGIC is defined, right? In that case, "PBKDF2-SHA1-DPAPI-variant" is probably the better description.

Yes exactly!

magnumripper · 2017-04-24T19:30:11Z

src/dpapimkv1_fmt_plug.c

+{
+	/* Convert key to UTF-16LE (--encoding aware) */
+	enc_to_utf16(saved_key[index], PLAINTEXT_LENGTH, (UTF8*)key, strlen(key));
+}


I'm delighted to see you implement proper encoding support 👍

Haha, actually all the credit goes to @kholia as he did this 👍

Oh, so the credit bounces back to me because I brute forced that into his head over several years 😆

Oh, I see what you did there: smart move to get credits back 😆 😜

Fist0urs · 2017-04-24T19:47:52Z

Okay, let's do it this way then: I'll add the extra features my implementation provides, including the "Preferred" stuff, and the generation of samples hashes for every possible format.
I'll also adapt the hash format extraction to fit my implementation
Great thanks 👍
👍
👍

magnumripper · 2017-04-24T21:30:45Z

src/pbkdf2_hmac_sha512.h

@@ -108,8 +108,12 @@ static void _pbkdf2_sha512(const unsigned char *S, int SL, int R, uint64_t *out,
 		SHA512_Update(&ctx, tmp_hash, SHA512_DIGEST_LENGTH);
 		SHA512_Final(tmp_hash, &ctx);

-		for (j = 0; j < SHA512_DIGEST_LENGTH/sizeof(uint64_t); j++)
+		for (j = 0; j < SHA512_DIGEST_LENGTH/sizeof(uint64_t); j++){


Please add one space before the curly bracket (twice in this file)

Fist0urs · 2017-04-24T23:20:42Z

Ok, now the main remarks should be addressed. Please check, I may have forgotten some things.

Remaining before merge:

Adapt efs2john.py so that it includes all features of DPAPImk2john.py
Adapt output hash format of efs2john.py
Rename efs2john.py to DPAPImk2john.py
Delete efs2john.py
Provide extra samples (masterkey files) to http://openwall.info/wiki/john/sample-non-hashes (may need a bit of time, as I need to have masterkeys of computers belonging to a domain and from various operating systems)
Write doc/DPAPImk-Auditing-HOWTO.md

Am I missing something?

kholia · 2017-04-25T09:23:46Z

src/dpapimk_fmt_plug.c

+
+#include <string.h>
+#include <assert.h>
+#include <errno.h>


Both <assert.h> and <errno.h> includes can be removed.

kholia · 2017-04-25T09:27:59Z

src/dpapimk_fmt_plug.c

+#define KEY_LEN1 24
+#define IV_LEN1 8
+#define DIGEST_LEN1 20
+


Nitpick: The values of these define statements can be aligned to match the existing alignment. It might look better :-)

kholia · 2017-04-25T09:28:36Z

src/dpapimk_fmt_plug.c

+};
+
+
+


Tighten line spacing (i.e. remove extra newlines).

kholia · 2017-04-25T09:30:00Z

src/dpapimk_fmt_plug.c

+	static char *ptrSID;
+
+	memset(&cs, 0, sizeof(cs));
+	ctcopy += FORMAT_TAG_LEN;     /* skip over "$DPAPImkv2$*" */


Comment needs to be updated to reflect the new format tag.

kholia · 2017-04-25T09:30:30Z

src/dpapimk_fmt_plug.c

+	return (void *)&cs;
+}
+
+


Tighten line spacing (i.e. remove extra newlines).

kholia · 2017-04-25T09:43:01Z

src/dpapimk_fmt_plug.c

+		SHA_CTX ctx;
+		MD4_CTX ctx2;
+
+		int i;


Tighten line spacing. Various variable declarations can be clubbed together without newlines.

All done and merged in previous commit 👍

Fist0urs · 2017-04-25T13:57:40Z

Ok, now everything should be ready for merge, can you take a look please?

Remaining:

Provide extra samples (masterkey files) to http://openwall.info/wiki/john/sample-non-hashes (may need a bit of time, as I need to have masterkeys of computers belonging to a domain and from various operating systems)
Write doc/DPAPImk-Auditing-HOWTO.md

Let me know when it is okay for you, I'll merge all commits in the first one in order to keep history clean.

kholia · 2017-04-25T14:11:01Z

@Fist0urs This looks ready to merge now. Go ahead with the squashing process. Once done, I will happily merge this PR 👍

kholia · 2017-04-25T14:11:55Z

The wiki and documentation stuff can be done afterwards for sure.

…P to Win10

Fist0urs · 2017-04-25T14:26:41Z

Squash done, thank you very much for your time guys! 👍

kholia · 2017-04-25T15:47:25Z

Thanks 👍

frank-dittrich · 2017-09-10T13:26:51Z

Unfortunately, this broke backwards compatibility to 1.8.0-jumbo-1 release.

Looks like --format=efs is gone, and hashes like these are not considered valid anymore:

/* Windows XP, openwall.efs */
{"$efs$0$S-1-5-21-1482476501-1659004503-725345543-1003$b3d62a0b06cecc236fe3200460426a13$4000$d3841257348221cd92caf4427a59d785ed1474cab3d0101fc8d37137dbb598ff1fd2455826128b2594b846934c073528f8648d750d3c8e6621e6f706d79b18c22f172c0930d9a934de73ea2eb63b7b44810d332f7d03f14d1c153de16070a5cab9324da87405c1c0", "openwall"},
/* Windows XP, openwall.efs.2 */
{"$efs$0$S-1-5-21-1482476501-1659004503-725345543-1005$c9cbd491f78ea6d512276b33f025bce8$4000$091a13443cfc2ddb16dcf256ab2a6707a27aa22b49a9a9011ebf3bb778d0088c2896de31de67241d91df75306e56f835337c89cfb2f9afa940b4e7e019ead2737145032fac0bb34587a707d42da7e00b72601a730f5c848094d54c47c622e2f8c8d204c80ad061be", "JtRisthebest"},

Is it possible to consider the old hashes as valid, and just convert the hashes in split()?

Do we need a new document describing format name changes, and how to convert the old hashes?

Are there other formats affected as well? I didn't really check.

Fist0urs · 2017-09-16T15:35:42Z

Hi @frank-dittrich,

I'll add the backward compatibility in order to support older versions of this hashformat.

Cheers!

magnumripper reviewed Apr 24, 2017

View reviewed changes

kholia requested changes Apr 25, 2017

View reviewed changes

kholia reviewed Apr 25, 2017

View reviewed changes

kholia mentioned this pull request Apr 25, 2017

Fix EFS format to support Windows 7 and other operating systems #898

Closed

kholia approved these changes Apr 25, 2017

View reviewed changes

Add hash extractor + support for cracking DPAPI masterkey file from X…

4be0413

…P to Win10

kholia merged commit 58b4eb4 into openwall:bleeding-jumbo Apr 25, 2017

Fist0urs deleted the DPAPI branch April 25, 2017 16:31

Fist0urs mentioned this pull request May 8, 2017

Add new format -m 15300 Dpapi master key file version 1 and version 2 hashcat/hashcat#1238

Merged

Fist0urs mentioned this pull request May 20, 2017

Add feature to crack .kdbx with keyfile & Windows User Account #2555

Open

magnumripper mentioned this pull request May 31, 2017

Add GPU support for DPAPI format #2563

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support for cracking DPAPI masterkey files from XP to Win10 #2521

Support for cracking DPAPI masterkey files from XP to Win10 #2521

Fist0urs commented Apr 24, 2017 •

edited

Loading

kholia commented Apr 24, 2017

kholia commented Apr 24, 2017

Fist0urs commented Apr 24, 2017 •

edited

Loading

kholia commented Apr 24, 2017

magnumripper left a comment

magnumripper Apr 24, 2017

Fist0urs Apr 24, 2017

magnumripper Apr 24, 2017

magnumripper Apr 24, 2017

Fist0urs Apr 24, 2017

magnumripper Apr 24, 2017 •

edited

Loading

Fist0urs Apr 24, 2017

magnumripper Apr 24, 2017

Fist0urs Apr 24, 2017

magnumripper Apr 24, 2017

Fist0urs Apr 24, 2017

Fist0urs commented Apr 24, 2017

magnumripper Apr 24, 2017

Fist0urs commented Apr 24, 2017

kholia Apr 25, 2017

kholia Apr 25, 2017

kholia Apr 25, 2017

kholia Apr 25, 2017

kholia Apr 25, 2017

kholia Apr 25, 2017

Fist0urs Apr 25, 2017

Fist0urs commented Apr 25, 2017

kholia commented Apr 25, 2017

kholia commented Apr 25, 2017

Fist0urs commented Apr 25, 2017

kholia commented Apr 25, 2017

frank-dittrich commented Sep 10, 2017

Fist0urs commented Sep 16, 2017

Support for cracking DPAPI masterkey files from XP to Win10 #2521

Support for cracking DPAPI masterkey files from XP to Win10 #2521

Conversation

Fist0urs commented Apr 24, 2017 • edited Loading

Summary

Other Information

kholia commented Apr 24, 2017

kholia commented Apr 24, 2017

Fist0urs commented Apr 24, 2017 • edited Loading

kholia commented Apr 24, 2017

magnumripper left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

magnumripper Apr 24, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Fist0urs commented Apr 24, 2017

Choose a reason for hiding this comment

Fist0urs commented Apr 24, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Fist0urs commented Apr 25, 2017

kholia commented Apr 25, 2017

kholia commented Apr 25, 2017

Fist0urs commented Apr 25, 2017

kholia commented Apr 25, 2017

frank-dittrich commented Sep 10, 2017

Fist0urs commented Sep 16, 2017

Fist0urs commented Apr 24, 2017 •

edited

Loading

Fist0urs commented Apr 24, 2017 •

edited

Loading

magnumripper Apr 24, 2017 •

edited

Loading