Ethereum KDF - Scrypt

[Chick3nman]

New algorithm: Ethereum Scrypt

Use: Currently in use by Ethereum wallets to secure the wallet encryption password.

The algorithm design is explained very well here: https://github.com/ethereum/wiki/wiki/Web3-Secret-Storage-Definition

The page does not go quite into the same detail regarding the Scrypt variant, however it is almost identical to the PBKDF2 variant, with Scrypt in place for the initial password hashing.

To summarize, the password is hashed with Scrypt, then the "second leftmost 16 bytes" of the derived hash are concatenated with the ciphertext, which is then hashed with SHA3-256. The SHA3-256 hash is then directly compared to the "mac" value to verify whether or not the password is correct. If the SHA3 hashes match, the password is correct and the decryption of the ciphertext/wallet proceeds.

The page linked above includes this example of a valid wallet:password pair:

Test Vectors

Details:

• Address: 008aeeda4d805471df9b2a5b0f38a0c3bcba786b
• ICAP: XE542A5PZHH8PYIZUBEJEO0MFWRAPPIL67
• UUID: 3198bc9c-6672-5ab3-d9954942343ae5b6
• Password: testpassword
• Secret: 7a28b5ba57c53603b0b07b56bba752f7784bf506fa95edc395f5cf6c7514fe9d

Scrypt

Test vector using AES-128-CTR and Scrypt:

{
    "crypto" : {
        "cipher" : "aes-128-ctr",
        "cipherparams" : {
            "iv" : "83dbcc02d8ccb40e466191a123791e0e"
        },
        "ciphertext" : "d172bf743a674da9cdad04534d56926ef8358534d458fffccd4e6ad2fbde479c",
        "kdf" : "scrypt",
        "kdfparams" : {
            "dklen" : 32,
            "n" : 262144,
            "r" : 1,
            "p" : 8,
            "salt" : "ab0c7876052600dd703518d6fc3fe8984592145b591fc8fb5c6d43190334ba19"
        },
        "mac" : "2103ac29920d71da29f15d75b4a16dbe95cfd7ff8faea1056c33131d846e3097"
    },
    "id" : "3198bc9c-6672-5ab3-d995-4942343ae5b6",
    "version" : 3
}

Intermediates:

• Derived key: fac192ceb5fd772906bea3e118a69e8bbb5cc24229e20d8766fd298291bba6bd
• MAC Body bb5cc24229e20d8766fd298291bba6bdd172bf743a674da9cdad04534d56926ef8358534d458fffccd4e6ad2fbde479c
• MAC: 2103ac29920d71da29f15d75b4a16dbe95cfd7ff8faea1056c33131d846e3097
• Cipher key: fac192ceb5fd772906bea3e118a69e8b

I have written ether2hashcat to extract the relevant information from the json blobs and present it in a proposed hash format for hashcat. The proposed format would be as follows:

$ethereum$s*n*r*p*salt*mac*ciphertext

Using the example data from the test wallet above, we would get the following hash:

$ethereum$s*262144*1*8*ab0c7876052600dd703518d6fc3fe8984592145b591fc8fb5c6d43190334ba19*2103ac29920d71da29f15d75b4a16dbe95cfd7ff8faea1056c33131d846e3097*d172bf743a674da9cdad04534d56926ef8358534d458fffccd4e6ad2fbde479c

This would be cracked as testpassword

Because there are multiple KDF variations, one using PBKDF2 and one using Scrypt, the format includes a P or S to denote the different versions. The PBKDF2 variant is addressed in a separate issue, as it is technically a different algorithm.

Both algorithms were previously discussed on this forum thread and example code for cracking both algorithms was written by @philsmd

This was previously requested in issue #262, however that issue/request did not seem to be completed with the required information.

The sister algorithm request for the PBKDF2 variant is issue #1227

[ethtester]

Hello Chick3nman, does this also work with Mist created keystores?

[Chick3nman]

@ethtester from what I can find regarding MIST wallets, the keystore seemingly shares the same format as Geth wallets and therefore both this algorithm as well as my extract script should work on MIST wallets as well. If you have a MIST wallet on hand that I could look at that would be helpful as I do not have one and can't seem to confirm 100% what the format of the JSON is and if it's compatible with ether2hashcat.

[kholia]

@Chick3nman

The first asterisk character in the output hash $ether$*s*262144*... is a mistake and should be removed. This historical mistake started with my pdf2john output hash format. Instead the output hash should be $ether$s*262144*... (no starting stray asterisk character).

I would like to use the full name ethereum instead of ether in the output hash format. I don't see any advantages of using the potentially ambiguous short name.

[ethtester]

Hello Chick3nman,
Here are a couple of test wallets generated from Mist 0.3.9 and myetherwallet. I'm currently using a password cracker called ethcracker. https://github.com/lexansoft/ethcracker This tool can only test 2 passwords per second on Mist created wallets, but will do about 180 plus passwords per second on myetherwallet.com created wallets. I think it has some to do with the "n" iterations value being much higher in Mist. Would be great if we could leverage hashcat to speed things up. Thank You.

Test wallet passwords = password123
testkeys.zip

[Chick3nman]

@kholia I was simply following the format of other extracted hashes however I do agree with the change. I will edit my extract script and both issues to reflect the corrected format.

@ethtester those wallets look directly compatible as they follow the same format.

[kholia]

@Chick3nman Thank you! My hash extraction script can be seen at openwall/john#2525.

[Chick3nman]

@kholia awesome, would you like me to push yours as the main extract script instead of https://github.com/Chick3nman/ether2hashcat.py/blob/master/ether2hashcat.py ? Mine is a lot less clean than yours, plus no error handling.

[kholia]

@Chick3nman Sure. I think that having one "standard" extraction script is a better option overall. Thanks.

[Chick3nman]

Perfect, I'll edit both issues to refer to your script and will use it as the extract script from now on

[ethtester]

Hello Chick3nman,
Correct me if I'm wrong, hashcat will require an update to add a new hash mode before we can use the hash format you listed above, right?

[Chick3nman]

@ethtester Correct, hashcat will need to have the modes written and added to use them, this is simply a request to have them added.

[ethtester]

How long does it usually take for new modes to be coded after the initial request?

[Chick3nman]

@ethtester the developers are quite busy right now dealing with other, more important things. If no one from the community steps in to add this, it could be a few weeks before anyone can get to them. For now, @kholia's addition to JTR has been merged here: openwall/john#2525

[jsteube]

I was playing a bit with the data you provided. Some notes:

• The AES part is not needed at all, neither in PBKDF2 nor in SCRYPT mode
• We will need to rewrite the keccak GPU implementation to get this to work
• GPU's will not be able to crack this because of the memory requirement. Math behind: Mem per candidate: size_scrypt = (128 * scrypt_r) * scrypt_N = 256MB. But to make use of the GPU power, we need to spawn at least 1280 parallel computations, so we end up with a memory requirement of 320GB ram per GPU. It would require a minimum of a 256 divisions TMTO to get it working on a GTX1080, which drops the performance < 1 H/s.
• CPU cracking will run fine, tho. By assuming the keccak part will be so fast that it doesn't change the cracking speed, the only relevant slowdown is the SCRYPT. I've tested this to run to approx. 12-13 H/s on my 4770k. That is ~20% faster than JtRs native SCRYPT CPU code.

[ethtester]

@jsteube Thanks for the update. Is that 12-13H/s achieved by using all 4 cores of the 4770k?

[jsteube]

Yes Am 03.06.2017 4:20 nachm. schrieb "ethtester" <notifications@github.com>:

…

@jsteube <https://github.com/jsteube> Thanks for the update. Is that 12-13H/s achieved by using all 4 cores of the 4770k? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1228 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AJ83OdqizQ6yKreTJ0RjdC9QKUTPlBYiks5sAWuxgaJpZM4M-v3Y> .

[ethtester]

If one were to build a rig with 320GB of ram per GPU, would this negate the need for 256 divisions TMTO?

[jsteube]

Yes

[solardiz]

I'm sorry for commenting out of context, but this issue was referenced on a JtR issue, and this caught my attention:

approx. 12-13 H/s on my 4770k. That is ~20% faster than JtRs native SCRYPT CPU code.

What scrypt settings are these? JtR on i7-4770K (stock clocks) gives me 17 c/s with:

Loaded 1 password hash (scrypt [Salsa20/8 128/128 AVX])
Cost 1 (N) is 262144 for all loaded hashes
Cost 2 (r) is 8 for all loaded hashes
Cost 3 (p) is 1 for all loaded hashes
Will run 8 OpenMP threads

Is it the same here? Or are we talking 256 MiB with some other combination of N and r?

That's without use of huge pages. We got to update JtR's code to newer yescrypt tree, which will use huge pages for scrypt too (after some threshold, which I had previously tuned at 12 MiB, so it surely would be reached here). This should provide some speedup (above 17 c/s), provided that huge pages are allocated (with sysctl).

[solardiz]

This should provide some speedup (above 17 c/s), provided that huge pages are allocated (with sysctl).

Tested it - no, turns out 17 c/s was already with huge pages despite of the old code not requesting that yet. It's CentOS 7, and per /proc/meminfo I saw AnonHugePages growing to 2+ GiB (this matches expectations: 256 MiB times 8 hardware threads), so perhaps the kernel was using huge pages anyway. I wonder if this possibly wasn't happening on atom's machine.

[Chick3nman]

@solardiz It's possible @atom was talking about JTR's implementation of the Ethereum Scrypt mode and not the Scrypt portion alone. That's how I interpreted it at least.

[jsteube]

Note that Ethereum default is 256k:1:8 and not 256k:8:1. That may explain the difference between 17H/s and 13H/s. When I run JtR against a 1k Wordlist on my 4770k it takes 1m22s with 256k:1:8, but 1m12s with 256k:8:1. So that would be "only" 12H/s but there's some startup time involved that should be subtracted so I think 13H/s is correct for JtR. Also note that I am using MPI (mpirun -n 8) - not OpenMP. From my experience with JtR, MPI gives the best performance (it may be different with SCRYPT based algorithms). As a comparison, hashcat takes 1m18s to process the same system, same wordlist and the same hash. That is almost on par with JtR performance.

The reason why I wrote hashcat was ~20% (it was actually ~40% but I didn't want to overact because it was not fully finished) is because I compared hashcat performance when I was in the middle of the development and compared only the SCRYPT part but without the surrounding PBKDF2 and the Keccak round. From my perspective that was ok, because on GPU the PBKDF2 in SCRYPT (and in this case also including the raw Keccak round) take about no time compared to the SCRYPT part, I didn't even come on the idea it will be different on CPU. The same list was processed in only 57s. I think the large data part (1k of data) is it that bloats up the PBKDF2 around the SCRYPT.

[ethtester]

Hello @jsteube , retail versions of the Radeon Pro SSG cards with upto 1TB of on board nand memory should be shipping very soon. Do you think this hardware development marks a game changer for scrypt hashing rates and what kind of hash rate would you estimate if hashcat were to run against this card? I would be wiling to purchase a few of these units and donate access to them. Thx

https://youtu.be/vyKgT5QUU2Q?t=59m50s
https://www.amd.com/Documents/Radeon-Pro-SSG-Technical-Brief.pdf
http://www.amd.com/en-us/press-releases/Pages/amd-radeon-pro-2016jul25.aspx

[jsteube]

There's too many unknown factors to make an estimation.

[ethtester]

Hello @jsteube, you stated above. "But to make use of the GPU power, we need to spawn at least 1280 parallel computations, so we end up with a memory requirement of 320GB ram per GPU"
Why the 1280 computations number? could you not spawn just enough processes according to how much GPU ram there is?

[jsteube]

Yeah but then where's the advantage to cracking on CPU? The GPU is faster only because there's so many "small" CPU that can run in parallel.

[stealthsploit]

I saw the ether2hashcat note said it was considered deprecated and replaced by ethereum2john.
The ether2hashcat format shown above was $ethereum$s*n*r*p*salt*mac*ciphertext , however ethereum2john converts it to $ethereum$s*n*r*p*salt*ciphertext*mac.

Hashcat only cracks $ethereum$s*n*r*p*salt*ciphertext*mac for me and fails to crack $ethereum$s*n*r*p*salt*mac*ciphertext (?)

E.g. (password of P@ssw0rd1!)

{“ciphertext”:”7f5c865554d67604394ae54d7a4f9735bdb85c90e606a672d18add1d167d793b“,
“cipherparams”:{“iv”:”ae4e8c9c2ac201d6c2baa58ff670fd39″},
“cipher”:”aes-128-ctr”,
“kdf”:”scrypt”,
“kdfparams”:{“dklen”:32,”salt”:”437964c9bd1b5f63bde56560808c894792f8f670694590b776e22381e32dd
33b“,”n”:1024,”r”:8,”p”:1},
“mac”:”96f2a849321cc04cb6c0fcee1bd4b195ca681ca28064dc45000f02e47230c5b6“}}

through ethereum2john gives (and successfully cracks):

$ethereum$s*1024*8*1*437964c9bd1b5f63bde56560808c894792f8f670694590b776e22381e32dd33b*7f5c865554d67604394ae54d7a4f9735bdb85c90e606a672d18add1d167d793b*96f2a849321cc04cb6c0fcee1bd4b195ca681ca28064dc45000f02e47230c5b6

Is there an ethereum2john post i've missed somewhere?

[jsteube]

Do you think we need to update parser code? /cc @Chick3nman

[Chick3nman]

I now follow @kholia's extraction script as the master, hence the message that ether2hashcat was deprecated and replaced, so hashcat should follow his formats for scrypt/pbkdf2/presale. I had noticed the difference in our scripts and i believe there was a forum post that pointed it out as well, however I was not aware that hashcat parsed it under my old format. The OP on this thread was changed to follow @kholia's format directly after his script superseded mine.

[jsteube]

OK, please create a separate issue for this

[stealthsploit]

Hey jsetube,

With regards to the below,

GPU's will not be able to crack this because of the memory requirement. Math behind: Mem per candidate: size_scrypt = (128 * scrypt_r) * scrypt_N = 256MB. But to make use of the GPU power, we need to spawn at least 1280 parallel computations, so we end up with a memory requirement of 320GB ram per GPU. It would require a minimum of a 256 divisions TMTO to get it working on a GTX1080, which drops the performance < 1 H/s.

I've tried digging to find further math, but can't find any additional reading on why we have to spawn (at least) 1280 parallel computations for the total GPU memory requirement?

Thanks.

[jsteube]

That is because GPU are fast only when utilizing all of it's compute hardware in parallel. For todays AMD and NV GPU that is 64 threads per compute unit. For the GTX1080 example above: 1280 / 64 = 20, which is the number of compute units on a GTX1080. You can exchange this number with the number of compute units on your GPU. An alternative to this would be to not fulfil the threads minimum per compute unit but then you'd end up with an almost equal number of parallel computations as you have it on a CPU (my Ryzen 1700 for example shows 16 CPUs cores) and the advantage of a GPU is gone. Even worse since a CPU has much more advanced instructions that a GPU compute unit.

[stealthsploit]

Thanks for the clarification :)

[ZVitaliy]

Hello everyone!
I'm trying to crack my wallet but hashcat saying me no hash in this text.
$ethereum$s*1024*8*1*437964c9bd1b5f63bde56560808c894792f8f670694590b776e22381e32dd33b*7f5c865554d67604394ae54d7a4f9735bdb85c90e606a672d18add1d167d793b*96f2a849321cc04cb6c0fcee1bd4b195ca681ca28064dc45000f02e47230c5b6
I trying both hashes for ethereum 15600 an 15700, using mask of my password but nothing. Maybe it's because my wallet file newest version not a json?

[jsteube]

works fine for me:

root@et:~/hashcat# ./hashcat -m 15700 hash.txt -a 3 'P@ssw0rd1!' --quiet -D1 --self-test-disable
$ethereum$s*1024*8*1*437964c9bd1b5f63bde56560808c894792f8f670694590b776e22381e32dd33b*7f5c865554d67604394ae54d7a4f9735bdb85c90e606a672d18add1d167d793b*96f2a849321cc04cb6c0fcee1bd4b195ca681ca28064dc45000f02e47230c5b6:P@ssw0rd1!
root@et:~/hashcat# 

[ZVitaliy]

@jsteube is there a difference in the commands for Linux and Windows? I'm using Win10. And how to make hashcat using more than one video cards?

[anormore]

Hi all, I'm a bit confused by this...

I've got a presale wallet, which I have generated just fine. I'm running hashcat on my GTX 1080.

However I'm trying a newer wallet (Mist 0.9.3) and it simply doesn't run on the GPU. If I switch to CPU I get the CL_OUT_OF_RESOURCES error. Would applying that regedit patch fix this and start cracking?

I'm trying to solve the problem of the "Special Characters Bug" which I believe I just proved over at: ethereum/mist#3513

[Chick3nman]

I'm guessing your wallet uses the incredibly high 262144 settings? That scrypt setting is beyond the capability of most GPUs currently(by design), and if your CPU is running out of resources, its due to a lack of enough system RAM.

[Krazylo]

Hello,

I lost my password to access my wallet, I have the JSON and the private key (which allows me to access the wallet), I'd like to recover the password because I have other wallets which the same string that I lost. Is it possible to recover the password with the PrivateKey and/or JSON file ? I've tried HashCat but it fails miserably with a old wallet version while password for new wallet can be recovered (tested succesfully). Here is the wallet that can't be decrypted

{"version":3,"id":"36296de5-ea5b-475e-8253-0c8600e26257","address":"a1c9abe25d3ddcc59abf1fe19a77ab975b81b9dc","Crypto":{"ciphertext":"14c43f85c715b56293d7a3eb6b083d47ee25cda3ec65453eba522634387f63aa","cipherparams":{"iv":"2c8ab71c77616f28ea18996a4f46bd02"},"cipher":"aes-128-ctr","kdf":"scrypt","kdfparams":{"dklen":32,"salt":"90787e856c5e6fa9bf970a83cd0eb8b706373571bd0605c050a4032f56e3771e","n":1024,"r":8,"p":1},"mac":"b40bd31f0872bc623a9960301e80c2d057f4466c738aef1fd2f8913e6a0e043d"}}

can someone help?

[emelendez89]

@jsteube is there a difference in the commands for Linux and Windows? I'm using Win10. And how to make hashcat using more than one video cards?

I've also tried the same and got no response, it only says "Exhausted"

[WinUser]\hashcat-5.1.0>hashcat64.exe -m 15700 $ethereum$s*1024*8*1*437964c9bd1b5f63bde56560808c894792f8f670694590b776e22381e32dd33b*7f5c865554d67604394ae54d7a4f9735bdb85c90e606a672d18add1d167d793b*96f2a849321cc04cb6c0fcee1bd4b195ca681ca28064dc45000f02e47230c5b6 -a 3 'P@ssw0rd1!' -w 3 --self-test-disable
hashcat (v5.1.0) starting...

OpenCL Platform #1: Intel(R) Corporation
========================================
* Device #1: Intel(R) Core(TM) i7-4790K CPU @ 4.00GHz, 2032/8129 MB allocatable, 8MCU

Hashes: 1 digests; 1 unique digests, 1 unique salts
Bitmaps: 16 bits, 65536 entries, 0x0000ffff mask, 262144 bytes, 5/13 rotates

[Optimizers available, watchdog, wordlist and some other stuff... ]

Session..........: hashcat
Status...........: Exhausted
Hash.Type........: Ethereum Wallet, SCRYPT
Hash.Target......: $ethereum$s*1024*8*1*437964c9bd1b5f63bde56560808c89...30c5b6
Time.Started.....: Thu Aug 06 02:21:41 2020 (0 secs)
Time.Estimated...: Thu Aug 06 02:21:41 2020 (0 secs)
Guess.Mask.......: 'P@ssw0rd1!' [12]
Guess.Queue......: 1/1 (100.00%)
Speed.#1.........:      524 H/s (1.59ms) @ Accel:1 Loops:1 Thr:1 Vec:1
Recovered........: 0/1 (0.00%) Digests, 0/1 (0.00%) Salts
Progress.........: 1/1 (100.00%)
Rejected.........: 0/1 (0.00%)
Restore.Point....: 1/1 (100.00%)
Restore.Sub.#1...: Salt:0 Amplifier:0-1 Iteration:0-1
Candidates.#1....: 'P@ssw0rd1!' -> 'P@ssw0rd1!'

Started: Thu Aug 06 02:21:36 2020
Stopped: Thu Aug 06 02:21:42 2020

I'm not sure if it's actually working or not for 15700 EthWallets IN WINDOWS10 (I've already got a positive crack on WPA2 hashes)

[Chick3nman]

It's clear that your runtime is broken @emelendez89 as you are passing --self-test-disable, and in doing so, bypassing critical blocking errors. This is not hashcat's fault, it tried to warn you that your runtime was broken by throwing the self test error. Please get a proper runtime installation that doesn't cause those errors and repeat the test, you should find that the kernel works just fine.

[emelendez89]

It's clear that your runtime is broken @emelendez89 as you are passing --self-test-disable, and in doing so, bypassing critical blocking errors. This is not hashcat's fault, it tried to warn you that your runtime was broken by throwing the self test error. Please get a proper runtime installation that doesn't cause those errors and repeat the test, you should find that the kernel works just fine.

IDK wtf happend, just had to uninstall and reinstall opencl_runtime_18.1_x64_setup.msi

$ethereum$s*1024*8*1*437964c9bd1b5f63bde56560808c894792f8f670694590b776e22381e32dd33b*7f5c865554d67604394ae54d7a4f9735bdb85c90e606a672d18add1d167d793b*96f2a849321cc04cb6c0fcee1bd4b195ca681ca28064dc45000f02e47230c5b6:P@ssw0rd1!

Status...........: Cracked

tyvm @Chick3nman