Reuse PBKDF2 intermediate keys if duplicate essid is detected not working on large hccap's

[ZerBea]

Reuse PBKDF2 intermediate keys if duplicate essid is detected is not working on large hccap's

Session..........: hashcat
Status...........: Running
Hash.Type........: WPA/WPA2
Hash.Target......: test.hccap
Time.Started.....: Sat Jan 7 10:57:24 2017 (1 hour, 40 mins)
Time.Estimated...: Sat Jan 7 14:39:11 2017 (2 hours, 1 min)
Input.Base.......: File (testlist)
Input.Queue......: 1/1 (100.00%)
Speed.Dev.#1.....: 156.3 kH/s (4.91ms)
Recovered........: 252/85743 (0.29%) Digests, 245/83842 (0.29%) Salts
Recovered/Time...: CUR:2,166,N/A AVG:2,151,3624 (Min,Hour,Day)
Progress.........: 945155872/2086240486 (45.30%)
Rejected.........: 0/945155872 (0.00%)
Restore.Point....: 0/24883 (0.00%)
Candidates.#1....: ~!@#$%^&* -> Tr1angle
HWMon.Dev.#1.....: Temp: 75c Fan: 59% Util: 99% Core:1277Mhz Mem:3004Mhz Lanes:16

Speed must be much faster, because there are many, many duplicate essid's in the hccap.
It's also seen in the output:
ESSID1:xxxxxxxxxxxx:yyyyyyyyyyyy:password1
ESSID1:xxxxxxxxxxxx:yyyyyyyyyyyy:password2
ESSID1:xxxxxxxxxxxx:yyyyyyyyyyyy:password3
ESSID2:xxxxxxxxxxxx:yyyyyyyyyyyy:anotherpassword
ESSID1:xxxxxxxxxxxx:yyyyyyyyyyyy:password4
ESSID1:xxxxxxxxxxxx:yyyyyyyyyyyy:password5
ESSID3:xxxxxxxxxxxx:yyyyyyyyyyyy:anotherpassword
ESSID4:xxxxxxxxxxxx:yyyyyyyyyyyy:anotherpassword
ESSID1:xxxxxxxxxxxx:yyyyyyyyyyyy:password5

Best regards
ZerBea

[jsteube]

I need the .hccap you're using to reproduce locally

[jsteube]

OK, I got the file. Now I need one example of a duplicate which is not detected as duplicate. Please name me one.

[ZerBea]

Send you a pm.......

[ZerBea]

Candidates.#1....: ~!@#$%^&* -> ZZZZZZZZZZ

[ZerBea]

So all founds on the same essid must be in a row.......
but they didn't.

[jsteube]

Yeah I think I found the root problem. Need to think about a good solution. Prepare for testing :)

[ZerBea]

It works on small hccap's < 100
but my test files a really big ....
Ok, I stay tuned......

[ZerBea]

Right now, I do a presort on the essid and run hashcat on this presorted hccap's.
Speed is above 179 M!

[ZerBea]

Maybe the idea of sorting by length of essid is not a good idea.
It could be a better way to sort by alphabetical order....

[jsteube]

OK, I've patched hashcat. There's also a new beta at https://hashcat.net/beta/

In an ideal world, the output of your file should look like this:

Hashes: 69295 digests; 66980 unique digests, 57302 unique salts

That is because your file has 11993 handshakes with the same ESSID. Now the problem is that there's a chance that some of those handshakes have the same ESSID, but they are actually from a different AP. There's no way for hashcat to know about this. Therefore we must check all the handshakes, even if they have the same ESSID. But we can still make use of the duplicate ESSID shortcut. I've patched hashcat, so don't get confused if you see this loading screen instead:

Hashes: 69295 digests; 69295 unique digests, 69295 unique salts

It will still make use of the duplicate PBKDF2 shortcut. You can find out if you meassure the time it takes to process a full wordlist. Simply ignore what the "Time.Estimated" in the status screen tells you.

[ZerBea]

Well done. Now everything works like expected.
Again faster then john.........
The test file is one of my smallest.......