"Tunable cost", allow --test with --cost #816
Comments
|
Yes, I also think this could be useful. I didn't look at the code yet, but couldn't you just replace the tests which don't fit the requested --cost settings with tests that do (provided that such tests exsist), similar to we do for some formats with --encoding, just not format specific, and exit with an error code if no such test exist... |
|
That might be a good idea... So instead of hacking that loop, just add a section before it that shuffles the test vectors if applicable and possible. I like this because it separates the Jumbo hacks from the core code. |
|
We should also list min and max costs used for benchmarking, if --cost was used or verbosity was bumped. This would make it easy to see why a format like Office performs better or worse than expected. |
|
I had a look three times now but I give up :) @frank-dittrich I leave it to you, whenever you get the time and inspiration. |
|
@magnumripper, with increased verbosity we might even want to report tunable costs in regular cracking sessions if all salts have the same tunable cost values. |
|
Definitely. I'd say at verbosity>3 already. |
|
On 10/25/2014 06:09 PM, magnum wrote:
Why would you give up? Is the code in question too complicated / confusing? |
|
Nothing wrong with the code, but it's always hard to "digest" other people's structs. I can do it for sure, but it will need to be on The Right Day. My IQ seem to fluctuate wildly over time :-) A hack like the "pot sync" took me many many tries before I got something to even keep and improve on. Sometimes I see old commits of mine and have absolutely no idea how I could grasp it at the time. |
|
Also, in bench.c nothing is "normal". We don't have a database so everything needs to be done bare handed. |
hashes have same cost. This was suggested in #816.
|
This is implemented but I might clean up the code a bit. Now, for example, we can see the individual speeds for different test vectors in PBKDF2-HMAC-SHA1: That last one using 4096 (a WPA-PSK key from a config file) should be more than double the speed of sniffed WPA-PSK. It's more than 2x due to half the number of iterations (we can early reject after creating half of the key) and less post processing. |
|
I never reflected over this until now but IMHO just saying -cost=4096 should infer "4096:4096" so a range would always need to be specified. For --salts it's another thing but for iterations I think that would have been better. Probably too late to change though. |
|
I would like to print what counts are actually used (like in normal cracking) but that would come between the "(8xOMP)" and "DONE" if I just printed it in that code block. Maybe it should be like these mockups: For --test we should print this regardless of verbosity (in real crack runs, it's only printed if --cost was specified or verbosity was bumped from default). |
Solar wanted me to implement it exactly like for --salts=, meaning: a single value means >=. |
|
Here's the output now: |
|
Changed output a little. First, the "tested only" was a bit misleading since we normally test all costs but benchmark just one or two. Although when using --cost you will actually also only test the specified cost(s) - this is intended, for devel reasons. |
Should it infer "4096:4096" or "0:4096" ? |
|
Instead of |
IMO the most intuitive would be -cost=4096 resulting in only that exact cost loaded. Not a big deal though. If Solar was specific about it, I'll keep my big mouth shut. |
|
The bad thing about this line of thinking, is that -salt is exactly opposite in design as this one. -salt=v meaning >=v is due to higher values being better performance. So for the intent to be the same, -salt=v --> >=v and -cost=v --> <=v do the same thing, they only deal with data at a specific level or 'faster'. But like you said, Solar's call. There are ways to specify exact meaning, so that is all that really is needed. |
|
May be I mis-interpreted him, or interpreted too much into his words. BTW: specifying cost ranges doesn't work well with --test. |
The ranges themselves work as they should, the problem is the benchmark code only uses the first two test vectors that fit. For eg. pbkdf2-hmac-sha1, this means only 1000 rounds are benchmarked without cost, or if -cost is used with any range that involved 1000. But there is only one vector for 4096 so if you say -cost=4096 it will benchmark it and one of the 10,000 iteration vectors. If you say 4096:4096 it will use the one vector for 4096 twice - so that does work properly. |
Note the scattering of "Speed for cost 1 (interation count) .... stuff. There was NO cost asked for. why are these being output?? |
|
Ah... I never tried it with --test=0 The above makes sense This doesn't. Maybe we should just mute it for --test=0. |
Because now, for the first time, you can actually know what is actually benchmarked in a mix format. I like it this way (except for --test=0) but we could opt to only print it if --verbosity is above 3 (and still mute it completely for -test=0). What do you guys think? |
|
Related: John complains if you specify more than FMT_TUNABLE_COSTS cost values, but not if you specify more cost falues then there are tunable costs for a format. Also: With --test, but wiithout --format, it shouldn't be allowed to use --costs=. |
I can fix this for --test, but you should fix it for non-test runs: |
|
All issues fixed.
I disagree. We should not overdo this. The new error messages should be descriptive enough. A myriad of obscure edge case support fixes might introduce bugs. Also, time is better spent writing actual enhancements. And for wildcards, they might be relevant with cost, eg |
I'd really like this, for example benchmarking only the WPA test vector from the pbkdf2-hmac-sha1 format with
--test --cost=4096. I had a look at bench.c (lines 182 and on) and saw now obvious way to implement it without rewriting more than I'm happy to. So I'll just let it mature in my head.The text was updated successfully, but these errors were encountered: