AVX2 version of "anime" algo produces rejects #236
Comments
|
Here's a bit about low difficulty shares. You aren't losing anything you're just submitting shares https://bitcointalk.org/index.php?topic=1326803.msg53705296#msg53705296 Is this a new problem or just newly discovered? It should be simple to fix given that lesser architectures work. There must be a targetting mismatch This line bothers me: Target diff: 2.46075e-316, Targ: ... Is that a copy/paste issue or is the target diff really 2.46075e-316? You can verify the low difficulty by comparing the share diff with the target diff for After re-reading my onw notes I remembered the connection with the stratum diff set |
|
Actually this is excellent news. Anime does not use the new aggressive targetting I discussed in BCT, It means the low-diffidulty shares are not caused by my code change and are most likely a pool Edit: you can also confirm it's a pool issue by trying a different miner. TPruvot doesn't support |
|
@JayDDee ofc I've tested this issue on non-avx2 build of cpuminer-opt. You can easily "Ctrl+F" on the attached file to find "rejected" - no results. It's a (rather) large period of time logged in that file without rejects. And if I start the avx2 build just right now (a few minutes after the end of the log), I see:
Oops, 2 of 11. Same settings, "-t 8", nothing else changed in bat file. I can provide more tests if I could identify what to test more :D Pool is a Yiimp one, but no issues with 2 other miners (CUDA & OpenCL). Update: testing the generic miner (which is very slow btw): Will report the results later. |
|
It's looking like the issue I've seen. Pay close attention to the stratum diff changes. If you get low-diff rejects try setting a different Since it's working with Cuda and OCL it's likely a miner issue and not a pool issue. I've seen it on 3 algos, anime makes 4, and each has its own poisoned stratum diff, It's been an interesting day, two bizarre problems. Something else you can do is check if the hashrate reported at the pool is reduced when you |
|
I have a new release almost ready to go so this issue will probably have to wait until the next one. |
|
The next release will be out shortly and will contain some extra data getjering to help with this Whe a low difficulty share is reported the debug code will be activated. On subsequrent share This will provide a full trace audit of the process. The block report will display the target diff. When a reply is received rejecting the share the hash and target values will be displayed again Everything sghould match up but something obvioyusly won't either test passed when the Whati s importatnt is what mismatched and by how much. The math for converting a 32 byte hash or target value to the corresponding diff is a simple The converter uses double precision with should give 52 precise bits before error creeps in. This is my area of focus with agressive submission. I want to ensure that close share results I will verify the math but if the rejects are close to passing I don't think I'll try to fix it. All the above ignores the association with a specific stratum difficulty. This will likely be important |
|
As of that generic version, it has been workin for around 10 hours with not a single reject.
I see hashrate drop (charts) and a separate table cell on the front-end web side with % of rejects. |
|
Upgraded to v3.12.0 cpuminer-zen -a anime -t 8 -o stratum+tcp://miningbase.tk:3033 -u ... -p c=ANI CPU: AMD Ryzen 9 3900X 12-Core Processor . Starting miner with AVX2 AES... [2020-02-06 10:47:13] 24 CPU cores available, 8 miner threads selected. As I see: UPDATE: Why AVX2 builds do not log this? Seems that avx2 versions (with "lanes") use another code. [2020-02-06 11:05:38] New job c736 I see no debug info on accepted shares which could be seen with sse42 build. UPDATE2: avx2 build hangs sometimes right after first submitted share without console output and CPU load... Started 10 times, two times had a hang. |
|
I messed up the debug logs. First it was only implemented for single llane hashing, second But that is only a distraction and has nothing to do with the problem. I only see stratum diff = 0.01 in your logs. Do you get rejects with different stratum? I also need to see both the target and hash from the submit and the reject for the same share. But I did notice somethuing odd, no target value displayed in th ereject reason log: [2020-02-06 11:05:56] Reject reason: Low difficulty share I don't know if it's significant because the submit logs are missing so I can't compare. |
|
Maybe a more formal test procedure woiuld help.
Edit: other than the the log snafu the share testing for anime still uses the old method for Your note about a hang with AVX2 may be an anomaly. Keep an eye on that issue but don't I repeat, the stratum diff is critical. I determines what the target is. I need the stratum diff, |
There is no different one sadly. Diff control params are not enabled on the pool side through passing user login data. I cannot send aes-avx build error logs as there are no rejects with it... [2020-02-06 18:23:12] 24 CPU cores available, 12 miner threads selected.
That's how it is logged in v3.12.0 avx2 windows build:
And on Ubuntu (mine build of v3.12.0 with your build.sh):
Please note I do not substring anything to "..." by myself or by reducing console windows size. So I cannot send you the data which is... invisible :-P in avx2 build and does not exist in other builds. Seems that I'll have to remove all substrings with "..." from the source code to proceed further. |
|
What is the stratum diff when you use aes-avx with no rejects? You don't necessaruilly have to control the stratum dif just observe what it is when things This is also important when you successfully tested other miners what stratum diff were they using? |
|
Diff seems to be the same, around 0.01 (I think pool sets it lower than for GPU miners, depending on the user agent string). Any cpu miner gets 0.01 startum diff. This log was partially cut-off: [2020-02-06 21:00:24] Stratum connection established So it's 0.01. I do not see any stratum diff changes during several hours. |
|
If I understand correctly, with aes-avx and stratum diif 0.01 you get no rejects but with Is that correct? This is not what I expected. I really need to capture the hash and target at submit time for a share that will be rejected. It will have to wait until I can release a fix or, if your willing, you can make the code change In file cpu-miner.c search for function submit_solution. It contains the following block of code: if ( lowdiff_debug ) Immediately following submit_solution is an almost identical function submit_lane_solution. Copy the missing block of code from submit_solution to submit_lane_solution so they look identical bool submit_lane_solution( struct work *work, const void *hash, if ( lowdiff_debug ) You can also just copy and paste the above to replace the existing function. If you want to give it a try I can help walk you through it. It's faster than waiting for a new release. |
Exactly! These rejects sometimes are as high as 30-35% of the total share count. All avx2-enabled builds affected. I'm not on the C/C++ side fairly speaking (work in C# .NET environment in VS only, and ofc web development), but I could try to debug on the submit/recieve side through web proxy (Win env). Upd: AVX2 version output with your fix: [2020-02-06 11:48:54] Hash[7:0}: 00000040 69c6c355 315dfbf4 4d504bb6 992736b0 d07e2f56 7ed5c73f a7d65cf3
|
|
Good data! I don't know why it was rejected, everything looks good. Even in the reject report the hash is Since there us a clear point of deviation between AVX and AVX2 I have another request. In algo/quark/anime.c (used by AVX) replace submit_solution with submit_lane_solution, use any number for the lane arg. in algo/quark/anime-4way.c (used by AVX2) replace submit_lane_soluttion with submit_solution. This is the only code that is different between them that also affects share submission. If that doesn't produce a breakthrough I would like a longer test with more samples so I can |
|
I didn't notice you posted a url to test with. I'm testing now and reproduced the problem. I was able to set the startum diff by adding d=0.08 to the password but I still see low diff shares. |
|
These 2 shares tell an interesting story. The target diff is 0.01 yet a share with diff 0.0352 was Rejecting a share with 3x the target diff is not a small error, and its inconsistent at that. It defies logic And then it's only the AVX2 build, AES-AVX works fine. I'm going to swap submit functions now. [2020-02-06 18:18:43] 12 submitted by thread 4, lane 1, job cfaa |
|
I'm starting to suspect it's a pool issue. Is there another pool you can try? Except for the fact that the avx build works everything else points to a pool issue rejecting valid The shares are actually valid hashes otherwise they would be rejected as invalid. That the The way this works in the miner is each algo has a hard coded target factor. The pool sends The target hash is used to validate the share hash, if it passes the nonce for the share is submitted. The pool hashes the nonce, which is just part of the original data that the miner can play with, to It's statistically impossible for the miner to submit a corrupt share that passes validation at the pool. There's no pattern to the rejected share diff. Some very high diff shares get rejected but some I've analyzed the miner code and there's nothing in the anime code or common code that can I've tested with both the old and new hash test and it didn't make a difference. The old one is If it wasn't for the fact the rejects don't occur with the AVX build I'd be absolutelty convinced it I don't know what to do from here. The only thing I can think of is to try another pool to see if it |
|
Another interesting note is I modified both the AVX and AVX2 versions to use the new hash |
|
@JayDDee maybe give a try to force the compiler not to vectorize some key places in the avx2 code when building to avx2 arch? This is rather simple, together with switching on vectorization report in GCC (I'm not sure about GCC report, as I use only Intel and MSVC compilers in my small and easy 20-lines-of-code C/C++ dllimports to C# code). As for my main language volatile things just do magic sometimes :-) |
|
There's nothing special about anime, it's very similar to quark and many other algos. It's not a if the miner was really submitting low diff shares the accepted hash arte at the pool would be A non vectorized version already exists, it's the AVX version. The compiler can't code parallel data As far as I'm concerned it's a pool issue until new data show otherwise. |
|
I see something interesting on the main page of https://miningbase.tk, top-left corner:
Is this something you've already fixed for this algo the days before? |
|
I know nothing about this. There's no mention of the version so I have no idea how long BTW have you tried older versions of cpuminer-opt? |
|
Haha, got it! Tested avx2-zen build history: latest zen - invalid shares Something was broken in version 3.9.2.5. Not a pool issue! That test was done with a special bat file which is able to launch 12 versions with -t 2 in parallel (CPU almost died). |
|
Good work. I'll dig in to the changes in v3.9.2.5 Confirmed it broke in v3.9.2.5. Interestingly there were no changes to anime targetted code |
|
I have a fix. It looks like a pair of vector utilities have their polarity reversed. I swapped their usage in anime-4way A new release should be out in a few hours. This will need follow up because other algos use it with no problems, meaning they have There was some serious misdirection with this problem. I would have expected invalid shares |
|
cpuminer-opt-3.12.1 is released. Plewase test and report any problems. |
|
I confirm this algo is successfully fixed. Hope others were not affected (I just do not have credentials for all of those affected by the code changes in 3.12.1). Some of the algos among those do work. Thanks. |
|
Te follow up will be done off line to ensure the utility is logically correct on all tagets and any algo Thanks for your testing.Closing |
|
Closing. |
|
I have made the correction to the utilities, only AVX2 affected, and corrected the usage in quark |
|
This issue doesn't want to seem to go away. It turns out one of the utilities is corect but the other The whole point of that code was to skip code whose results will be later discarded to save time. I'm testing a solution that seems to be working properly. No rejects and consistently higher More testing is require before release. |
|
Another twist. I discovered "the right way to do it" using the intrinsic _mm256_movemask_epi8 It seems to be working fine and the AVX2 code npow looks more like the AVX512 code. Still testing. |
|
cpuminer-opt-3.12.3 is released with a redesign of the code responsible for this issue |
|
Great! Works as expected (with a visible hashrate improvement for avx2). Cannot test AVX512 build (no such CPUs). |
|
I think I can finally close this for good. |
[2020-02-05 10:51:00] New stratum diff 0.01, block 6205750, job b9cf
anime: miningbase.tk:3033
Diff: Net 0.778, Stratum 0.01, Target 0.01
[2020-02-05 10:51:00] 8 miner threads started, using 'anime' algorithm.
[2020-02-05 10:51:10] 1 submitted by thread 0, lane 1, job b9cf
[2020-02-05 10:51:10] 1 Accepted 1 S0 R0 B0, 9.493 sec (3ms)
Diff 0.0178 (2.29), Block 6205750, Job b9cf
[2020-02-05 10:51:20] 2 submitted by thread 5, lane 0, job b9cf
[2020-02-05 10:51:20] 2 A1 S0 Rejected 1 B0, 10.634 sec (4ms)
Diff 0.0882 (11.3), Block 6205750, Job b9cf
[2020-02-05 10:51:20] Reject reason: Low difficulty share
Share diff: 0.0882113, Hash: 0000000b5614178400000000...
Target diff: 2.46075e-316, Targ: ...
[2020-02-05 10:51:27] New job b9d0
After an hour I got these stats on the pool side:
Reject*
17.8%
It's rather high percentage - could be banned soon.
My tests show that non-avx builds work properly. I've also checked linux builds (mine) and rejects are there, too. Btw, AVX2 adds around x1.5 hashrate improvement...
As I see, these are not "stale" shares (as they could be due to a very short block times).
Test data (wallet is not mine - test one provided by pool):
-a anime -o stratum+tcp://miningbase.tk:3033 -u AJ7q7fXheNfnt89bw4XQJekqUFaBoVf3c9 -p c=ANI
The text was updated successfully, but these errors were encountered: