[GBT] Failure to restore connection right after getwork fallback attempt

[YetAnotherRussian]

1. Connect to a wallet and start mining using GBT
2. Close wallet (or emulate some network problems)
3. Wait for
   ...
   [2023-08-28 10:28:19] JSON decode failed(1): '[' or '{' expected near '<'
   [2023-08-28 10:28:19] getblocktemplate failed, falling back to getwork
   [2023-08-28 10:28:49] HTTP request failed: Connection timed out after 30000 milliseconds
   [2023-08-28 10:28:49] json_rpc_call failed, retry after 10 seconds
   [2023-08-28 10:28:49] HTTP request failed: Connection timed out after 30000 milliseconds
   [2023-08-28 10:28:49] json_rpc_call failed, retry after 10 seconds
   ...
4. Reopen wallet (or ... - see step 2)
5. Connection won't be restored, ever

Workaround: --no-getwork

I may leave this upon your decision, but this is not expected to restore connection with different parameters, and then use 'em permanently. Thanks.

[JayDDee]

Does anyone use getwork anymore? Maybe I should make no_getwork the default.

I presume this occurs after a successful GBT connection was established and later failed. I don't think this was considered in the initial GBT implementation. The fallback to getwork was intented for GBT failing at startup, assuming GBT wasn't supported.
With GBT in wide use it may no longer be necessary or useful to automatically fallback to getwork.

It might be possible to distinguish startup failure from a network failure but that could be more complicated and require more testing than simply disabling automatic fallback. I'll look into it before deciding.

[YetAnotherRussian]

Does anyone use getwork anymore? Maybe I should make no_getwork the default.

I think almost noone. Old (still popular) Bitcoin Core wallet versions use it, but this is not about CPU mining.

[JayDDee]

Does anyone use getwork anymore? Maybe I should make no_getwork the default.

I think almost noone. Old (still popular) Bitcoin Core wallet versions use it, but this is not about CPU mining.

IIRC bitcoin core dropped support for getwork a coule of years ago, I doubt the old wallets still work. Getwork would only likely be used on old coins forked from Bitcoin that haven't been updated for GBT.

I think the solution might be as simple as automatically activating no-getwork after having successfully connected using GBT. It should be a simple change.

[JayDDee]

It's not that simple, 2 flags are involved. What happens when you set --no-getwork? Does it error out or retry?
The network failure will set the have-gbt flag to a failed state which should cause the miner to exit.
This may be desired behaviour at startup but not for a network interruption unless no-retry is set.
Coordinating multiple flags complicates the logic.

[YetAnotherRussian]

What happens when you set --no-getwork? Does it error out or retry?

Retries in a loop. The line "getblocktemplate failed, falling back to getwork" is not printed, others (from the log in post # 1) are.

The main thing - connection is restored successfully in that case:

Everything is OK.

[JayDDee]

Excellent, only requires a one line change.
Your timing is good, I was preparing a new release this week.

[YetAnotherRussian]

I was preparing a new release this week.

Mybe you could look into max-diff option code then, meanin disable useless CLI output spam (higlighted section):

Yep new block info is useful, but fake hashrate lines are a complete spam. If the diff is high for a prolonged period, then it's a mess.

Btw, atempt to mine BTC (hahaha) using sha256d and latest Bitcoin Core wallet with GBT produces a cpuminer-opt crash, but this task (attempt to fix) is useless. I just used it to test connection.

[JayDDee]

Mybe you could look into max-diff option code then, meanin disable useless CLI output spam (higlighted section):

That line shouldn't be printed without a blue log header, that's a bug.
Diff doesn't change until a new block is issued, so the log is redundant for "New work".
Displaying TTF for new work is done in cases of long block times to keep the TTF visible.
I'll investigate the bug and filtering the New work log during idle time.

Btw, atempt to mine BTC (hahaha) using sha256d and latest Bitcoin Core wallet with GBT produces a cpuminer-opt crash, but this task (attempt to fix) is useless. I just used it to test connection.

I suspect it's due to a missing BIP, something I've been afraid of. It's almost impossible for cpuminer to keep up with Bitcoin improvements and many of them will be picked up by other coins so this problem could spread like a virus.
If you can post the logs (-D -P) and a backtrace I'll take a look, but no promises.
Edit: Try Pooler's cpuminer for sha256, if it works I have something to work with.

[JayDDee]

I understand the logging bug. The log header wasn't printed because the work data didn't change, the work wasn't new.
The TTF report was printed because the code assumes it won't be hit unless there's new work. It works fine in most cases,
except when waking up from a conditional mining pause to retest the condition every 5 seconds while mining solo.
The fix is not complicated but involves a strcmp.

Update: I have a fix that avoids adding a strcmp. The logs will still be displayed while paused but only when new work is received.
It's still redundant if it's not a new block but it serves as a heartbeat confirming the miner and server are still alive.

[YetAnotherRussian]

I suspect it's due to a missing BIP, something I've been afraid of. It's almost impossible for cpuminer to keep up with Bitcoin improvements and many of them will be picked up by other coins so this problem could spread like a virus.
If you can post the logs (-D -P) and a backtrace I'll take a look, but no promises.
Edit: Try Pooler's cpuminer for sha256, if it works I have something to work with.

Pooler's version seems to work:

Crash here:

[JayDDee]

It looks like a crash printing a log, displayed the timestamp but not the content. There are also differences in the HTTP parameters but I don't know what they mean or why.
I'll try to fix the crash for the next release, probably a buffer overflow, then we'll see what happens next.
Understanding the parameter differenences will be a lot trickier.

Edit: I need more logs, there are no cpuminer logs in your snapshot to provide a point of reference to find where it crashed.

[JayDDee]

cpuminer-opt-3.23.0 is released with the title issue and log spam issue fixed as noted above.
I wil not pursue the sha256 issue any further.

[YetAnotherRussian]

and log spam issue fixed

I now see a total lack of fast hashrate reporting:

So, a single output per block. I'm not sure if that was expected (@ your side). However, it is enough to see this info once in a block, for fast (5sec) output and speed testing better to use benchmark mode.

Reconnect issue is fixed.

[JayDDee]

Actually it should be a log for any new work but shouldn't go over two minutes. Slow algos have fewer intermediate jobs so it's mostly block time.