Discussion of the future of the PoW algorithm #2

[dEBRUYNE-1]

A productive meeting about the future of the PoW algorithm took place yesterday (logs can be found here). Furthermore, common ground was found as well. That is, rough consensus was that:

(i) RandomX, if ready, would be preferred over another tweak in October.

(ii) The path of least resistance seems to be a course of action where RandomX is adopted in October and a switch to an ASIC friendly algorithm (such as SHA3) is made in case of a RandomX failure. This path was preferred over precommiting to a set date for a switch to an ASIC friendly algorithm.

I'd like to use this issue to further discuss a few things that were brought up in the meeting. The most salient point of discussion is how to qualify failure of RandomX. I personally mentioned ASIC miners driving out all other miners, thereby showing a significant efficiency advantage. sech1's comments are consistent with this notion:

steep increase in hashrate (and profitability drop) without price changing is a strong indicator of more efficient hardware (ASICs)

And:

I think "RandomX failed" condition comes true only when there is no debate about it in the community at all. Like when hashrate spiked so much and for so long time that everyone gets 20% (at best) from money spent on electricity.

Other aspects we can use to gauge failure (though they are more prone to manipulation / gaming):

• Mining profitability compared to other coins.

• Reports of miners leaving the network.

• Efficiency statistics in case an ASIC manufacturer creates an ASIC for RandomX and publicly sells it.

Some other points of discussion:

• How to find capable reviewers that can check the degree of ASIC resistance for RandomX?

• In case of a RandomX failure, should we switch to SHA3 immediately or allow a bit of a grace time (e.g. 3 months after RandomX is qualified as failure)?

Lastly, I'd like to kindly ask anyone to stay on topic and ensure this issue retains a high signal to noise ratio. The previous issue got kind of cluttered when people digressed.

[SamsungGalaxyPlayer]

I believe the best way to gauge failure is the presence of private ASICs that a) control a majority of network hashrate, or b) are expected to control a majority of the hashrate within 3 months. I use 3 months as a benchmark for a quick switch to SHA3 (or other similarly ASIC-friendly algorithm), since most manufacturers and expects say that this time is appropriate.

For a) and b), Monero should evaluate RandomX to see if a fatal, patchable flaw is known. If there is a patch that will improve RandomX and remove this flaw, it should be considered instead of SHA3. If, however, no flaw is known and any tweaks would just break the current round of ASICs, this option should not be considered.

Then, under a), Monero needs to either a1) wait it out until SHA3 is implemented, or a2) emergency fork with a tweak until SHA3 will be implemented 3 months later. I think this is a good point of discussion. Both have pros and cons.

Under b), Monero still has the same two options, but the community may decide to prefer the former over the latter to avoid an emergency hardfork. This may be especially the case if ASICs are expected to threaten the network near the end of the 3 month window.

Some of the other mentioned measurements are indicators that ASICs may be present, but I would not commit us to using one of those at certain thresholds. I would instead say that people should make the case for network issues using whatever indicators are sensible. Ultimately, a call will need to be made at that time, and it could be controversial. I believe an expressed intent to follow the actions outlined above is enough.

[tevador]

I'm reposting the overview of RandomX performance compared to CryptoNight (CNv0 = original CryptoNight, CNv4 = current PoW).

These 4 machines were used as the testing sample, representing low-end, mid-range and high-end systems:

CPU	Architecture	RAM	OS	AES
AMD Ryzen 7 1700	x86_64	16 GB DDR4	Ubuntu 16.04	HW
Intel Core i7-8550U	x86_64	16 GB DDR4	Windows 10	HW
Intel Core i3-3220	x86_64	2 GB DDR3	Ubuntu 16.04	software
Raspberry Pi 3	armv7l	1 GB DDR2	Ubuntu 16.04	software

Software used:

RandomX 0.3.3: https://github.com/tevador/RandomX
xmrig version 2.14.1 (CNv0 and CNv4): https://github.com/xmrig/xmrig

RandomX has 2 modes: "light" mode requiring 256 MiB of memory and "full" mode, which requires 2 GiB of memory.

Single threaded performance

Table lists the performance using 1 CPU thread.

CPU	CNv0	CNv4	RandomX (light)	RandomX (full)
AMD Ryzen 7 1700	70 H/s	68 H/s	68 H/s	590 H/s
Intel Core i7-8550U	79 H/s	74 H/s	94 H/s	770 H/s
Intel Core i3-3220	29 H/s	26 H/s	78 H/s	-
Raspberry Pi 3	3.1 H/s	1.0 H/s	3.2 H/s	-

Multi-threaded performance

The number of threads was selected for maximum performance.

CPU	CNv0	CNv4	RandomX (light)	RandomX (full)
AMD Ryzen 7 1700	536 H/s (8T)	522 H/s (8T)	640 H/s (16T)	4250 H/s (8T)
Intel Core i7-8550U	255 H/s (4T)	198 H/s (4T)	128 H/s (4T)	1660 H/s (4T)
Intel Core i3-3220	42 H/s (4T)	42 H/s (4T)	187 H/s (4T)	-
Raspberry Pi 3	10.7 H/s (4T)	3.5 H/s (4T)	12.3 H/s (4T)	-

RandomX initialization times

For fair comparison, it should be noted that RandomX needs to be reinitialized every 2048 blocks. The table lists this initialization time in seconds using the optimal number of CPU threads:

CPU	RandomX 256 MiB	RandomX 2 GiB
AMD Ryzen 7 1700	0.61 s	3.0 s
Intel Core i7-8550U	0.55 s	3.5 s
Intel Core i3-3220	0.75 s	-
Raspberry Pi 3	8.3 s	-

Time to verify 1000000 consecutive blocks

This simulates the time of initial blockchain download, assuming inifinite download speed and no other verification except PoW. In reality, the differences will be smaller. For RandomX, the times include reinitialization every 2048 blocks.

CPU	CNv0	CNv4	RandomX (light)	RandomX (full)
AMD Ryzen 7 1700	31 minutes	32 minutes	31 minutes	28 minutes
Intel Core i7-8550U	65 minutes	84 minutes	135 minutes	38 minutes
Intel Core i3-3220	400 minutes	400 minutes	95 minutes	-
Raspberry Pi 3	26 hours	79 hours	24 hours	-

In this case, there is a large difference especially for low-end machines without hardware AES, where RandomX is much faster than CN/R even in "light" mode.

Time to verify a new incoming block

This is important for block propagation, DoS attack resistant and also for pools, which have to verify a large number of hashes.

CPU	CNv0	CNv4	RandomX (light)	RandomX (full)
AMD Ryzen 7 1700	14.3 ms	14.7 ms	14.7 ms	1.7 ms
Intel Core i7-8550U	12.7 ms	13.5 ms	10.6 ms	1.3 ms
Intel Core i3-3220	34.0 ms	38.2 ms	12.8 ms	-
Raspberry Pi 3	320 ms	1000 ms	310 ms	-

In this case, RandomX is faster across the board, allowing much faster verification for higher-end systems and systems without hardware AES.

[needmoney90]

Here we go again

[OrvilleRed]

From yesterday's meeting, the specific ASIC-friendly algorithm to use (should one be needed) seems to warrant further discussion.
To avoid confusion, I would suggest referring to "an ASIC-friendly algorithm" rather than specifically "SHA3" until there's consensus on that point.

[dEBRUYNE-1]

@sgp:

b), Monero should evaluate RandomX to see if a fatal, patchable flaw is known. If there is a patch that will improve RandomX and remove this flaw,

I'd be strongly opposed to trying to patch RandomX for the following reasons:

• We cannot know with certainty what to patch unless we have access to the device that is enjoying the significant efficiency advantage. Put differently, patching is conjecturing unless we're in possession of the ASIC (i.e. how do we know the exploit we're trying to patch is actually the exploit used to gain a significant efficiency advantage?).

• A patch may well introduce another efficiency advantage that can later be exploited.

• Patching is essentially tweaking and thus reintroduces the risks associated with tweaking listed here:

monero-project/monero#3387

• Patching once is going to open the door to continue a strategy of tweaking, a course of action we're seeking to discontinue.

In my opinion, patching should not be considered. History has proven that the tweaks are ineffective, inherently centralizing, and potentially dangerous. In my opinion, it would be foolish and irresponsible to continue them in any kind of form. If RandomX fails, we should not try to apply band-aid, but rather switch to an ASIC friendly algorithm such as SHA3. Arguably, the risks of switching, at that point, are significantly less than the risks of trying to patch a failed algorithm.

[SamsungGalaxyPlayer]

@dEBRUYNE-1 in my opinion, your hard line against all forms of patching is not appropriate. Is SHA3 has a fatal flaw for example, it's likely a patch would be implemented. To take such a firm line against all forms of improvement is far too aggressive.

In my opinion, patching should not be considered. History has proven that the tweaks are ineffective, inherently centralizing, and potentially dangerous. In my opinion, it would be foolish and irresponsible to continue them in any kind of form. If RandomX fails, we should not try to apply band-aid, but rather switch to an ASIC friendly algorithm such as SHA3. Arguably, the risks of switching, at that point, are significantly less than the risks of trying to patch a failed algorithm.

I still believe you are not understanding my point, or at least you are taking the impact of a completely different type of patch and claiming it will have the same impact in this other case. I do not advocate for RandomX patches that would merely break existing ASICs, except as a potential short-term measure as Monero transitions to SHA3. This would only occur if the network is already in a severely vulnerable state as I outlined in a) in the previous comment. And even so, Monero would need to evaluate in the moment if it would be worth doing this instead of waiting it out or immediately transitioning to SHA3 without any established resources and a nonexistent market.

I instead advocate for RandomX patches in the case some vulnerability is discovered or exploited. This is not merely the case that someone made something more efficient. It's the case that the audits overlooked some critical vulnerability that can be patched. This may or may not happen with ASICs having substantial control over the network.

Only in this case do I recommend a patch to update RandomX to be useful again. I feel that this action will have fewer consequences than another "tweak" in the sense we experience them now. Most notably, we are not committing ourselves to "tweak" it again in the future. It instead falls in line with any other critical vulnerability patch, including any needed patches for SHA3, RingCT, or anything else related to Monero. If Monero switches to SHA3, for example, we should expect that Monero will patch any related vulnerabilities if they arise.

[dEBRUYNE-1]

Your initial comment was rather ambiguous with respect to what a patch meant. In light of this debate, it was logical to assume it meant another tweak to the RandomX to either curb ASICs or to further try to strengthen (emphasis on try) ASIC resistance. My previous comment was a response to that notion. If you want to properly discuss this, you should be more clear.

Code bugs should be patched, sure, but I don't think they are particularly relevant in this discussion.

[stoffu]

I imagine that when RandomX ASICs finally appear and dominate mining, someone will start claiming "RandomX was missing XYZ (e.g. IO-hard) aspect which is essential for beating ASICs. We should try RandomX 2.0 with XYZ!", to which the community would respond "We're sold, why not give it a try."

ASIC resistance is such a pipe dream that is hard to give up. Let's see if my prediction becomes true.

[OrvilleRed]

The PoW algo is just a subsystem, like other subsystems.

Incremental improvements to RX would presumably be dealt with in the same way as other non-critical enhancements, and bundled in the next (semi-yearly or less) release.

Similarly, critical flaws or vulns should be patched at the earliest opportunity. This would equally be the case for a SHA3 or EthHash implementation (even the reference ones...)

That is very different from a reactive "ZOMG! Make a random RandomX tweak to brick the ASICs!"

[dEBRUYNE-1]

@denisgoddard

Incremental improvements to RX

"Improvements". They are simply tweaks and my previous post fully applies here:

#321 (comment)

would presumably be dealt with in the same way as other non-critical enhancements, and bundled in the next (semi-yearly or less) release.

They are vastly different. We cannot deal with them similarly than normal code improvements that require a consensus change.

We've got to get off this crazy train of thought where tweaks are seen as a viable strategy. I instigated this debate to get away from them and seek common ground on a long-term solution. To reiterate, history has proven that the tweaks are ineffective, inherently centralizing, and potentially dangerous.

On an administrative note, discussion regarding this subject (i.e. tweaks) is, in my opinion, already cluttering this ticket and it's merely a day old.

---

@stoffu:

The idea is to find a course of action (currently RandomX in October -> switch to an ASIC friendly algorithm (such as SHA3) in case of failure seems to be the path of least resistance) we can all reasonably agree on and to stick to that as to avoid the scenario you described.

[hyc]

@stoffu with respect, you have no idea what you're talking about. You might get a better understanding of what makes single-purpose hardware fast by reading other articles on the subject, like this one http://yosefk.com/blog/its-done-in-hardware-so-its-cheap.html

Key points:

• specialization lets you gain efficiency from removing the overhead of generic control flow
• parallelization lets you gain performance - but not efficiency

You've been spouting your skepticism for a while now, but at this point it's irresponsible of you to continue to do so without actually educating yourself on the topic.

[timolson]

@hyc

@stoffu with respect, you have no idea what you're talking about.

If we are slinging ad hominem attacks, then maybe YOU don’t know what you’re talking about. How many ASIC’s have you designed, hyc?

Recently an actual hardware designer (from Linzhi) showed up in the RandomX technical thread with some criticism of RandomX, so maybe we should listen to her not you.

Stoffu has every right to express his concerns, especially in light of recent tweaks not living up to their anti-ASIC claims.

[WhyIsThisSoSlow]

@dEBRUYNE-1 @stoffu @timolson
I do understand that you guys may be ASIC advocates that really want to see RandomX fail, and see ASIC adoption ASAP, but let's put preference aside for a moment and think logically.

There will be a need for Tweeks or Improvements to the RandomX algo, as nothing is perfect and everything can be improved.
These updates (there was one a few days ago) will make the algo function better and can be done once an year or whenever the rest of the updates go in.
In this sense when it comes to the ASIC part we should be ok with improvements that may reduce ASIC performance but do not brick them entirely.

On the other hand, critical bugs may be discovered and need to be patched ASAP. Bugs can occur on any algo, no matter if its ASIC friendly or not.

We all understand that an ASIC dominated network is the goal when they become commodity but let's not destroy years of work by rushing into it for no good reason.
There is no long-term solution when it comes to technology like there is no BAD ASIC,there is just monopoly and centralisation,thus we need to adapt and improve.

And to get back on topic i believe that if we can analyze the pattern of new HR/DAY or WEEK and compare it to profitability we for sure have valuable data. No one likes to mine at a loss for a long period of time.
There is also always the nonce analysis that can verify if the hash analysis is correct.