Add VRF nonce to the consensus. #590

Jim8y · 2021-05-18T16:50:25Z

This pull request adds VRF support to the dBFT consensus.
Proof of the VRF only exists in the PrepareRequest message.

src/DBFTPlugin/PrepareRequest.cs

shargon · 2021-05-19T08:11:49Z

src/DBFTPlugin/ConsensusService.cs

+
+            if (nonce == null)
+            {
+                Log($"Random number verification failed: {message.VRFProof}", LogLevel.Warning);


I prefer throw an error in random method instead of break the consensus

And maybe ask ChangeView.

And maybe ask ChangeView.

I just follow the way how you process other check failures. If you think ChangeView is better, I can add it here.

I prefer throw an error in random method instead of break the consensus

Updated by adding exception

src/DBFTPlugin/ConsensusContext.cs

vncoelho · 2021-05-19T13:47:20Z

src/DBFTPlugin/ConsensusContext.cs

+        private Tuple<byte[], uint> GetNonce(byte[] prikey)
+        {
+            byte[] aplha;
+            if (Block.Index > 1000)


Maybe the % with 1000?

vncoelho · 2021-05-19T13:48:06Z

src/DBFTPlugin/ConsensusService.cs

+            try
+            {
+                if (message.BlockIndex > 1000)
+                    // To prevent the primary uses the same block hash in Height (1000, 2000)


I did not understand this comment.

I want to use the 1000th ahead block hash as the current VRF input to generate the random number. However, if the block index is less than 1000, there will be out of index error. So I have to make the first 1000 block to use the prevHash as VRF input. The problem here is, if the first 1000 block uses prevHash, and Block of 1000 - 2000 use the hash of 1000th ahead block, it is possible that the same primary use the same block hash to generate random numbers, which means there might have a same random number, one among (0..1000), another among (1000, 2000).

Co-authored-by: Shargon <shargon@gmail.com>

shargon · 2021-05-26T11:04:58Z

src/DBFTPlugin/ConsensusService.cs

+            byte[] nonce;
+            try
+            {
+                nonce = VRF.Verify(context.Validators[message.ValidatorIndex], message.VRFProof, message.PrevHash);


We should add the current merkletree in data

If we add the current Merkle tree in the data, the result would be mutable for the primary cause he can decide the Merkle tree.

he can decide the Merkle tree.

But not the PrevHash, it makes the things more complicated

Using the block hash is a normal and the most easy way to get a well known input. Currently, almost all projects that use VRF take block hash as input, either prehash or previous n blocks. But if u have a better input source, definitely it is a good thing.

src/DBFTPlugin/ConsensusContext.cs

nicolegys · 2021-05-31T08:44:07Z

Sometimes run into an exception.
31bytes

nicolegys · 2021-05-31T10:52:46Z

@Liaojinghui I think here's why

nicolegys · 2021-06-01T02:41:55Z

src/DBFTPlugin/VRF.cs

+            var kBytes = k.ToByteArray(true, true);
+
+            // Step 6: c = ECVRF_hash_points(H, Gamma, k*B, k*H)
+            var u_point = DerivePubkeyPoint(k.ToByteArray(true, true));
+            var v_point = h_point * kBytes;


Fixed the error, the issue is neo added extra length check to the prikey @nicolegys

We should append leading zeros to kBytes after var kBytes = k.ToByteArray(true, true) in Prove method, because the operator * requires the second param to be 32 bits.

@Liaojinghui I think you've forgotten this issue 0.0

@Liaojinghui I think you've forgotten this issue 0.0

My fault TT

nicolegys · 2021-06-02T10:26:18Z

I sent a tx to extract neo from the CNs' multi-sig address, then the CNs threw exception and stop generating blocks, and cannot recover.

Jim8y · 2021-06-02T10:53:20Z

Fixed, I changed the index of nonce_tx in the txs, and missed checking its verification code. @nicolegys
Can we have a test project for the dbft~

nicolegys · 2021-06-03T07:11:01Z

Fixed, I changed the index of nonce_tx in the txs, and missed checking its verification code. @nicolegys
Can we have a test project for the dbft~

I used the newest code, and the issue still existed. TwT

if the block has no transaction, no need to verify the nonce_tx.

nicolegys · 2021-06-07T07:36:06Z

Failed to generate block_1 in multiple CN mode.
Cause:
The primary (index=1) create a nonce_tx but it has no gas to pay for the network fee, then throw exception Insufficient GAS.

nicolegys · 2021-06-08T02:24:19Z

If view changed, tx.Nonce is still the one created by old primary, but the backup CNs receive the new nonce, so CheckNonce will fail.

nicolegys

No nonce_tx now.

nicolegys · 2021-06-08T14:24:05Z

src/DBFTPlugin/ConsensusContext.cs

@@ -342,7 +354,8 @@ internal void EnsureMaxBlockLimitation(IEnumerable<Transaction> txs)
            uint maxTransactionsPerBlock = neoSystem.Settings.MaxTransactionsPerBlock;

            // Limit Speaker proposal to the limit `MaxTransactionsPerBlock` or all available transactions of the mempool
-            txs = txs.Take((int)maxTransactionsPerBlock);
+            txs = txs.Take((int)maxTransactionsPerBlock - 1);


Suggested change

txs = txs.Take((int)maxTransactionsPerBlock - 1);

txs = txs.Take((int)maxTransactionsPerBlock);

nicolegys · 2021-06-09T08:38:43Z

Could you please also update HeaderFromJson in RpcClient?
https://github.com/Liaojinghui/neo-modules/blob/master/src/RpcClient/Utility.cs#L137

nicolegys · 2021-06-10T09:51:46Z

src/RpcClient/Utility.cs

@@ -142,7 +142,7 @@ public static Header HeaderFromJson(JObject json, ProtocolSettings protocolSetti
                PrevHash = UInt256.Parse(json["previousblockhash"].AsString()),
                MerkleRoot = UInt256.Parse(json["merkleroot"].AsString()),
                Timestamp = (ulong)json["time"].AsNumber(),
-                Nonce = (ulong)json["nonce"].AsNumber(),
+                Nonce = UInt64.Parse(json["nonce"].AsString()),


I think you should useUInt64.Parse(json["nonce"].AsString(), NumberStyles.AllowHexSpecifier) to convert a hex string to uint64.

…i/neo-modules * 'master' of https://github.com/Liaojinghui/neo-modules: fix hex_str to uint64 * 'master' of https://github.com/Liaojinghui/neo-modules: fix hex_str to uint64

Jim8y added 5 commits May 18, 2021 12:47

Add VRF nonce to the consensus.

246ebf0

remove unused code

9674f12

Add vrf proof to MakeRecoveryMessage and Serialize

7ccd167

add comment

fcc06ea

Update the feed of VRF

24a0fa6

shargon reviewed May 19, 2021

View reviewed changes

src/DBFTPlugin/PrepareRequest.cs Outdated Show resolved Hide resolved

shargon reviewed May 19, 2021

View reviewed changes

src/DBFTPlugin/ConsensusContext.cs Outdated Show resolved Hide resolved

shargon and others added 4 commits May 19, 2021 10:12

Clean enter

0b504c3

Prevent the same primary using the same prevHash in Block 1000..2000

f14c1a9

update comment error

d5d00d2

throw exception when VRF verification fails

7f32ad4

vncoelho reviewed May 19, 2021

View reviewed changes

Jim8y and others added 8 commits May 19, 2021 10:38

Typo correction

fb72197

Update src/DBFTPlugin/PrepareRequest.cs

bf37e14

Co-authored-by: Shargon <shargon@gmail.com>

Simplify the code

d4f5033

use prevHash directly

151acbb

move VRF from neo core to plugin

44854ec

convert Uint256 to Array

9579c48

Move nonce to a transaction

ba8a045

Broadcat the nonce_tx and verify nonce

f3c3217

shargon reviewed May 26, 2021

View reviewed changes

Jim8y added 2 commits May 26, 2021 22:07

remove nonce from size

fb788f5

insert nonce_tx at the begining of txs

7f447c9

nicolegys suggested changes May 28, 2021

View reviewed changes

src/DBFTPlugin/ConsensusContext.cs Outdated Show resolved Hide resolved

Update the nonce_tx sender

162a629

fix length error

79c5b93

nicolegys reviewed Jun 1, 2021

View reviewed changes

fix exception error

4ac12fb

jinqiao0423 and others added 6 commits June 3, 2021 21:42

fix tx fee

26f21e7

update createNonceTransaction

2873847

fix the prove kbytes length error

1884d71

Update VRF.cs

2e15b0b

Update ConsensusService.cs

329e18d

if the block has no transaction, no need to verify the nonce_tx.

nonce_tx size check

f708b44

Update ConsensusContext.cs

c37d90e

Jim8y added 3 commits June 8, 2021 19:32

add nonce to header

fc21ae1

set nonce ulong

265ec38

on view change

65a964b

nicolegys reviewed Jun 8, 2021

View reviewed changes

maxTransactionsPerBlock

28f2014

Jim8y added 3 commits June 9, 2021 16:58

add nonce to HeaderFromJson

fa00ae0

remove reference

81401b7

ulong to hexstring in ToJson

a4f3791

nicolegys reviewed Jun 10, 2021

View reviewed changes

nicolegys and others added 3 commits June 10, 2021 19:13

fix hex_str to uint64

b7636cf

fix format

d87307a

Merge branches 'master' and 'master' of https://github.com/Liaojinghu…

3fdf6e4

…i/neo-modules * 'master' of https://github.com/Liaojinghui/neo-modules: fix hex_str to uint64 * 'master' of https://github.com/Liaojinghui/neo-modules: fix hex_str to uint64

Jim8y mentioned this pull request Jun 10, 2021

Add Random number to N3 #596

Merged

Jim8y closed this Jun 10, 2021

Qiao-Jin mentioned this pull request Aug 27, 2021

BLS Consensus Structure #637

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add VRF nonce to the consensus. #590

Add VRF nonce to the consensus. #590

Jim8y commented May 18, 2021 •

edited

Loading

shargon May 19, 2021

vncoelho May 19, 2021

Jim8y May 20, 2021

Jim8y May 20, 2021

vncoelho May 19, 2021

vncoelho May 19, 2021

Jim8y May 19, 2021

shargon May 26, 2021

Jim8y May 26, 2021

shargon May 27, 2021

Jim8y May 27, 2021

nicolegys commented May 31, 2021

nicolegys commented May 31, 2021

nicolegys Jun 1, 2021

nicolegys Jun 3, 2021

Jim8y Jun 5, 2021

nicolegys commented Jun 2, 2021

Jim8y commented Jun 2, 2021 •

edited

Loading

nicolegys commented Jun 3, 2021

nicolegys commented Jun 7, 2021

nicolegys commented Jun 8, 2021

nicolegys left a comment

nicolegys Jun 8, 2021

nicolegys commented Jun 9, 2021

nicolegys Jun 10, 2021

	txs = txs.Take((int)maxTransactionsPerBlock - 1);
	txs = txs.Take((int)maxTransactionsPerBlock);

Add VRF nonce to the consensus. #590

Add VRF nonce to the consensus. #590

Conversation

Jim8y commented May 18, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nicolegys commented May 31, 2021

nicolegys commented May 31, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nicolegys commented Jun 2, 2021

Jim8y commented Jun 2, 2021 • edited Loading

nicolegys commented Jun 3, 2021

nicolegys commented Jun 7, 2021

nicolegys commented Jun 8, 2021

nicolegys left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nicolegys commented Jun 9, 2021

Choose a reason for hiding this comment

Jim8y commented May 18, 2021 •

edited

Loading

Jim8y commented Jun 2, 2021 •

edited

Loading