Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Notary functions in SMC to align with phase 1.1 spec #93

Merged
merged 27 commits into from
Apr 28, 2018
Merged

Notary functions in SMC to align with phase 1.1 spec #93

merged 27 commits into from
Apr 28, 2018

Conversation

terencechain
Copy link
Member

@terencechain terencechain commented Apr 19, 2018

This PR is based off of #78. I've made the following SMC modifications to align with phase 1.1 spec:

  • notary register/deregister/release functions
  • notary update sample size function
  • notary is in committee function
  • test cases:
    • register notary
    • register without enough deposit (negative test case)
    • register then re-register (negative test case)
    • deregister notary
    • deregister then old notary register
    • release notary
    • release notary before lockup period ends (negative test case)
    • notary committee list is different for every shard
    • is notary in committee with non-registered notary (negative test case)
    • is notary in committee with deregistered notary (negative test case)

References:
SMC phase 1.1 spec
Notary update sample size spec
Pyevm's SMC design (vyper)

@terencechain terencechain changed the title notary functions i SMC to align with phase 1.1 spec wip: notary functions in SMC to align with phase 1.1 spec Apr 19, 2018
struct CollationHeader {
uint256 shard_id; // pointer to shard
bytes32 parent_hash; // pointer to parent header
bytes32 chunk_root; // pointer to collation body

This comment was marked as resolved.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is it chunks root? That doesn't sound correct

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pointer? wouldn't this just be the root hash?

@rauljordan why plural?

mapping (address => Notary) public notary_registry;
mapping (address => Proposer) public proposer_registry;
// shard_id => (header_hash => tree root)
mapping (uint => mapping (bytes32 => bytes32)) public collation_trees;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you elaborate a bit more on this mapping?

uint constant depositSize = 1000 ether;
uint constant PERIOD_LENGTH = 5;
// Exact collation body size (1mb)
uint constant COLLATION_SIZE = 2 ** 20;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need this in the SMC? I don't this var think is relevant


// Log the latest period number of the shard
mapping (int => int) public periodHead;
mapping (int => int) public period_head;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought the best practice in solidity was camel case? Not sure though.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.


// Uses a block hash as a seed to pseudorandomly select a signer from the collator pool.
// [TODO] Chance of being selected should be proportional to the collator's deposit.
event LOG(uint L);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This event is just called LOG?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It should be more descriptive, LOG is very vague. also naming the variable L is very vague again should be more descriptive

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed. This was from the old SMC code. I'll check with @enriquefynn on the purpose of this LOG event

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It has no any usage, was using it to debug, you can remove it

var receiptId = numReceipts;
++numReceipts;
// Removes the notary from the notary pool and sets deregistered period
function deregister_notary() public returns(bool) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Make these funcs all camelCase plz

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

int numReceipts;
// Indexs of empty slots caused by the function `withdraw`
mapping (int => int) emptySlotsStack;
mapping (int => bytes32) shardead;

This comment was marked as resolved.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

shard_head

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I mean shardHead

@enriquefynn
Copy link
Contributor

Why those functions return true at the end? I don't think is particularly useful, since false is never returned in the function.

We should vote for camel case, like normally in solidity code or underscore, used in Vyper. Since we're not using any library, I think it's a free choice.

@prestonvanloon prestonvanloon self-requested a review April 19, 2018 13:31
// mapping (int => Notary) public notaries;
address[] public notary_pool;
// proposerId => Proposer
// mapping (int => Proposer) public proposers;

This comment was marked as resolved.

// mapping (int => Proposer) public proposers;
Proposer[] public proposer_pool;

// Notary registry (deregistered is 0 for not yet deregistered notaries)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What does this mean "deregistered is 0"? If you are saying that an address will return 0 because the notary is not in the mapping then you can omit this comment because that is common sense.


// Log the latest period number of the shard
mapping (int => int) public periodHead;
mapping (int => int) public period_head;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

int128 period; // Period which header should be included
int128 height; // Collation's height
address proposer_address; // Proposer's address
uint256 proposer_bid; // Proposer's bid

This comment was marked as resolved.

// Shard id of the deposit
uint shardId;
// Deposit in ether in each shard
uint[] deposit;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should this be a mapping? Why is it an array? How is this supposed to work?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it should just be a mapping of shard_Id => deposit

require(!notary_registry[notary_address].deposited);
require(msg.value == NOTARY_DEPOSIT);

// update notary_sample_size

This comment was marked as resolved.

update_notary_sample_size();

uint index;
if (!empty_stack()) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you invert this if statement so you don't have to negate the empty_stack()?

if (empty_stack()) {
  // do thing to empty stack
} else {
  // do thing to non-empty stack
}

index = stack_pop();
notary_pool[index] = notary_address;
}
else {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Formatting:
should be

} else { 

// Update the latest period number
periodHead[_shardId] = int(_expectedPeriodNumber);
notary_registry[notary_address] = Notary({
deregistered: 0,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

deregistered 0? this doesn't make much sense, wouldn't it be false?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's period. Changed it to deregisteredPeriod

var receiptId = numReceipts;
++numReceipts;
// Removes the notary from the notary pool and sets deregistered period
function deregister_notary() public returns(bool) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

bytes32 parentHash; // Hash of the parent collation
bytes32 chunkRoot; // Root hash of the collation body
uint period; // Period which header should be included
uint height; // Height of the collation
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you remove this field? If we need it, please elaborate on what it means (the tree height since genesis)

PROPOSER_LOCKUP_LENGTH = proposer_lockup_length;
}

event LOG(uint L);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's remove this please

event LOG(uint L);


function isNotaryInCommittee(uint shardId, uint period) public view returns(bool) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add function comments

sampleSize = currentPeriodNotarySampleSize;
}

uint periodToLook = period * PERIOD_LENGTH - 1;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is this? Looks like you convert the period to a block number then subtract one?
But the variable is called "periodToLook". Consider renaming this so it is easier to understand.

Also consider adding a comment to explain where you are looking "Check the most recent block before the period started" or something like that.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah! good catch! it should be block


uint periodToLook = period * PERIOD_LENGTH - 1;

for (uint i = 1; i <= QUORUM_SIZE; i++) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Personal preference:
Please start with i = 0

for (uint i = 0; i < QUORUM_SIZE; i++) {

function updateGasPrice(int _receiptId, uint _txGasprice) public payable returns(bool) {
require(receipts[_receiptId].sender == msg.sender);
receipts[_receiptId].txGasprice = _txGasprice;
function updateNotarySampleSize() private returns(bool) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Function comment please

require(notaryRegistry[notaryAddress].deposited);
require(notaryPool[index] == notaryAddress);

updateNotarySampleSize();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we need to do this as part of deregistering?

TxToShard(_to, _shardId, receiptId);
return receiptId;
delete notaryRegistry[notaryAddress];
notaryAddress.transfer(NOTARY_DEPOSIT);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't we use .send and check that the call was successful?
Seems like good practice, but I am not sure

uint constant LOOKAHEAD_LENGTH = 4;
// Number of notaries to select from notary pool for each shard in each period
uint constant COMMITTEE_SIZE = 135;
// Threshold(number of notaries in committee) for a proposal to be deem accepted
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Typo remove "deem"


// Number of notaries
uint public notaryPoolLength;
// Indexes of empty slots caused by the function `withdraw`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we still need this now that the withdraw function is gone?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still need it to track empty notary slot indices (in register/deregister function)
The comment is mislead, I will change it

// Log the latest period number of the shard
mapping (int => int) public periodHead;

function isNotaryInCommittee(uint shardId, uint period) public view returns(bool) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add function level comments

(notaryRegistry[notaryAddress].deregisteredPeriod + NOTARY_LOCKUP_LENGTH));

delete notaryRegistry[notaryAddress];
notaryAddress.transfer(NOTARY_DEPOSIT);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't we use .send and check that the call was successful?
Seems like good practice, but I am not sure

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

.transfer() throws on failure, it's used in most cases as it's best practice and safest way to send ether
.send() returns false on failure, it's only used in rare cases when you want to handle failure in the contract

require(!notaryRegistry[notaryAddress].deposited);
require(msg.value == NOTARY_DEPOSIT);

updateNotarySampleSize();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we do this? Can you add a comment so that a reader can understand?

require(notaryRegistry[notaryAddress].deposited);
require(notaryPool[index] == notaryAddress);

updateNotarySampleSize();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we do this? Can you add comment so the reader understands?

activateCollatorNum += 1;
function computeHeaderHash(uint256 shardId, bytes32 parentHash,
bytes32 chunkRoot, uint256 period, address proposerAddress)
public returns(bytes32) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Formatting.
Please format each argument on another line.

https://solidity.readthedocs.io/en/v0.3.1/style-guide.html#function-declaration

}

function addHeader(uint _shardId, uint period, bytes32 chunkRoot,
address proposerAddress) public returns(bool) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Formatting.
Please format each argument on another line.

https://solidity.readthedocs.io/en/v0.3.1/style-guide.html#function-declaration

require(notaryRegistry[notaryAddress].deposited == true);
require(notaryRegistry[notaryAddress].deregisteredPeriod != 0);
require((block.number / PERIOD_LENGTH) >
(notaryRegistry[notaryAddress].deregisteredPeriod + NOTARY_LOCKUP_LENGTH));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Formatting

@terencechain terencechain changed the title wip: notary functions in SMC to align with phase 1.1 spec Notary functions in SMC to align with phase 1.1 spec Apr 24, 2018
@@ -87,14 +87,9 @@ contract SMC {

// Get the most recent block number before the period started
uint latestBlock = period * PERIOD_LENGTH - 1;
uint index = uint(keccak256(block.blockhash(latestBlock), _index, shardId)) % sampleSize;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it possible to make this line a bit simpler? or abstract away with variables?

txOpts[i].Value = notaryDeposit
genesis[notaryPoolAddr[i]] = core.GenesisAccount{
Balance: accountBalance2000Eth,
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add a few comments to this section for readers to easily understand

Copy link
Member

@prestonvanloon prestonvanloon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we're missing a big piece of refactoring where we need to rename "collator" to "notary" . That might need to be in the next pull request to keep this one small enough to review.

return err
}

// If output is non-empty and the addr == coinbase
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This comment is not accurate to the code below. Please update

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good, Will fix it.

for s := int64(0); s < sharding.ShardCount; s++ {
// Checks if we are an eligible collator according to the SMC
addr, err := c.SMCCaller().GetEligibleCollator(&bind.CallOpts{}, big.NewInt(s), period)
for i := int64(0); i < sharding.NotaryCommitSize; s++ {
addr, err := c.SMCCaller().GetNotaryInCommittee(&bind.CallOpts{}, big.NewInt(s), big.NewInt(i))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is going on in this method?
According to the method name and comment, this was supposed to check if we are an eligible collator, but it is checking to see if we are a notary?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's checking if we are a notary that's in the committee for the given shard and index

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My concerns about this are addressed in #96

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in #96

@@ -85,11 +86,11 @@ func checkSMCForCollator(c client.Client, head *types.Header) error {
func isAccountInCollatorPool(c client.Client) (bool, error) {
account := c.Account()
// Checks if our deposit has gone through according to the SMC
b, err := c.SMCCaller().IsCollatorDeposited(&bind.CallOpts{}, account.Address)
if !b && err != nil {
b, err := c.SMCCaller().NotaryRegistry(&bind.CallOpts{}, account.Address)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Again, this is checking notary registry. Are there no more collators?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No. There's no more collator. Collator has became Notary

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in #96


/// Registers notary to notatery registry, locks in the notary deposit,
/// and returns true on success
function registerNotary() public payable returns(bool) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we return a boolean? Wouldn't the transaction fail if there was an error?
Is this standard practice for solidity?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm following the latest spec and py-evm's implementation: https://ethresear.ch/t/minimal-sharding-protocol-extension/1823. We can remove it if there's a strong consensus of doing so

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's remove it if we aren't doing anything with it.

backend := backends.NewSimulatedBackend(genesis)
_, _, smc, _ := deploySMCContract(backend, notaryPoolPrivKeys[0])

// Test notary 0 has not registered
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What does this mean? You are testing that notary 0 has not been registered?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. it's a negative test case where notary 0's not registered

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think my confusion is coming from the word "test". Can you make this say "Notary 0 has not be registered"?


// Verify notary has saved the deregistered period as: current block number / period length
notary, _ = smc.NotaryRegistry(&bind.CallOpts{}, addr)
currentPeriod := big.NewInt(int64(100 / sharding.PeriodLength))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Magic number again. What is the significance of 100?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It can be any number. We want to check notary.DeregisteredPeriod has set correct. Using 100 blocks for example, the notary.DeregisteredPeriod should be set as 100/5 = 20

}

// Fast forward 10 blocks
for i := 0; i < 10; i++ {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Magic number. Same comment as above.

if depositsEventsIterator.Event.Index.Cmp(big.NewInt(0)) != 0 {
t.Fatalf("Collator index mismatch: %d should be 0", depositsEventsIterator.Event.Index)

balance, err := backend.BalanceAt(ctx, addr, nil)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

error check?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok will fix.

t.Fatalf("Collator address mismatch: %x should be %x", depositsEventsIterator.Event.Collator, addr)

if notary.Deposited {
t.Fatalf("Notary deposit flag should be false after released")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this can be t.Errorf(...). The general rule of go tests is to go as far as you can before calling Fatal. In this case, I think the test can proceed to the next part where the balance is checked. Please take a look at the test and see if there are other areas where you could call t.Errorf rather than t.Fatal because the test could still continue (although would likely fail for the other conditions). It would be nice to see many errors, if there were many.

transactOpts.Value = collatorDeposit
smc.Deposit(transactOpts)
// Fast forward 10 blocks
for i := 0; i < 10; i++ {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Magic number comment. Why are you doing this?

Copy link
Member

@prestonvanloon prestonvanloon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is wonderful. Thanks @terenc3t!

Just 1 major concern: Please remove the SMC methods return (boolean) as it serves no purpose.

@@ -85,11 +86,11 @@ func checkSMCForCollator(c client.Client, head *types.Header) error {
func isAccountInCollatorPool(c client.Client) (bool, error) {
account := c.Account()
// Checks if our deposit has gone through according to the SMC
b, err := c.SMCCaller().IsCollatorDeposited(&bind.CallOpts{}, account.Address)
if !b && err != nil {
b, err := c.SMCCaller().NotaryRegistry(&bind.CallOpts{}, account.Address)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in #96

for s := int64(0); s < sharding.ShardCount; s++ {
// Checks if we are an eligible collator according to the SMC
addr, err := c.SMCCaller().GetEligibleCollator(&bind.CallOpts{}, big.NewInt(s), period)
for i := int64(0); i < sharding.NotaryCommitSize; s++ {
addr, err := c.SMCCaller().GetNotaryInCommittee(&bind.CallOpts{}, big.NewInt(s), big.NewInt(i))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in #96

backend := backends.NewSimulatedBackend(genesis)
_, _, smc, _ := deploySMCContract(backend, notaryPoolPrivKeys[0])

// Test notary 0 has not registered
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think my confusion is coming from the word "test". Can you make this say "Notary 0 has not be registered"?

@terencechain terencechain self-assigned this Apr 27, 2018
@terencechain terencechain merged commit 2922eca into prysmaticlabs:master Apr 28, 2018
@terencechain terencechain deleted the smc_phase1.1 branch April 28, 2018 00:07
@rauljordan rauljordan added this to To do in Sharding Manager Contract via automation May 1, 2018
@rauljordan rauljordan added this to the Ruby milestone May 1, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
No open projects
Development

Successfully merging this pull request may close these issues.

None yet

5 participants