Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implements ShardLayout and ShardConfig #4614

Merged
merged 27 commits into from
Aug 14, 2021
Merged

Implements ShardLayout and ShardConfig #4614

merged 27 commits into from
Aug 14, 2021

Conversation

mzhangmzz
Copy link
Contributor

Part of NEP near/NEPs#241

@mzhangmzz
Copy link
Contributor Author

Still adding tests

@mzhangmzz mzhangmzz marked this pull request as ready for review August 3, 2021 20:10
core/primitives/src/epoch_manager.rs Outdated Show resolved Hide resolved
core/primitives/src/epoch_manager.rs Outdated Show resolved Hide resolved
core/primitives/src/shard_layout.rs Outdated Show resolved Hide resolved
core/primitives/src/shard_layout.rs Outdated Show resolved Hide resolved
core/primitives/src/shard_layout.rs Outdated Show resolved Hide resolved
Comment on lines +74 to +76
if is_top_level_account(fixed_account, account_id) {
return shard_id as ShardId;
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am a bit confused by this. Could you explain the motivation here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The comments are in ShardLayoutV1.

For this shard layout, there are some fixed shards, each fixed shard is mapped to an account and all subaccounts of this account. The rest of accounts are divided by boundary_accounts. For example, For ShardLayoutV1{ fixed_shards: ["aurora"], boundary_accounts: ["near"], }
Account "aurora" and all its subaccounts will be mapped to shard_id 0. For the rest of accounts, accounts <= "near" will be mapped to shard_id 1 and accounts > "near" will be mapped shard_id 2.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Account "aurora" and all its subaccounts will be mapped to shard_id 0

Is this actually what we want? Also adding a comment here will be helpful :)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry I missed this comment.

I thought this is what we want, but maybe I misunderstood something. Is you question here regarding whether we should map all subaccounts of "aurora" to shard_id 0? I'll add a comment here.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is you question here regarding whether we should map all subaccounts of "aurora" to shard_id 0?

Yes. I don't think we've agreed on that. Might require more discussion. @djsatok is there plan to create aurora subaccounts? If so, is it important that they are on the same shard with aurora?

nearcore/src/shard_tracker.rs Outdated Show resolved Hide resolved
@frol
Copy link
Collaborator

frol commented Aug 10, 2021

I don't have enough context about sharding, so I cannot review this PR properly. Let me know if github requires my review as a codeowner of some files, but the decision whether the implemented sharding mechanics make sense should be made before that.

(epoch_info.protocol_version(), epoch_info.seat_price())
};
let config = self.config.for_protocol_version(protocol_version);
let stake_divisor = { config.minimum_stake_divisor as Balance };
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit not related to this pr, but divisor being Balance seems weird - if we divide near by near, we will have just a number, not something measured in near

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah might be better to write as u128 instead

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll do that in a separate PR

@mzhangmzz
Copy link
Contributor Author

I don't have enough context about sharding, so I cannot review this PR properly. Let me know if github requires my review as a codeowner of some files, but the decision whether the implemented sharding mechanics make sense should be made before that.

@frol, yes the implemented sharding mechanics in this PR is recorded in NEP near/NEPs#241. See section protocol change and protocol-level sharding representation. You don't have to read it of course, but in case you are interested.

Copy link
Collaborator

@bowenwang1996 bowenwang1996 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving to unblock, but please address the comments

(epoch_info.protocol_version(), epoch_info.seat_price())
};
let config = self.config.for_protocol_version(protocol_version);
let stake_divisor = { config.minimum_stake_divisor as Balance };
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah might be better to write as u128 instead

core/primitives/src/epoch_manager.rs Outdated Show resolved Hide resolved
Comment on lines +74 to +84
let mut config = genesis_epoch_config.clone();
if let Some(ShardConfig {
num_block_producer_seats_per_shard,
avg_hidden_validator_seats_per_shard,
shard_layout,
}) = simple_nightshade_shard_config
{
config.num_block_producer_seats_per_shard = num_block_producer_seats_per_shard;
config.avg_hidden_validator_seats_per_shard = avg_hidden_validator_seats_per_shard;
config.shard_layout = shard_layout;
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
let mut config = genesis_epoch_config.clone();
if let Some(ShardConfig {
num_block_producer_seats_per_shard,
avg_hidden_validator_seats_per_shard,
shard_layout,
}) = simple_nightshade_shard_config
{
config.num_block_producer_seats_per_shard = num_block_producer_seats_per_shard;
config.avg_hidden_validator_seats_per_shard = avg_hidden_validator_seats_per_shard;
config.shard_layout = shard_layout;
}
let config = match simple_nightshade_shard_config {
Some(config) => config.clone(),
None => genesis_epoch_config.clone()
};

Copy link
Contributor Author

@mzhangmzz mzhangmzz Aug 12, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ShardConfig is different from EpochConfig here. simple_nightshade_shard_config is a ShardConfig and config is a EpochConfig

@mzhangmzz
Copy link
Contributor Author

Approving to unblock, but please address the comments

@bowenwang1996 thanks! Will address all comments before I merge the PR

@bowenwang1996
Copy link
Collaborator

@mzhangmzz are we ready to merge this one?

@mzhangmzz
Copy link
Contributor Author

@mzhangmzz are we ready to merge this one?

yes, merge now

@mzhangmzz mzhangmzz merged commit df2a421 into master Aug 14, 2021
@mzhangmzz mzhangmzz deleted the shardlayout branch August 14, 2021 18:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants