Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merge dev into feat-2.0 #4228

Merged
merged 77 commits into from
Aug 30, 2023
Merged

Conversation

rafal-ch
Copy link
Contributor

@rafal-ch rafal-ch commented Aug 22, 2023

Bring the recent changes from dev (mainly, the private chain support and fix for the slow historical sync) into feat-2.0.

Michał Papierski and others added 30 commits June 14, 2022 22:23
 Conflicts:
	execution_engine/src/core/runtime/auction_internal.rs
	execution_engine/src/system/auction.rs
	execution_engine/src/system/auction/detail.rs
	execution_engine/src/system/auction/providers.rs
	types/src/system/auction/unbonding_purse.rs
Once a contract is disabled it can't be called unless it's enabled again
with this new API.
3106: Initial support for private chain r=mpapierski a=mpapierski

This is backport of original casper-network#3029 on top of v1.4.6 with least amount of extra backported changes to make the original code from PR unchanged.


This PR brings preliminary support for private chains by introducing multiple chainspec configuration flags that change the behavior of a chain.

These settings are orthogonal to each other; any particular combination (and all other pre-existing settings) is valid for a given network, for a wide variety of possible permutations. 

- Administrator accounts: by specifying accounts in a `[[administrators]]` section of `accounts.toml` accounts with special privileges are created and later referred to as 'administrative accounts'
- `core.allow_auction_bids` disables add_bid, delegate entrypoints in auction contract if set to true
- `core.compute_rewards` rewards 0 tokens after each switch block. This means no new tokens are minted and validators aren't gaining more weight.
- `core.allow_unrestricted_transfers` if set to false, the normal user can't send tokens to other normal users; the normal user can transfer only to the administrator; either source or target has to be an admin.
- `core.refund_handling` defines how refunds are computed. 
  - `refund_handling = { type = "refund", refund_ratio = [1, 1] }` means 100% of `payment_amount - gas_spent` is refunded. 
  - `refund_handling = { type = "refund", refund_ratio = [0, 100 ] }` doesn't refund anything
  - `Refund` mode requires that the fraction specified is proper fraction (that is `refund_ratio <= 1`)
- `core.fee_handling` defines how gas spent is processed after deploy is executed.
  - `fee_handling = { type = "accumulate" }` creates special rewards purse in the mint system contract where fees are accumulated into
  - `fee_handling = { type = "pay_to_proposer" }` is a default setting for a public chain - rewards are transferred to block's proposer.
  - `fee_handling = { type = "burn" }` fees are burned.


Co-authored-by: Michał Papierski <michal@casperlabs.io>
 Conflicts:
	Cargo.lock
	execution_engine_testing/tests/src/test/regression/gov_74.rs
	execution_engine_testing/tests/src/wasm_utils.rs
	types/src/system/mod.rs
Squashed commit of the following:

commit 04a0557
Author: Michał Papierski <michal@casperlabs.io>
Date:   Tue Aug 30 15:49:52 2022 +0200

    Remove ca_cert from the identity struct.

commit 8491fed
Author: Michał Papierski <michal@casperlabs.io>
Date:   Tue Aug 30 15:14:02 2022 +0200

    Remove outdated TODO comment

commit 6060eed
Author: Michał Papierski <michal@casperlabs.io>
Date:   Tue Aug 30 15:12:17 2022 +0200

    Apply @marc-casperlabs comments

commit 2f04a17
Author: Michał Papierski <michal@casperlabs.io>
Date:   Thu Aug 25 17:20:51 2022 +0200

    Remove unnecessary arg.

commit a158903
Author: Michał Papierski <michal@casperlabs.io>
Date:   Thu Aug 25 16:19:23 2022 +0200

    Simplify CA validation.

    This does not enforce ECC algorithm used, only basic expiration dates
    etc.

commit b98f083
Author: Michał Papierski <michal@casperlabs.io>
Date:   Thu Aug 25 14:00:25 2022 +0200

    Use #[source] instead of #[from]
3310: [private chain] Backport 1.4.8 fixes and support for CA signed client certificates  r=mpapierski a=mpapierski

This PR contains updates `feat-private-chain` for a new private chain release independent of 1.5.0:

- Merge of v1.4.8 security release
- Backport of casper-network#3291

After this PR is merged, it should be possible to release `private-1.4.8`.

Co-authored-by: Michał Papierski <michal@casperlabs.io>
Co-authored-by: Joe Sacher <321623+sacherjj@users.noreply.github.com>
Co-authored-by: Fraser Hutchison <fraser@casperlabs.io>
Co-authored-by: casperlabs-bors-ng[bot] <82463608+casperlabs-bors-ng[bot]@users.noreply.github.com>
 Conflicts:
	Cargo.lock
	execution_engine/src/core/engine_state/engine_config.rs
	execution_engine/src/core/engine_state/error.rs
	execution_engine/src/core/engine_state/genesis.rs
	execution_engine/src/core/engine_state/mod.rs
	execution_engine/src/core/engine_state/upgrade.rs
	execution_engine/src/core/execution/error.rs
	execution_engine/src/core/resolvers/v1_function_index.rs
	execution_engine/src/core/resolvers/v1_resolver.rs
	execution_engine/src/core/runtime/auction_internal.rs
	execution_engine/src/core/runtime/externals.rs
	execution_engine/src/core/runtime/handle_payment_internal.rs
	execution_engine/src/core/runtime/mint_internal.rs
	execution_engine/src/core/runtime/mod.rs
	execution_engine/src/core/runtime/scoped_instrumenter.rs
	execution_engine/src/core/runtime/standard_payment_internal.rs
	execution_engine/src/core/tracking_copy/ext.rs
	execution_engine/src/shared/host_function_costs.rs
	execution_engine/src/system/auction.rs
	execution_engine/src/system/auction/detail.rs
	execution_engine/src/system/auction/providers.rs
	execution_engine/src/system/handle_payment.rs
	execution_engine/src/system/handle_payment/internal.rs
	execution_engine/src/system/handle_payment/runtime_provider.rs
	execution_engine/src/system/mint.rs
	execution_engine/src/system/standard_payment.rs
	execution_engine/src/system/standard_payment/mint_provider.rs
	execution_engine_testing/test_support/src/auction.rs
	execution_engine_testing/test_support/src/lib.rs
	execution_engine_testing/test_support/src/wasm_test_builder.rs
	execution_engine_testing/tests/src/lmdb_fixture.rs
	execution_engine_testing/tests/src/test/regression/ee_966.rs
	execution_engine_testing/tests/src/test/regression/gh_2280.rs
	execution_engine_testing/tests/src/test/regression/gh_2605.rs
	execution_engine_testing/tests/src/test/regression/gov_74.rs
	execution_engine_testing/tests/src/test/regression/regression_20220303.rs
	execution_engine_testing/tests/src/test/storage_costs.rs
	execution_engine_testing/tests/src/test/system_contracts/auction/bids.rs
	execution_engine_testing/tests/src/test/system_contracts/genesis.rs
	execution_engine_testing/tests/src/test/system_contracts/upgrade.rs
	execution_engine_testing/tests/src/test/system_costs.rs
	execution_engine_testing/tests/src/test/wasmless_transfer.rs
	node/Cargo.toml
	node/src/components/block_proposer/deploy_sets.rs
	node/src/components/block_proposer/tests.rs
	node/src/components/block_validator/tests.rs
	node/src/components/chainspec_loader.rs
	node/src/components/consensus/config.rs
	node/src/components/consensus/era_supervisor.rs
	node/src/components/consensus/protocols/highway.rs
	node/src/components/consensus/protocols/highway/tests.rs
	node/src/components/consensus/validator_change.rs
	node/src/components/contract_runtime.rs
	node/src/components/deploy_acceptor.rs
	node/src/components/deploy_acceptor/tests.rs
	node/src/components/event_stream_server/sse_server.rs
	node/src/components/event_stream_server/tests.rs
	node/src/components/fetcher/tests.rs
	node/src/components/gossiper/gossip_table.rs
	node/src/components/gossiper/tests.rs
	node/src/components/linear_chain/pending_signatures.rs
	node/src/components/linear_chain/signature_cache.rs
	node/src/components/linear_chain/state.rs
	node/src/components/network/config.rs
	node/src/components/network/error.rs
	node/src/components/network/tests.rs
	node/src/components/small_network.rs
	node/src/components/small_network/message.rs
	node/src/components/small_network/tasks.rs
	node/src/components/storage/tests.rs
	node/src/crypto/asymmetric_key.rs
	node/src/crypto/asymmetric_key_ext.rs
	node/src/reactor/initializer.rs
	node/src/reactor/participating/tests.rs
	node/src/testing.rs
	node/src/testing/multi_stage_test_reactor/test_chain.rs
	node/src/tls.rs
	node/src/types.rs
	node/src/types/block.rs
	node/src/types/chainspec.rs
	node/src/types/chainspec/accounts_config.rs
	node/src/types/chainspec/accounts_config/account_config.rs
	node/src/types/chainspec/accounts_config/delegator_config.rs
	node/src/types/chainspec/accounts_config/validator_config.rs
	node/src/types/chainspec/activation_point.rs
	node/src/types/chainspec/core_config.rs
	node/src/types/chainspec/deploy_config.rs
	node/src/types/chainspec/global_state_update.rs
	node/src/types/chainspec/highway_config.rs
	node/src/types/chainspec/network_config.rs
	node/src/types/chainspec/protocol_config.rs
	node/src/types/deploy.rs
	node/src/types/node_id.rs
	node/src/utils/external.rs
	resources/local/chainspec.toml.in
	resources/production/chainspec.toml
	resources/test/valid/0_9_0/chainspec.toml
	resources/test/valid/0_9_0_unordered/chainspec.toml
	resources/test/valid/1_0_0/chainspec.toml
	smart_contracts/contract/src/ext_ffi.rs
	smart_contracts/contracts_as/test/authorized-keys/assembly/index.ts
	smart_contracts/contracts_as/test/set-action-thresholds/package.json
	types/Cargo.toml
	types/src/cl_type.rs
	types/src/contracts.rs
	types/src/lib.rs
	types/src/system.rs
	types/src/system/auction.rs
	types/src/system/error.rs
	types/src/timestamp.rs
	utils/global-state-update-gen/Cargo.toml
	utils/global-state-update-gen/src/auction_utils.rs
	utils/global-state-update-gen/src/main.rs
	utils/global-state-update-gen/src/validators.rs
	utils/nctl/sh/scenarios/chainspecs/bond_its.chainspec.toml.in
	utils/nctl/sh/scenarios/chainspecs/itst13.chainspec.toml.in
	utils/nctl/sh/scenarios/chainspecs/itst14.chainspec.toml.in
	utils/nctl/sh/scenarios/chainspecs/upgrade_scenario_1.chainspec.toml.in
	utils/nctl/sh/scenarios/chainspecs/upgrade_scenario_10.chainspec.toml.in
	utils/nctl/sh/scenarios/chainspecs/upgrade_scenario_3.chainspec.toml.in
	utils/nctl/sh/scenarios/chainspecs/upgrade_scenario_4.chainspec.toml.in
	utils/nctl/sh/scenarios/chainspecs/upgrade_scenario_5.chainspec.toml.in
	utils/nctl/sh/scenarios/chainspecs/upgrade_scenario_6.chainspec.toml.in
	utils/nctl/sh/scenarios/chainspecs/upgrade_scenario_7.chainspec.toml.in
	utils/nctl/sh/scenarios/chainspecs/upgrade_scenario_8.chainspec.toml.in
	utils/nctl/sh/scenarios/chainspecs/upgrade_scenario_9.chainspec.toml.in
Before 1.5.0 there wasn't an immediate switch block committed at
genesis. Because historical sync relies on sync leaps to get the
validators for a particular era, we encounter a special case
when syncing the blocks of era 0 if they were created before 1.5.0
because sync leap will not be able to include a switch block that
has the validators for era 0.
Since the auction delay is at least 1, we can generally rely on
the fact that the validator set for era 1 and era 0 are the same.

Add a test to check if a node that has synced back to some block in
era 0 can continue syncing to genesis after it was restarted (and
lost its validator matrix).

Signed-off-by: Alexandru Sardan <alexandru@casperlabs.io>
Signed-off-by: Alexandru Sardan <alexandru@casperlabs.io>
Rename `lowest_available_block_height_on_node` to
`node_has_lowest_available_block_at_or_below_height` in order to be
consistent with the other predicates.

Signed-off-by: Alexandru Sardan <alexandru@casperlabs.io>
These were used only for development purposes to test the private chain
settings in nctl environment.
 Conflicts:
	execution_engine_testing/tests/src/test/regression/gh_3710.rs
This should make it easy for maintaining these tests.
These tests are supposed to be run manually as for now
4187: Fix node changelog r=Fraser999 a=Fraser999

This PR fixes the `casper-node` changelog, which was broken when merging `release-1.5.2` back to `dev`.

Co-authored-by: Fraser Hutchison <fraser@casperlabs.io>
rafal-ch and others added 3 commits August 25, 2023 00:19
4237: Output `wasm-strip` version r=rafal-ch a=rafal-ch

Output the `wasm-strip` version to console when building contracts. It's just a convenience for debugging purposes.

Example:
```
: ~/Casper/casper-node$ make build-contracts-rs 
Using 'wasm-strip' version 1.0.33 (git~1.0.33-25-gba73887a)
```

Co-authored-by: Rafał Chabowski <rafal@casperlabs.io>
@rafal-ch rafal-ch marked this pull request as ready for review August 25, 2023 11:55
@rafal-ch rafal-ch closed this Aug 25, 2023
@rafal-ch rafal-ch reopened this Aug 25, 2023
4205: Attempt to fix block synchronizer request storm by tweaking the latching. r=alsrdn a=alsrdn

Attempt to fix block synchronizer request storm by tweaking the latching.

Don't decrement the latch when responses for old requests that are no longer needed (are received in a state that expects other data).
For example if the synchonizer is the `HaveWeakFinality` state, it will try to fetch the block body from some peers. The latch will be incremented by `num_peers`. If a late finality signature response comes in this state it will decrement the latch. If multiple late responses come, the latch can become 0, and the synchronizer would try to fetch the block body again even if no response for the initial requests were received yet.

Added a test for this also. It's a bit ugly and copy-pasty but I think it showcases the issue.

Attempted fix for: casper-network#4202


Co-authored-by: Alexandru Sardan <alexandru@casperlabs.io>
.into_uref()
.ok_or(Error::InvalidKeyVariant)?;

// let purse_balance_key = match tracking_copy
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure why we left this block commented out. Is this coming from dev?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This check is now moved several lines down the function. It was duplicated after the merge, so I commented it out with the intention to remove it completely after making sure it's not needed. And I just forgot to remove it, which is now fixed in f0dd96a.

Copy link
Contributor

@AlexanderLimonov AlexanderLimonov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See comment on reintroducing code slated for removal

@@ -0,0 +1,337 @@
use tracing::trace;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This file should not be reintroduced, it's only relevant for "old style" rewards that are removed entirely in 2.0 for both protocols

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, good catch. File and the associated code is removed in this commit, however I added a TODO to actually take care of the private chain support in the 2.0 rewards scheme. Please have a look @AlexanderLimonov as this may require additional ticket related to 2.0 rewards.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One could use the new rewards just as one would old rewards for private chains

@rafal-ch
Copy link
Contributor Author

bors r+

@casperlabs-bors-ng
Copy link
Contributor

Build succeeded:

@casperlabs-bors-ng casperlabs-bors-ng bot merged commit cb547b0 into casper-network:feat-2.0 Aug 30, 2023
4 checks passed
@rafal-ch rafal-ch deleted the merge_dev branch August 30, 2023 15:05
casperlabs-bors-ng bot added a commit that referenced this pull request Sep 1, 2023
4258: Fix after private-chain merge r=mpapierski a=mpapierski

After doing the same merge three basically times (feat-1.6, #4109 and #4228) there's two issues fixed in this PR after #4109 :

- Due to conflicts, VFTA tests for locked amounts were incorrect (default should be 0 lockup, tests should exercise 90 days) - this was introduced in #3231, but conflicts in private chain changes made it revert to 90days.
- Payment errors penalty could've been paid to block proposer regardless of `fee_handling`/`refund_handling` option.

Once merged I'll apply this to both feat-1.6 (again) and feat-2.0

Co-authored-by: Michał Papierski <michal@casperlabs.io>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

8 participants