Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: implement lotus-sim #6406

Merged
merged 94 commits into from
Jun 24, 2021
Merged

feat: implement lotus-sim #6406

merged 94 commits into from
Jun 24, 2021

Conversation

Stebalien
Copy link
Member

@Stebalien Stebalien commented Jun 7, 2021

Implements #6156.

Usage

> make build
> cd cmd/lotus-sim
> go build .
> ./lotus-sim create
> ./lotus-sim run

Limitations: to run this, you need 2 weeks worth of state-trees to get power lookback (used to determine per-miner onboarding rates). Ideally, we'd do something else (maybe just look at the number of sectors and make assumptions?

Goals

  • Test how the lotus VM and actors as the network grows.
  • Find potential bugs that only appear at scale.
  • Determine state growth rate.
  • Test network upgrades.

Non Goals

This simulation does not:

  1. Use real proofs/sectors.
  2. Sign anything.
  3. Verify any signatures.
  4. Use a real miner. It just keeps using the last miner to mine a block (from when the network started).
  5. Do any "network" related operations. It just creates and validates (skipping proofs) a fake chain.

Design

This simulation:

  1. Forks the network at the current head (or at a desired point, see ./lotus-sim create --help).
  2. At every epoch:
    1. Funds the "funding" account (f0100) if necessary (by taking from random actors).
    2. Submits window posts for every "healthy" sector.
    3. Submits (batch) prove commit messages for any outstanding pre-commits that are ready to be proven.
    4. Submits (batch) pre-commit messages in the remaining space.

When necessary, the "funding" account will send funds to actors.

@BigLep BigLep linked an issue Jun 14, 2021 that may be closed by this pull request
3 tasks
@Stebalien Stebalien force-pushed the feat/lotus-sim branch 2 times, most recently from 2c17b83 to 520cfa8 Compare June 18, 2021 18:30
@Stebalien Stebalien marked this pull request as ready for review June 18, 2021 22:43
@Stebalien Stebalien requested a review from magik6k June 18, 2021 22:43
And make it run forever by default.
Hopefully, this'll make this code a bit easier to approach.
Add a helper function so we don't need to constantly repeat ourselves.
It doesn't really matter, and it ensures they all get executed in-order.
Instead of packing till we see "full". Prove-commits are large, we may
have room for some more pre-commits.
This way, we can easily walk the chain and:

1. Inspect messages and their results.
2. Inspect tipset state.
Useful to backup old simulations before creating a new one.
@Stebalien
Copy link
Member Author

To the reviewer: Most of the changes here are just a part of the simulation, but not all:

  1. It adds some methods to the actors shims.
  2. It exposes the ActorStore from the VM (so we can read intermediate state and, importantly, nonces).
  3. It sets ActorErr in one VM.ApplyMessage case where it should have been set but wasn't (not consensus critical).
  4. It modifies StateTree.ForEach to iterate over un-flushed actors.
  5. It fixes the genesis logic to set the balance (to zero) for some actors. This won't actually affect anything as we default to zero on serialize anyways. Unfortunately, that doesn't help when looking at unseralized actors in ForEach (see 4).

Please pay careful attention to 4. It shouldn't cause any problems (mostly used in genesis and some specific upgrades), and brings the function in-line with expectations. But it's still something that's good to look at carefully.

Copy link
Contributor

@magik6k magik6k left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly cosmetic nits.

It would also be really great to have at least a basic integration test, so that we know when it would get completely broken by some change.

Should also have a make target, and be added to BINS so it's built in make buildall

chain/actors/builtin/miner/state.go.template Show resolved Hide resolved
chain/vm/vm.go Outdated Show resolved Hide resolved
cmd/lotus-sim/simulation/stages/commit_queue.go Outdated Show resolved Hide resolved
cmd/lotus-sim/simulation/stages/funding_stage.go Outdated Show resolved Hide resolved
cmd/lotus-sim/upgrade.go Outdated Show resolved Hide resolved
Stebalien and others added 7 commits June 21, 2021 09:05
Co-authored-by: Łukasz Magiera <magik6k@users.noreply.github.com>
This is primarily for testing, so we can just pass an in-memory repo.
I wanted to expose the node's _fields_, but this also exposed the
methods. That got rather confusing.

(probably could have used a new type, but eh)

foo
I'd _really_ like to use lookback, but don't have that when starting
from a snapshot.
@magik6k magik6k merged commit ebd746c into master Jun 24, 2021
@magik6k magik6k deleted the feat/lotus-sim branch June 24, 2021 16:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Large scale testing post FIP13/FIP8
2 participants