Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce v2 actors #3936

Merged
merged 80 commits into from
Oct 6, 2020
Merged
Show file tree
Hide file tree
Changes from 77 commits
Commits
Show all changes
80 commits
Select commit Hold shift + click to select a range
ebad0de
Introduce v1 actors
arajasek Sep 19, 2020
35562bd
fixup v1 actors for new methods
Stebalien Sep 23, 2020
858f119
compile fix
Stebalien Sep 23, 2020
8806f27
fix runtime reflection for upgrade
Stebalien Sep 23, 2020
edb31e6
correctly load new actors
Stebalien Sep 23, 2020
87351fa
move version to actors, from builtin
Stebalien Sep 24, 2020
d9656f5
add a generic load method for actor state
Stebalien Sep 24, 2020
8b35f48
initial vm conversion
Stebalien Sep 25, 2020
7d3bd14
rebase fixup
Stebalien Sep 25, 2020
b8d9d78
fix vm version
Stebalien Sep 25, 2020
271ceb9
add v1 support to the state tree
Stebalien Sep 25, 2020
5f3160c
continue expanding vm abstraction layer
Stebalien Sep 28, 2020
947d1e2
Merge branch 'master' into asr/spec-v1
Stebalien Sep 28, 2020
2049daa
fix actor imports
Stebalien Sep 28, 2020
ca9448b
rename actors v1 -> actors v2
Stebalien Sep 28, 2020
6fcf57e
use actor state abstraction in payment channel test
Stebalien Sep 28, 2020
1a7fb06
improve lotuspond
Stebalien Sep 28, 2020
52e064d
add gas numbers for v2
Stebalien Sep 28, 2020
233d8a9
update even more imports
Stebalien Sep 28, 2020
c88d124
use built-in actor ID
Stebalien Sep 28, 2020
d1000e3
wire up network upgrade logic for v2 actors upgrade
Stebalien Sep 28, 2020
3e6323a
make setNetworkName work with any actors version
Stebalien Sep 28, 2020
c0b316f
finish wiring up actors v2
Stebalien Sep 28, 2020
a314cfd
process error when setting network name
Stebalien Sep 28, 2020
b95b871
actually upgrade to actors v2
Stebalien Sep 28, 2020
01272f8
fix state test
Stebalien Sep 28, 2020
9344bc4
update api docs for new network version
Stebalien Sep 28, 2020
2315db1
cleanup state-tree loading
Stebalien Sep 28, 2020
1c03862
fix policy for specs-actors update
Stebalien Sep 29, 2020
c0e1903
check state invariants after upgrade
Stebalien Sep 29, 2020
e658ab3
fix policy test
Stebalien Sep 29, 2020
dd65e57
set policy correctly from test vectors
Stebalien Sep 29, 2020
be9d23b
Centralize some params in builtin
arajasek Sep 29, 2020
4989b1c
Merge branch 'master' into asr/spec-v1
Stebalien Sep 30, 2020
724306c
update specs-actors
Stebalien Sep 30, 2020
23b729a
remove proof parameter from payment channels
Stebalien Sep 30, 2020
4c60d7b
abstract over paych messages
Stebalien Sep 30, 2020
5253c0d
fix: import the correct v2 methods
Stebalien Sep 30, 2020
2e9915a
move common from message param into message builder constructor
Stebalien Sep 30, 2020
b01a1d4
abstract multisig calls
Stebalien Sep 30, 2020
941c194
Merge branch 'master' into asr/spec-v1
Stebalien Sep 30, 2020
99fe637
regen lotuspond info
Stebalien Sep 30, 2020
35e606d
remove direct specs-actors miner access from expiration calculation
Stebalien Sep 30, 2020
8dcbd52
abstract over deal collateral
Stebalien Sep 30, 2020
6825a48
update message extraction tool for specs-actors update
Stebalien Sep 30, 2020
486812e
name test networks
Stebalien Sep 30, 2020
42d5d8d
run upgrade logic on stmgr.Call
Stebalien Sep 30, 2020
9d59635
fix error messages
Stebalien Sep 30, 2020
9705b25
fix testground actor versions
Stebalien Sep 30, 2020
47a83b4
handle state forks in gas estimation
Stebalien Sep 30, 2020
196bf39
use non-faulty sectors, instead of active, in post proof
Stebalien Oct 1, 2020
3d91633
Update to v2/actors master
arajasek Oct 1, 2020
1cc0f74
make the upgrade schedule fully configurable
Stebalien Oct 1, 2020
e865ba0
improve upgrade schedule configurable
Stebalien Oct 1, 2020
8ad8a94
make it possible to override options when testing
Stebalien Oct 1, 2020
101d07f
fix version off by one error
Stebalien Oct 1, 2020
ed86ac1
use upgrade epochs from upgrade schedule
Stebalien Oct 2, 2020
c5de617
decouple network version and upgrades
Stebalien Oct 3, 2020
a57288a
run post before, while, and after upgrading
Stebalien Oct 3, 2020
f1e5e9f
better upgrade test times
Stebalien Oct 3, 2020
863a6ed
fix lints
Stebalien Oct 3, 2020
6aba3e3
fix deal error message logging
Stebalien Oct 3, 2020
f6ccab1
ignore context canceled error in window post test
Stebalien Oct 3, 2020
8292d60
Merge branch 'master' into asr/spec-v1
Stebalien Oct 3, 2020
b9f8831
fix specs-actors import in payment channel abstraction
Stebalien Oct 5, 2020
b6500be
Merge branch 'master' into asr/spec-v1
Stebalien Oct 5, 2020
420c44e
fix msig start height
Stebalien Oct 5, 2020
8bcc201
fix v2 GetSectorExpiration
Stebalien Oct 5, 2020
5a08b88
Test upgrade before/during/after cc upgrade
Stebalien Oct 5, 2020
f8b2022
fix linters
Stebalien Oct 5, 2020
9621dbb
fix tests to use v2 types
Stebalien Oct 5, 2020
17846aa
add some more policy toggles for testground
Stebalien Oct 5, 2020
1e78cd2
fix lotus soup build
Stebalien Oct 5, 2020
6d25f3d
Add some comments
arajasek Oct 5, 2020
89b88c1
fix conformance tests
Stebalien Oct 6, 2020
16243b9
make state diffing work with partial state trees
Stebalien Oct 6, 2020
14ad91c
Add type assertion for v2 runtime
arajasek Oct 6, 2020
4a6e9c9
set genesis name in test
Stebalien Oct 6, 2020
256be28
only enable gas tampering if the breeze upgrade is enabled
Stebalien Oct 6, 2020
b1818a1
fix base fee tests for different smoke heights
Stebalien Oct 6, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
23 changes: 22 additions & 1 deletion api/test/ccupgrade.go
Original file line number Diff line number Diff line change
Expand Up @@ -12,15 +12,36 @@ import (

"github.com/filecoin-project/go-state-types/abi"

"github.com/filecoin-project/lotus/build"
"github.com/filecoin-project/lotus/chain/stmgr"
"github.com/filecoin-project/lotus/chain/types"
"github.com/filecoin-project/lotus/node"
"github.com/filecoin-project/lotus/node/impl"
)

func TestCCUpgrade(t *testing.T, b APIBuilder, blocktime time.Duration) {
_ = os.Setenv("BELLMAN_NO_GPU", "1")

for _, height := range []abi.ChainEpoch{
1, // before
162, // while sealing
520, // after upgrade deal
5000, // after
} {
height := height // make linters happy by copying
t.Run(fmt.Sprintf("upgrade-%d", height), func(t *testing.T) {
testCCUpgrade(t, b, blocktime, height)
})
}
}

func testCCUpgrade(t *testing.T, b APIBuilder, blocktime time.Duration, upgradeHeight abi.ChainEpoch) {
ctx := context.Background()
n, sn := b(t, 1, OneMiner)
n, sn := b(t, 1, OneMiner, node.Override(new(stmgr.UpgradeSchedule), stmgr.UpgradeSchedule{{
Network: build.ActorUpgradeNetworkVersion,
Height: upgradeHeight,
Migration: stmgr.UpgradeActorsV2,
}}))
client := n[0].FullNode.(*impl.FullNodeAPI)
miner := sn[0]

Expand Down
5 changes: 4 additions & 1 deletion api/test/deals.go
Original file line number Diff line number Diff line change
Expand Up @@ -402,9 +402,12 @@ func testRetrieval(t *testing.T, ctx context.Context, client *impl.FullNodeAPI,
IsCAR: carExport,
}
updates, err := client.ClientRetrieveWithEvents(ctx, offers[0].Order(caddr), ref)
if err != nil {
t.Fatal(err)
}
for update := range updates {
if update.Err != "" {
t.Fatalf("%v", err)
t.Fatalf("retrieval failed: %s", update.Err)
}
}

Expand Down
29 changes: 21 additions & 8 deletions api/test/paych.go
Original file line number Diff line number Diff line change
Expand Up @@ -8,17 +8,21 @@ import (
"testing"
"time"

"github.com/filecoin-project/specs-actors/actors/builtin"
builtin0 "github.com/filecoin-project/specs-actors/actors/builtin"
paych0 "github.com/filecoin-project/specs-actors/actors/builtin/paych"

"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/go-state-types/big"
"github.com/filecoin-project/specs-actors/actors/builtin/paych"
"github.com/ipfs/go-cid"

"github.com/filecoin-project/go-address"
cbor "github.com/ipfs/go-ipld-cbor"

"github.com/filecoin-project/lotus/api"
"github.com/filecoin-project/lotus/api/apibstore"
"github.com/filecoin-project/lotus/build"
"github.com/filecoin-project/lotus/chain/actors/adt"
"github.com/filecoin-project/lotus/chain/actors/builtin/paych"
"github.com/filecoin-project/lotus/chain/events"
"github.com/filecoin-project/lotus/chain/events/state"
"github.com/filecoin-project/lotus/chain/types"
Expand Down Expand Up @@ -133,17 +137,26 @@ func TestPaymentChannels(t *testing.T, b APIBuilder, blocktime time.Duration) {
t.Fatal("Unable to settle payment channel")
}

creatorStore := adt.WrapStore(ctx, cbor.NewCborStore(apibstore.NewAPIBlockstore(paymentCreator)))

// wait for the receiver to submit their vouchers
ev := events.NewEvents(ctx, paymentCreator)
preds := state.NewStatePredicates(paymentCreator)
finished := make(chan struct{})
err = ev.StateChanged(func(ts *types.TipSet) (done bool, more bool, err error) {
act, err := paymentCreator.StateReadState(ctx, channel, ts.Key())
act, err := paymentCreator.StateGetActor(ctx, channel, ts.Key())
if err != nil {
return false, false, err
}
state, err := paych.Load(creatorStore, act)
if err != nil {
return false, false, err
}
toSend, err := state.ToSend()
if err != nil {
return false, false, err
}
state := act.State.(paych.State)
if state.ToSend.GreaterThanEqual(abi.NewTokenAmount(6000)) {
if toSend.GreaterThanEqual(abi.NewTokenAmount(6000)) {
return true, false, nil
}
return false, true, nil
Expand All @@ -156,7 +169,7 @@ func TestPaymentChannels(t *testing.T, b APIBuilder, blocktime time.Duration) {
return true, nil
}, func(ctx context.Context, ts *types.TipSet) error {
return nil
}, int(build.MessageConfidence)+1, build.SealRandomnessLookbackLimit, func(oldTs, newTs *types.TipSet) (bool, events.StateChange, error) {
}, int(build.MessageConfidence)+1, build.Finality, func(oldTs, newTs *types.TipSet) (bool, events.StateChange, error) {
return preds.OnPaymentChannelActorChanged(channel, preds.OnToSendAmountChanges())(ctx, oldTs.Key(), newTs.Key())
})
if err != nil {
Expand Down Expand Up @@ -215,7 +228,7 @@ func TestPaymentChannels(t *testing.T, b APIBuilder, blocktime time.Duration) {
}

// wait for the settlement period to pass before collecting
waitForBlocks(ctx, t, bm, paymentReceiver, receiverAddr, paych.SettleDelay)
waitForBlocks(ctx, t, bm, paymentReceiver, receiverAddr, paych0.SettleDelay)

creatorPreCollectBalance, err := paymentCreator.WalletBalance(ctx, createrAddr)
if err != nil {
Expand Down Expand Up @@ -271,7 +284,7 @@ func waitForBlocks(ctx context.Context, t *testing.T, bm *BlockMiner, paymentRec

// Add a real block
m, err := paymentReceiver.MpoolPushMessage(ctx, &types.Message{
To: builtin.BurntFundsActorAddr,
To: builtin0.BurntFundsActorAddr,
From: receiverAddr,
Value: types.NewInt(0),
}, nil)
Expand Down
3 changes: 2 additions & 1 deletion api/test/test.go
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ import (
"github.com/filecoin-project/lotus/api"
"github.com/filecoin-project/lotus/build"
"github.com/filecoin-project/lotus/miner"
"github.com/filecoin-project/lotus/node"
)

type TestNode struct {
Expand Down Expand Up @@ -44,7 +45,7 @@ type StorageMiner struct {
//
// storage array defines storage nodes, numbers in the array specify full node
// index the storage node 'belongs' to
type APIBuilder func(t *testing.T, nFull int, storage []StorageMiner) ([]TestNode, []TestStorageNode)
type APIBuilder func(t *testing.T, nFull int, storage []StorageMiner, opts ...node.Option) ([]TestNode, []TestStorageNode)
type testSuite struct {
makeNodes APIBuilder
}
Expand Down
50 changes: 38 additions & 12 deletions api/test/window_post.go
Original file line number Diff line number Diff line change
Expand Up @@ -15,9 +15,11 @@ import (
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/lotus/extern/sector-storage/mock"
sealing "github.com/filecoin-project/lotus/extern/storage-sealing"
"github.com/filecoin-project/lotus/node"

"github.com/filecoin-project/lotus/api"
"github.com/filecoin-project/lotus/build"
"github.com/filecoin-project/lotus/chain/stmgr"
"github.com/filecoin-project/lotus/chain/types"
bminer "github.com/filecoin-project/lotus/miner"
"github.com/filecoin-project/lotus/node/impl"
Expand Down Expand Up @@ -114,8 +116,29 @@ func pledgeSectors(t *testing.T, ctx context.Context, miner TestStorageNode, n,
}

func TestWindowPost(t *testing.T, b APIBuilder, blocktime time.Duration, nSectors int) {
ctx := context.Background()
n, sn := b(t, 1, OneMiner)
for _, height := range []abi.ChainEpoch{
1, // before
162, // while sealing
5000, // while proving
} {
height := height // copy to satisfy lints
t.Run(fmt.Sprintf("upgrade-%d", height), func(t *testing.T) {
testWindowPostUpgrade(t, b, blocktime, nSectors, height)
})
}

}
func testWindowPostUpgrade(t *testing.T, b APIBuilder, blocktime time.Duration, nSectors int,
upgradeHeight abi.ChainEpoch) {
ctx, cancel := context.WithCancel(context.Background())
defer cancel()

n, sn := b(t, 1, OneMiner, node.Override(new(stmgr.UpgradeSchedule), stmgr.UpgradeSchedule{{
Network: build.ActorUpgradeNetworkVersion,
Height: upgradeHeight,
Migration: stmgr.UpgradeActorsV2,
}}))

client := n[0].FullNode.(*impl.FullNodeAPI)
miner := sn[0]

Expand All @@ -129,17 +152,24 @@ func TestWindowPost(t *testing.T, b APIBuilder, blocktime time.Duration, nSector
}
build.Clock.Sleep(time.Second)

mine := true
done := make(chan struct{})
go func() {
defer close(done)
for mine {
for ctx.Err() == nil {
build.Clock.Sleep(blocktime)
if err := sn[0].MineOne(ctx, MineNext); err != nil {
if ctx.Err() != nil {
// context was canceled, ignore the error.
return
}
t.Error(err)
}
}
}()
defer func() {
cancel()
<-done
}()

pledgeSectors(t, ctx, miner, nSectors, 0, nil)

Expand All @@ -159,7 +189,7 @@ func TestWindowPost(t *testing.T, b APIBuilder, blocktime time.Duration, nSector
head, err := client.ChainHead(ctx)
require.NoError(t, err)

if head.Height() > di.PeriodStart+(di.WPoStProvingPeriod)+2 {
if head.Height() > di.PeriodStart+di.WPoStProvingPeriod+2 {
fmt.Printf("Now head.Height = %d\n", head.Height())
break
}
Expand Down Expand Up @@ -289,12 +319,11 @@ func TestWindowPost(t *testing.T, b APIBuilder, blocktime time.Duration, nSector
pledgeSectors(t, ctx, miner, 1, nSectors, nil)

{
// wait a bit more

head, err := client.ChainHead(ctx)
// Wait until proven.
di, err = client.StateMinerProvingDeadline(ctx, maddr, types.EmptyTSK)
require.NoError(t, err)

waitUntil := head.Height() + 10
waitUntil := di.PeriodStart + di.WPoStProvingPeriod + 2
fmt.Printf("End for head.Height > %d\n", waitUntil)

for {
Expand All @@ -315,7 +344,4 @@ func TestWindowPost(t *testing.T, b APIBuilder, blocktime time.Duration, nSector

sectors = p.MinerPower.RawBytePower.Uint64() / uint64(ssz)
require.Equal(t, nSectors+GenesisPreseals-2+1, int(sectors)) // -2 not recovered sectors + 1 just pledged

mine = false
<-done
}
1 change: 1 addition & 0 deletions build/params_2k.go
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ const BreezeGasTampingDuration = 0
const UpgradeSmokeHeight = -1
const UpgradeIgnitionHeight = -2
const UpgradeLiftoffHeight = -3
const UpgradeActorsV2Height = 10
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do need this to be > 0? (for genesis to work I guess)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. Ideally, we'd either:

  1. Get this working at height 0. I can set this to 1, but have problems at 0.
  2. Make genesis work with the new actors (kind of tricky).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, we should eventually get genesis working with v2 actors, but for now "you must start at v0 and have a positive UpgradeActorsV2Height" is fine

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we just call handleStateForks just after creating genesis if this is set to 0 / lower?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably, we just need to get that working.


var DrandSchedule = map[abi.ChainEpoch]DrandEnum{
0: DrandMainnet,
Expand Down
2 changes: 1 addition & 1 deletion build/params_shared_funcs.go
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ func DhtProtocolName(netName dtypes.NetworkName) protocol.ID {

func UseNewestNetwork() bool {
// TODO: Put these in a container we can iterate over
if UpgradeBreezeHeight <= 0 && UpgradeSmokeHeight <= 0 {
if UpgradeBreezeHeight <= 0 && UpgradeSmokeHeight <= 0 && UpgradeActorsV2Height <= 0 {
Stebalien marked this conversation as resolved.
Show resolved Hide resolved
return true
}
return false
Expand Down
15 changes: 5 additions & 10 deletions build/params_shared_vals.go
Original file line number Diff line number Diff line change
Expand Up @@ -7,12 +7,12 @@ import (
"os"

"github.com/filecoin-project/go-address"
"github.com/filecoin-project/lotus/chain/actors/policy"

"github.com/filecoin-project/go-state-types/network"

"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/specs-actors/actors/builtin"
miner0 "github.com/filecoin-project/specs-actors/actors/builtin/miner"
)

// /////
Expand All @@ -25,7 +25,7 @@ const UnixfsLinksPerLevel = 1024
// Consensus / Network

const AllowableClockDriftSecs = uint64(1)
const NewestNetworkVersion = network.Version3
const NewestNetworkVersion = network.Version4
const ActorUpgradeNetworkVersion = network.Version4

// Epochs
Expand All @@ -35,7 +35,7 @@ const ForkLengthThreshold = Finality
var BlocksPerEpoch = uint64(builtin.ExpectedLeadersPerEpoch)

// Epochs
const Finality = miner0.ChainFinality
const Finality = policy.ChainFinality
const MessageConfidence = uint64(5)

// constants for Weight calculation
Expand All @@ -47,13 +47,8 @@ const WRatioDen = uint64(2)
// Proofs

// Epochs
const SealRandomnessLookback = Finality

// Epochs
const SealRandomnessLookbackLimit = SealRandomnessLookback + 2000 // TODO: Get from spec specs-actors

// Maximum lookback that randomness can be sourced from for a seal proof submission
const MaxSealLookback = SealRandomnessLookbackLimit + 2000 // TODO: Get from specs-actors
// TODO: unused
const SealRandomnessLookback = policy.SealRandomnessLookback
Comment on lines +50 to +51
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can be removed?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure, that's the only reason I left it. I figured it might be used by some external test?


// /////
// Mining
Expand Down
14 changes: 7 additions & 7 deletions build/params_testground.go
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,8 @@ import (
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/go-state-types/network"
"github.com/filecoin-project/specs-actors/actors/builtin"
miner0 "github.com/filecoin-project/specs-actors/actors/builtin/miner"

"github.com/filecoin-project/lotus/chain/actors/policy"
)

var (
Expand All @@ -32,7 +33,7 @@ var (

AllowableClockDriftSecs = uint64(1)

Finality = miner0.ChainFinality
Finality = policy.ChainFinality
ForkLengthThreshold = Finality

SlashablePowerDelay = 20
Expand All @@ -47,9 +48,7 @@ var (
BlsSignatureCacheSize = 40000
VerifSigCacheSize = 32000

SealRandomnessLookback = Finality
SealRandomnessLookbackLimit = SealRandomnessLookback + 2000
MaxSealLookback = SealRandomnessLookbackLimit + 2000
SealRandomnessLookback = policy.SealRandomnessLookback

TicketRandomnessLookback = abi.ChainEpoch(1)
WinningPoStSectorSetLookback = abi.ChainEpoch(10)
Expand Down Expand Up @@ -77,13 +76,14 @@ var (
UpgradeSmokeHeight abi.ChainEpoch = -1
UpgradeIgnitionHeight abi.ChainEpoch = -2
UpgradeLiftoffHeight abi.ChainEpoch = -3
UpgradeActorsV2Height abi.ChainEpoch = 10

DrandSchedule = map[abi.ChainEpoch]DrandEnum{
0: DrandMainnet,
}

NewestNetworkVersion = network.Version2
ActorUpgradeNetworkVersion = network.Version3
NewestNetworkVersion = network.Version4
ActorUpgradeNetworkVersion = network.Version4

Devnet = true
)
3 changes: 3 additions & 0 deletions build/params_testnet.go
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,9 @@ const UpgradeSmokeHeight = 51000

const UpgradeIgnitionHeight = 94000

// TODO: Actual epoch needs to be filled in
const UpgradeActorsV2Height = 128888
magik6k marked this conversation as resolved.
Show resolved Hide resolved

// This signals our tentative epoch for mainnet launch. Can make it later, but not earlier.
// Miners, clients, developers, custodians all need time to prepare.
// We still have upgrades and state changes to do, but can happen after signaling timing here.
Expand Down