feat: height-based tx parsing

[ziscky]

Actors V2

Overview

ActorsV2 is a new implementation of the actor parsing logic that is more flexible and easier to maintain while properly handling all legacy spec-actors and new builtin-actors structs. This ensures that the parser can properly parse all actor messages for any network version.

Design

The ActorParser struct is tasked with parsing actor messages and is composed of a helper.Helper and a zap.Logger.

Each actor implements the Actor interface, which includes the following methods:

• Name() string: Returns the actor's name.
• Parse(network string, height int64, txType string, msg *parser.LotusMessage, msgRct *parser.LotusMessageReceipt, mainMsgCid cid.Cid, key filTypes.TipSetKey) (map[string]interface{}, *types.AddressInfo, error): Parses the actor message.
• TransactionTypes() map[string]any: Provides a map of all transaction types supported by the actor.

Actor Folder Structure

Each actor is organized within its own directory under actors/v2. The directory contains the following files:

• generic.go: Defines generic functions, typed with specific builtin-actors/spec-actors version structs, for parsing actor messages.
• parse.go: Contains the switch case logic for parsing specific transaction types for the actor.

Testing

There are different types of tests to ensure comprehensive coverage:

Actor Support Verification

Test Function: TestAllActorsSupported

This test verifies that all actors in the latest builtin-actor release and all legacy spec-actors are supported by the parser.
It will fail if any actor is not supported.

Note: No modification is needed to this test for new releases, the test automatically gets the latest builtin-actor release from github.

To add a new actor:

• Create a new package under actors/v2.
• Implement a struct that satisfies the Actor interface.
• Integrate the actor into the ActorParser.GetActor function.

Method Coverage Verification

Test Function: TestMethodCoverage

This test verifies that all methods exposed by the actor in all builtin-actor and spec-actor releases are supported by the actors.
It will fail if any method is not covered.

Note: No modification is needed to this test on new builtin-actor releases. The test will pass once support is added.

To add support for a new method:

• Implement the new method within the actor struct.
• Add the method to the TransactionTypes map.
• Incorporate the method into the Parse function.

Network Version Coverage Verification

Test Function: TestVersionCoverage

This test verifies that all the actor methods can correctly handle all network versions ( decided by the height of the block ).
It will fail if any network version is not supported.

Note: No modification is needed to this test on new builtin-actor releases. The test will pass once support is added.

To add support for a new network version:

• Include the network version in tools/version_mapping.go.
• Ensure the version is supported in the actor parsing methods (use existing switch cases as a reference).

Actor Functionality Tests

Test Location: actors/tests/{actor_name}_test.go

These tests, originally developed for actors version 1 (v1), are designed to validate the functionality of actors by comparing their function outputs against a set of pre-calculated expected values. These expected values are stored in the data/actors/{actor_name} directory.

Each actor undergoes both v1 and v2 tests, and passing both test suites is a mandatory requirement. This dual testing approach ensures backward compatibility and adherence to established specifications.

Important Note: The pre-computed data currently stored within data/actors corresponds to network version V20. If testing against a different network version is required, the cmd/tracedl tool provides a mechanism for automatically updating the stored data to the desired network version. This ensures that tests are always executed against the correct expected values for the target network version.

These tests are designed to ensure that the actor parser accurately handles all releases of both builtin-actors and spec-actors. They are configured to automatically fail upon the release of any new builtin-actor version. This failure mechanism guides developers to the necessary modifications, thereby eliminating the need for manual verification of the parser with each new Filecoin upgrade.

Compatibility

The ActorParser is designed to be backwards compatible and a drop in replacement for ActorsV1.

Misc

Filecoin Network Version - Actor Version Mapping

https://github.com/filecoin-project/builtin-actors/releases

filecoin-project/community#74

The following table shows the mapping of Filecoin network versions to actor versions:

Network Version	Actor Version	Height(Mainnet)	Height(Calibration)
v8	v2(spec-actors)	170000	UNKNOWN
v9	v2(spec-actors)	265200	UNKNOWN
v10	v3(spec-actors)	550321	UNKNOWN
v11	v3(spec-actors)	665280	UNKNOWN
v12	v4(spec-actors)	712320	193789
v13	v5(spec-actors)	892800	0 (RESET)
v14	v6(spec-actors)	1231620	312746
v15	v7(spec-actors)	1594680	682006
v16	v8(builtin-actors)	1960320	1044660
v17	v9(builtin-actors)	2383680	16800 (RESET)
v18	v10(builtin-actors)	2683348	322354
v19	v11(builtin-actors)	2809800	489094
v20	v11(builtin-actors)	2870280	492214
v21	v12(builtin-actors)	3469380	1108174
v22	v13(builtin-actors)	3817920	1427974
v23	v14(builtin-actors)	4154640	1779094
v24	v15(builtin-actors)	4461240	2081674

🔗 zboto Link

[emmanuelm41]

Just a few comments, but it looks amazing

[emmanuelm41]

@ziscky we need to fix CI first

[lucaslopezf]

I really like the strategy we took! And the fact that everything is tested!! Very good PR.
I'm approving it because I don't see anything blocking. I left some comments that could be addressed in a second iteration (if applicable).

• Personally, I find maps more convenient than switches, but that's just a matter of preference (I said this because we're using switch a lot, in some place I think we can use maps and in other places we can use a struct, I put an example below)
• I like the defensive strategy! And I imagine we did a lot of things for backward compatibility reasons, but since we're at it, in the future:
• If you like the idea, instead of using switch cases, we could use a struct and iterate over it (I left an example in the miner package).
• We could use Golem's log instead of Zap.
• Add ctx parameter to principal interface

[lucaslopezf]

I put here as an example and idea, you can discard if don't like it. But maybe instead of switch we can do something like this in every package (of course in a second iteration)

// VersionParser groups the configuration for parsing according to a specific version.
type VersionParser struct {
	// IsSupported is a function that checks whether the version is compatible with the requested network and height.
	IsSupported func(network string, height int64) bool
	// Template contains an instance (usually a struct) that will be used to parse the message or return value.
	Template interface{}
	// FieldKey indicates whether parser.ParamsKey or parser.ReturnKey should be used.
	FieldKey string
}

// Definition of supported versions for GetAvailableBalanceExported.
// It is easy to add new versions or modify the priority order.
var availableBalanceParsers = []VersionParser{
	{
		IsSupported: tools.V24.IsSupported,
		Template:    &miner15.GetAvailableBalanceReturn{},
		FieldKey:    parser.ReturnKey,
	},
	{
		IsSupported: tools.V23.IsSupported,
		Template:    &miner14.GetAvailableBalanceReturn{},
		FieldKey:    parser.ReturnKey,
	},
	{
		IsSupported: tools.V22.IsSupported,
		Template:    &miner13.GetAvailableBalanceReturn{},
		FieldKey:    parser.ReturnKey,
	},
	{
		IsSupported: tools.V21.IsSupported,
		Template:    &miner12.GetAvailableBalanceReturn{},
		FieldKey:    parser.ReturnKey,
	},
	{
		// For V19 and V20 grouped together.
		IsSupported: func(network string, height int64) bool {
			return tools.AnyIsSupported(network, height, tools.V19, tools.V20)
		},
		Template: &miner11.GetAvailableBalanceReturn{},
		FieldKey: parser.ReturnKey,
	},
	{
		IsSupported: tools.V18.IsSupported,
		Template:    &miner10.GetAvailableBalanceReturn{},
		FieldKey:    parser.ReturnKey,
	},
}

// GetAvailableBalanceExported uses the version configuration to select the correct parsing logic.
func (*Miner) GetAvailableBalanceExported(network string, height int64, rawReturn []byte) (map[string]interface{}, error) {
	for _, vp := range availableBalanceParsers {
		if vp.IsSupported(network, height) {
			return parseGeneric(rawReturn, nil, false, vp.Template, vp.Template, vp.FieldKey)
		}
	}
	// Handling for versions prior to V17 (for example)
	if tools.AnyIsSupported(network, height, tools.VersionsBefore(tools.V17)...) {
		return map[string]interface{}{}, fmt.Errorf("%w: %d", actors.ErrInvalidHeightForMethod, height)
	}
	return nil, fmt.Errorf("%w: %d", actors.ErrUnsupportedHeight, height)
}

[ziscky]

I really like this! Thanks