Create open source filecoin-project library for built-in actors decoding

[eshon]

Checklist

• This is not brainstorming ideas. If you have an idea you'd like to discuss, please open a new discussion on the lotus forum and select the category as Ideas.
• I have a specific, actionable, and well motivated feature request to propose.

Lotus component

• lotus daemon - chain sync
• lotus fvm/fevm - Lotus FVM and FEVM interactions
• lotus miner/worker - sealing
• lotus miner - proving(WindowPoSt/WinningPoSt)
• lotus JSON-RPC API
• lotus message management (mpool)
• Other

What is the motivation behind this feature request? Is your feature request related to a problem? Please describe.

Currently, Filecoin developers face significant friction when learning to decode built-in actors and their method parameters. This pain point includes:

• Navigating low-level CBOR and base64 encoding.

• Reverse-engineering method structures from Rust source code, often deeply embedded in nested calls.

• Lack of consistent tooling for decoding actor calls and method parameters, or even pointers to learn how others have done it.

• The need for builders and ecosystem participants to repeatedly create and maintain their own decoding tools — with limited documentation and inconsistent results.

As Filecoin’s built-in actors evolve these challenges compound. New developers and contractors entering the ecosystem experience a steep learning curve and no documentation, often struggling on their own and then canvassing the community for help.

Meanwhile, block explorers, indexers, analytics teams and integration builders are forced to independently decode messages, often duplicating work and arriving at incomplete implementations even after much painstaking effort.

By contrast, in the Ethereum ecosystem, contract ABIs are automatically produced during compilation and often self-published to Etherscan, enabling interoperability, user trust, and ease of exploration.

Filecoin native actors, however, have no ABI equivalent and there is no documentation about how to decode them. Builders usually have to ask for advanced technical support with a very steep learning curve.

Describe the solution you'd like

Create an open source filecoin-project repo for decoding actors and their parameters, including:

• decoders for all built-in actors, including recent versions (e.g. f06 / f07)
  • ABI-like definitions for all actors and methods (both native and EVM-compatible), enabling machine-readable and human-legible introspection.
• published ABIs for key EVM contracts including FWS and other commonly used contracts
• tests for the decoders

FilOz is ideally positioned to initiate and collaboratively maintain this repo given their deep familiarity with the actor codebase and active role in their evolution.

• Any updates to built-in actors should also be reflected in this proposed decoder repo

Describe alternatives you've considered

Prior Art

• Lily does some decoding but it is often out of date leading to inconsistencies in their BigQuery data (https://github.com/filecoin-project/lily/blob/c5db1917a9e94cf49598d2d26286f20c349ddbc5/lens/util/repo.go#L43)
• Of all block explorers, Filfox seems to have the best decoding although not complete.
  • Perhaps Filfox could be asked to open source some of their decoding tools as an alternative option to kickstart this initiative.

Additional context

Impact

A shared, well-maintained decoding library that is initially created by FilOz and collaboratively maintained along with other ecosystem groups would directly benefit:

• Block Explorers (Filfox, Beryx, Filscan, Filutils, Starboard, BlockScout etc.)
  • improved actor and params decoding
  • published ABIs for critical contracts that can then be more consistently verified across explorers
• Indexers (Lily - maintained by Starboard, Beryx Chain API - maintained by Zondax)
• Analytics Plaforms (Nexus, Kariba, etc.)
• Fil+ Tools - more clarity around f06/f07 (FIDL, CEL, FF, etc.)
• General Filecoin Builders and Integrators - improved onboarding and development experience.

[jennijuju]

collaboratively maintained along with other ecosystem groups would directly benefit

@eshon who are the ecosystem groups that have shown interest in co-maintaining something like this, and can they be the one developing this as well - we would be happy to provide technical sponsorship here?

[jennijuju]

collaboratively maintained along with other ecosystem groups would directly benefit

@eshon who are the ecosystem groups that have shown interest in co-maintaining something like this, and can they be the one developing this as well?

Many of the groups you've mentioned already have established tooling or interfaces for interacting with Filecoin via built-in actors. Is there any net new value or unblocker this initiative would provide that might justify reconsidering its prioritization?

[jennijuju]

For any FS contracts - they are written in solidity, and i think unlike built-in actors, instead of doing any manual work from any team, we should invest in better ecosystem tooling to support all FEVM actors.

consistent tooling for decoding actor calls and method parameters, or even pointers to learn how others have done it.

[eshon]

Many of the groups you've mentioned already have established tooling or interfaces for interacting with Filecoin via built-in actors.

@jennijuju - If you look across the various block explorers, it is obvious that their work is incomplete and it makes no sense for each team and new teams like CEL to re-implement these interfaces each time they need to interact with the built-in actors. There should be a shared repository where the community can find the best decodings currently available and where future collaboration is possible. For FS contracts, they too should have published ABIs in this shared repository as a consistent reference rather than relying on verification across all block explorers.

[davidgasquez]

Although groups might have their own tooling and interface to built-in actors, they're not accessible (open and easy-ish to use). In my specific use case, trying to decode an historical trace or something similar involves quite a setup and knowledge.

Not really sure about the best solution but I'd love to support anything that makes Filecoin data easier to work with.

Having a CLI with all the business logic or an easy way to decode these parameters would make getting our own Filecoin data more frictionless (right now, the solution is relying on Lily's tables, which might be incomplete/inaccurate) and increase the number of people that do that (e.g: how many folks in the ecosystem have all Filecoin Messages' params decoded).

[BigLep]

General

Thanks all so far the input. It's 100% acknowledged that the current situation is rough. I also think this is a tall order. I think it's useful hearing specific usecases that need to be solved as that maybe enables us to find something incremental / iterative that we can do on the route to a grander vision of improving this.

Some prior art

Go: I'm not saying it's easy to work with or meeting the needs, but https://github.com/filecoin-project/go-state-types is an example library that is doing param decoding.

Rust: I believe we have most of the decoding in Rust already as well, but you need to know where to look. (We'll have to ask maintainers; I don't know personally.)

ipld/js-ipld-schema#135 was a side project start at definining schemas and generating code for Rust, Go, JS. This work though was for the chain itself and not messages. It is outdated.

Scope of this issue

ABI-like definitions for all actors and methods

This makes sense. I wonder what the right format is (e.g, IPLD schema, Ethereum's ABI JSON schema),

decoders for all built-in actors

Agreed in general. What languages do we need these for?

tests for the decoders

Agreed this would be needed for any decoders that are built.

published ABIs for key EVM contracts including FWS and other commonly used contracts

That makes sense as something to want, but I don't think that belongs in this issue. It should be an expectation of the repos that house these contracts (e.g., https://github.com/FilOzone/pdp )

Usecases

Per the list in #13135 (comment) about Impact, I can imagine a lot of ways where this would be useful. I also want to make sure we know the most pressing things that need to get solved immediately so we can find a way to hopefully incrementally unblock people.

@davidgasquez , I believe you have a bunch of "raw traces" which you want to get decoded. You're proposing a CLI interface potentially that parameters could be piped to and I assume returned as json. I'm not up on how the data is all encoded. It may be useful to provide a few more specifics here like:

I run query X and get data back like Y. I want to be able to iterate over each row and pass it to a command like Z so I get decoded data like D.

Please share where there are other specific usecases where people are blocked right now and eager to get unblocked. It will be helpful to know what kind of interface or language support they want as well.

[davidgasquez]

@davidgasquez, I believe you have a bunch of "raw traces" which you want to get decoded. You're proposing a CLI interface potentially that parameters could be piped to and I assume returned as json. I'm not up on how the data is all encoded.

I'm also not 100% familiar on all the encodings and steps required to decode, but I can share an ideal workflow for someone (like me) that want's to access these params.

This scripts gets the traces (from StateCompute) and messages (from ChainGetMessagesInTipset) for the latest height.

#!/bin/bash

API="https://api.zondax.ch/fil/node/mainnet/rpc/v1"

# Get tipset if not set
TIPSET=${TIPSET:-$(curl -s -X POST "$API" -H 'Content-Type: application/json' --data '{"jsonrpc":"2.0","id":1,"method":"Filecoin.ChainHead","params":[]}' | jq '.result' -c)}

# Extract values and call StateCompute
HEIGHT=$(jq '.Height' <<< "$TIPSET")
CIDS=$(jq '.Cids' -c <<< "$TIPSET")

curl -s -X POST "$API" -H 'Content-Type: application/json' \
  --data "{\"jsonrpc\":\"2.0\",\"id\":1,\"method\":\"Filecoin.StateCompute\",\"params\":[$HEIGHT,null,$CIDS]}" \
  | jq '.result' > "state_compute_${HEIGHT}.json"

curl -s -X POST "$API" -H 'Content-Type: application/json' \
  --data "{\"jsonrpc\":\"2.0\",\"id\":1,\"method\":\"Filecoin.ChainGetMessagesInTipset\",\"params\":[$CIDS]}" \
  | jq '.result' > "tipset_messages_${HEIGHT}.json"

echo "Saved to: state_compute_${HEIGHT}.json and tipset_messages_${HEIGHT}.json"

In my case, I'm looking at the height 5038402.

In both the state_compute.json and tipset_messages.json there are instances where we have a message and a params field like this.

❯ jq '.Trace[0].MsgCid' state_compute_5038402.json
{
  "/": "bafy2bzaceb7ypdjfj7vznctty7fsji3d2woqqgmkmfpuo3cp6y7y67c7ykevy"
}
❯ jq '.Trace[0].Msg.Params' state_compute_5038402.json
"hRGBggBAgYINWMC0KHYrtWVfGMTsx5ZCItiQY4Zh69V3Kvxeshz+0+Ujuigl+WDTjndjps7qhvz4T7qNTJ+vZMlJri7VpmHjdAAmPznFD43zD6+qxhwmAPWVl/sVGzSpfIZsmzAk5JMB2hsMapzoAPYv3rIAQtGPIzpx+7QL3bW0UxdLzRJDUnrHWc+rQIRyPfv4Ep+d1ggQpw21yR0OHVS2AFucESkf/OFiQDEqE85dhH8PzHLICNyYPB2iTEumpBQ4x0w7XRHjjigaAEzhKFggov84bDbY1FGfjSaPqHQz2q72DVvZL7TzsBYAThZkCYA="
❯ jq '.Trace[0].ExecutionTrace.Msg.Params' state_compute_5038402.json
"hRGBggBAgYINWMC0KHYrtWVfGMTsx5ZCItiQY4Zh69V3Kvxeshz+0+Ujuigl+WDTjndjps7qhvz4T7qNTJ+vZMlJri7VpmHjdAAmPznFD43zD6+qxhwmAPWVl/sVGzSpfIZsmzAk5JMB2hsMapzoAPYv3rIAQtGPIzpx+7QL3bW0UxdLzRJDUnrHWc+rQIRyPfv4Ep+d1ggQpw21yR0OHVS2AFucESkf/OFiQDEqE85dhH8PzHLICNyYPB2iTEumpBQ4x0w7XRHjjigaAEzhKFggov84bDbY1FGfjSaPqHQz2q72DVvZL7TzsBYAThZkCYA="

Decoding these is not difficult since the StateDecodeParams call works.

❯ curl -s -X POST "$API" \
  -H 'Content-Type: application/json' \
  --data "{
    \"jsonrpc\": \"2.0\",
    \"id\": 1,
    \"method\": \"Filecoin.StateDecodeParams\",
    \"params\": [
      \"f01832632\",
      5,
      \"hRgYgoIAQIIBQIGCDVjAg59b3h/LrGoNQfV3gt5hCASEYURxyGe9PdTKb1cZCcCN1dYfV3vSuKGqxDWtCVPrmeRiu0bPF6XOBsABEeYhWVPwHfbKEU21eBXr/fZT78HwVZGnVI8NLYXeF4X7IAEECy/I6RCYwkFtAv2EmoaGtOvF0JMctfYs4Z9ZDpOuO9oLPapsqcLx1iplvrfApjeokGe7ENI+uXlx0RzqMEdQaba+H8UVD+vKlf0+RcL6rLpF7cRg0nYIxfm2ukRMWujrGgBM4VlYIGWLZCTKHBJzvdDjgBzlN6k9LwdyCuzCutrjmKUudisj\",
      $CIDS
    ]
  }"
{
  "id": 1,
  "jsonrpc": "2.0",
  "result": {
    "Deadline": 24,
    "Partitions": [
      {
        "Index": 0,
        "Skipped": [
          0
        ]
      },
      {
        "Index": 1,
        "Skipped": [
          0
        ]
      }
    ],
    "Proofs": [
      {
        "PoStProof": 13,
        "ProofBytes": "g59b3h/LrGoNQfV3gt5hCASEYURxyGe9PdTKb1cZCcCN1dYfV3vSuKGqxDWtCVPrmeRiu0bPF6XOBsABEeYhWVPwHfbKEU21eBXr/fZT78HwVZGnVI8NLYXeF4X7IAEECy/I6RCYwkFtAv2EmoaGtOvF0JMctfYs4Z9ZDpOuO9oLPapsqcLx1iplvrfApjeokGe7ENI+uXlx0RzqMEdQaba+H8UVD+vKlf0+RcL6rLpF7cRg0nYIxfm2ukRMWujr"
      }
    ],
    "ChainCommitEpoch": 5038425,
    "ChainCommitRand": "ZYtkJMocEnO90OOAHOU3qT0vB3IK7MK62uOYpS52KyM="
  }
}

The issue is running this over old/historical tipsets. Then, you start running into IPLD and other errors and is when you have to resort to low-level CBOR, exploring Rust bindings, ...

From a data consumer point of view, it would be ideal to have a decode flag to the JSON RPC so you get a plain JSON file with all the params (like Beryx does). Indexing the chain would be then just looping through all the heights and saving the JSONs!

Another alternative, perhaps more lightweight, is to have a CLI that takes care of this logic so you can run it in parallel over your traces/messages and get the params decoded.

Right now, probably Zondax are the only ones that have almost all traces and messages decoded and that is something they've worked hard on getting (in engineering and compute resources).

Having an open and frictionless way to get this data will make it easier to rely on traces/messages for analytics instead of the custom Lily tables, which have different schemas, and has to be manually run after fixing any related bug. Ultimately, what I'd love to see is a dump of all the decoded (plain JSON) Filecoin traces, messages, events, blocks, .... so we can recompute things on the fly without having to run nodes, backfills ...

[eshon]

How about for now...

• Create a shared repo where the ecosystem can start compiling all of the decoding libraries we have for specific built-in actors, just a place to share
• If teams like FIDL have decodings for f06 and f07 that would help CEL, for example (CEL have a contractor building on those who had a hard time figuring out how to decode their functions), then this would be a place to share all of this
• If block explorer teams have some decoding tools we could try asking them to open source that to this repo or link to their own open source implementations?
• We could also create a README with some developer pointers, like the encoding code in Go, examples in any other languages, and a basic list of steps of how to do this with info about the CBOR steps and tools?
• If it makes sense, perhaps decoding are an area that PLDG devs could help with (@rvagg?)
• When new builtin-actors are added or modified, it'd be a good habit to add or update decodings for them