| fip | 0104 |
|---|---|
| title | Filecoin Common Node Interface V1 |
| author | David Ansermino (@ansermino), Aatif Syed (@aatifsyed), Alexey Krasnoperov (@AlexeyKrasnoperov) |
| discussions-to | #1032 |
| status | Draft |
| type | FRC |
| created | 2024-06-25 |
Establish a basic specification for the common JSON-RPC API provided by Filecoin node implementations.
Lacking a common API for all node implementations, client diversity is hindered as applications are unlikely to be interoperable with different node implementations. Lotus is the defacto reference implementation, but lacks a sufficiently specified interface for other implementers to replicate.
In this FIP a subset of existing RPC methods are formally specified to enable consistency across implementations, and provide clarity on the methods users can expect when interacting with a node.
The JSON-RPC interface provided by Filecoin node implementations is the primary interaction point for many systems, however a standard interface has yet to be established definitively. Lacking such standards, builders are forced to invest additional time and resources in reverse engineering implementations. Node interoperability is hindered as there are no guarantees about what APIs will be encountered in the wild. A proper specification will enable the development of applications that are implementation-agnostic.
Ultimately the Lotus RPC API is the defacto specification currently, which includes 280+ methods. Based on our exploration and discussions with the Lotus team we have established that some methods provide duplicated functionality (e.g., ChainGetGenesis() is equivalent toChainGetTipSetByHeight(0)). Another subset of methods are related to implementation-specific internals (e.g., ChainHotGC). And yet another subset are for the purpose of configuring the instance, which could be exposed through other means (e.g., CLI, config file) depending on the design principles of a node implementation. This leads us to conclude that the canonical RPC interface should only include a subset of the methods currently supported by Lotus.
The defacto Lotus specification is lacking in several features: complete JSON schemas, descriptions for each method, and reliable consistency with the implementation1. This motivates us to propose a normative OpenRPC schema for the purpose of defining this interface in a machine readable format. This format in turn enables:
- Compliance checks - enable implementations of the interface to verify the syntax
- Specificity - it provides a language agnostic format for defining the ABI of the methods in the interface
- Discoverability - allow consumers to dynamically interact with providers
- Lack of desire to reinvent the wheel - It's not perfect, but it works...
Additionally, by establishing a specification we seek to establish a greater responsibility on implementers and encourage more discourse for future iterations and further interfaces. The present API shows signs of rapid iteration and the resulting technical debt.
Using the OpenRPC document format we define several groups of methods.
This specification can be found in the /resources/frc-0104 directory. A list of the methods is included below for posterity.
All methods are reproduced identically to their present implementation in Lotus 1.33.0.
Note: Lotus hosts three different API's at HTTP paths
/rpc/v0,/rpc/v1, and/rpc/v2with the latter two being marked as unstable. This document aims to specify thev1API.
Categories have been assigned to indicate the identified use case (see Design Rationale).
Node implementations must include all specified methods in their API without any modifications.
Conformance to the specification requires the following conditions to be met:
-
For each method, the
name,paramStructureanddeprecatedfields must match the specification. Additionally, the number and order of parameters must also match the specification. -
For each parameter of a method, the
name,schema,requiredanddeprecatedfields must all match the specification. -
For the result of a method, the
name,schema,requiredanddeprecatedfields must all match the specification.
Implementations may include additional fields in their OpenRPC documents. Implementations may also expose additional methods beyond what is included in this specification.
Working document: https://docs.google.com/spreadsheets/d/1fFkQuEjvFAd2s1dGX5zGmhxsEMLMUZ4uQFnIXgSZA5w/edit?usp=drivesdk
| Method | Category | Description |
|---|---|---|
| ChainGetBlock | State query | Returns the block with the specified CID. |
| ChainGetBlockMessages | State query | Returns all messages from the specified block. |
| ChainGetEvents | State query | Returns the events under the given event AMT root CID. |
| ChainGetFinalizedTipSet | State query | Returns the latest F3 finalized tipset, or falls back to EC finality if F3 is not operational on the node or if the F3 finalized tipset is further back than EC finalized tipset. |
| ChainGetMessage | State query | Returns the message with the specified CID. |
| ChainGetParentMessages | State query | Returns the messages included in the blocks of the parent tipset. |
| ChainGetParentReceipts | State query | Returns the message receipts included in the blocks of the parent tipset. |
| ChainGetPath | State query | Returns the path between the two specified tipsets. |
| ChainGetTipSet | State query | Returns the tipset with the specified CID. |
| ChainGetTipSetAfterHeight | State query | Looks back and returns the tipset at the specified epoch. If there are no blocks at the given epoch, returns the first non-nil tipset at a later epoch. |
| ChainGetTipSetByHeight | State query | Returns the tipset at the specified height. |
| ChainHasObj | State query | Checks if a given CID exists in the chain blockstore. |
| ChainHead | State query | Returns the chain head (heaviest tipset). |
| ChainReadObj | State query | Reads IPLD nodes referenced by the specified CID from the chain blockstore and returns raw bytes. |
| ChainTipSetWeight | State query | Returns the weight of the specified tipset. |
| GasEstimateFeeCap | Transactions | Returns the estimated fee cap for the given parameters. |
| GasEstimateGasLimit | Transactions | Returns the estimated gas limit for the given parameters. |
| GasEstimateGasPremium | Transactions | Returns the estimated gas premium for the given parameters. |
| GasEstimateMessageGas | Transactions | Returns the estimated gas for the given parameters. |
| GetActorEventsRaw | State query | Returns all user-programmed and built-in actor events that match the given filter. Results may be limited by MaxFilterResults, MaxFilterHeightRange, and the node's available historical data. |
| MinerCreateBlock | Miner operations | Fills and signs a block template on behalf of the given miner, returning a suitable block header. |
| MinerGetBaseInfo | Miner operations | Retrieves the Miner Actor at the given address and tipset, returning basic information such as power and mining eligibility. |
| MpoolBatchPush | Transactions | Adds a set of signed messages to the message pool. |
| MpoolBatchPushUntrusted | Transactions | Adds a set of messages to the message pool with additional verification checks. |
| MpoolGetNonce | Transactions | Returns the current nonce for the specified address. |
| MpoolPending | Transactions | Returns the pending messages for a given tipset. |
| MpoolPush | Transactions | Adds a signed message to the message pool. |
| MpoolPushMessage | Miner Operations | Assigns a nonce, signs, and pushes a message to the mempool. |
| MpoolPushUntrusted | Transactions | Adds a message to the message pool with verification checks. |
| MpoolSelect | Transactions | Returns a list of pending messages for inclusion in the next block. |
| NetAddrsListen | Node operation | Returns a list of listening addresses and the peer ID. |
| NetAgentVersion | Node operation | Returns the agent version string. |
| NetConnect | Node operation | Connects to a specified peer. |
| NetDisconnect | Node operation | Disconnects from the specified peer. |
| NetPeers | Node operation | Returns a list of currently connected peers. |
| NetProtectAdd | Node operation | Protects a peer from having its connection(s) pruned in the event the libp2p host reaches its maximum number of peers. |
| NetProtectList | Node operation | Returns the current list of protected peers. |
| NetProtectRemove | Node operation | Remove a peer from the protected list. |
| StateAccountKey | State query | Returns the public key address for the given ID address (secp and bls accounts). |
| StateCall | State query | Runs the given message and returns its result without persisting changes. The message is applied to the tipset's parent state. |
| StateCirculatingSupply | State query | Returns the exact circulating supply of Filecoin at the given tipset. |
| StateDealProviderCollateralBounds | State query | Returns the minimum and maximum collateral a storage provider can issue, based on deal size and verified status. |
| StateGetActor | State query | Returns the nonce and balance for the specified actor. |
| StateGetAllAllocations | State query | Returns all allocations available in the verified registry actor. |
| StateGetAllClaims | State query | Returns all claims available in the verified registry actor. |
| StateGetAllocation | State query | Returns the allocation for a given address and allocation ID. |
| StateGetAllocationForPendingDeal | State query | Returns the allocation for the specified pending deal. Returns null if no pending allocation is found. |
| StateGetAllocationIdForPendingDeal | State query | Returns the allocation ID for the specified pending deal. |
| StateGetAllocations | State query | Returns all allocations for a given client. |
| StateGetBeaconEntry | State query | Returns the beacon entries for the specified epoch. |
| StateGetClaim | State query | Returns the claim for a given address and claim ID. |
| StateGetClaims | State query | Returns all claims for a given provider. |
| StateGetNetworkParams | State query | Returns current network parameters. |
| StateGetRandomnessDigestFromBeacon | State query | Samples the beacon for randomness. |
| StateGetRandomnessDigestFromTickets | State query | Samples the chain for randomness. |
| StateGetRandomnessFromBeacon | State query | Returns the beacon entry for the specified Filecoin epoch. If unavailable, the call blocks until it becomes available. |
| StateGetRandomnessFromTickets | State query | Samples the chain for randomness. |
| StateListActors | State query | Returns the addresses of every actor in the state. |
| StateListMessages | State query | Returns all messages with a matching to or from address up to the given height. |
| StateListMiners | State query | Returns the addresses of every miner with claimed power in the Power Actor. |
| StateLookupID | State query | Retrieves the ID address of the given address. |
| StateLookupRobustAddress | State query | Returns the public key address for non-account addresses (e.g., multisig, miners). |
| StateMarketBalance | State query | Returns the Escrow and Locked balances of the specified address in the Storage Market. |
| StateMarketDeals | State query | Returns information about every deal in the Storage Market. |
| StateMarketParticipants | State query | Returns the Escrow and Locked balances of all participants in the Storage Market. |
| StateMarketStorageDeal | State query | Returns information about the specified deal. |
| StateMinerActiveSectors | State query | Returns information about sectors actively proven by a given miner. |
| StateMinerAllocated | State query | Returns a bitfield containing all sector numbers marked as allocated to the provided miner ID. |
| StateMinerAvailableBalance | State query | Returns the portion of a miner's balance available for withdrawal or spending. |
| StateMinerDeadlines | State query | Returns all proving deadlines for the given miner. |
| StateMinerFaults | State query | Returns a bitfield of the faulty sectors for the given miner. |
| StateMinerInfo | State query | Returns information about the specified miner. |
| StateMinerInitialPledgeCollateral | State query | Returns the initial pledge collateral for the specified miner's sector. |
| StateMinerPartitions | State query | Returns all partitions in the specified deadline. |
| StateMinerPower | State query | Returns the power of the specified miner. |
| StateMinerPreCommitDepositForPower | State query | Returns the sector precommit deposit for the specified miner. |
| StateMinerProvingDeadline | State query | Calculates the deadline and related details for a given epoch during a proving period. |
| StateMinerRecoveries | State query | Returns a bitfield of recovering sectors for the given miner. |
| StateMinerSectorAllocated | State query | Checks if a sector number is marked as allocated. |
| StateMinerSectorCount | State query | Returns the number of sectors in a miner's sector and proving sets. |
| StateMinerSectors | State query | Returns information about the given miner's sectors. If no filter is provided, all sectors are included. |
| StateNetworkVersion | State query | Returns the network version at the given tipset. |
| StateReadState | State query | Returns the state of the specified actor. |
| StateReplay | State query | Replays a given message, assuming it was included in a block in the specified tipset. |
| StateSearchMsg | State query | Returns the receipt and tipset the specified message was included in. |
| StateSectorExpiration | State query | Returns the epoch at which the specified sector will expire. |
| StateSectorGetInfo | State query | Returns on-chain information for the specified miner's sector. Returns null if not found. Use StateSectorExpiration for accurate expiration epochs. |
| StateSectorPartition | State query | Finds the deadline/partition for the specified sector. |
| StateSectorPreCommitInfo | State query | Returns the PreCommit information for the specified miner's sector. Returns null if not precommitted. |
| StateVerifiedClientStatus | State query | Returns the data cap for the given address. Returns null if no entry exists in the data cap table. |
| StateVerifiedRegistryRootKey | State query | Returns the address of the Verified Registry's root key. |
| StateVerifierStatus | State query | Returns the data cap for the given address. |
| StateVMCirculatingSupplyInternal | State query | Returns an approximation of Filecoin's circulating supply at the given tipset. |
| StateWaitMsg | State query | StateWaitMsg searches up to limit epochs for a message in the chain. If not found, it blocks until the message appears on-chain and reaches the required confidence depth. |
| SyncSubmitBlock | Miner operations | Submits a newly created block to the network. |
| WalletBalance | Miner operations | Returns the balance of a wallet. |
| WalletHas | Miner operations | Indicates whether the given address exists in the wallet. |
| WalletList | Miner operations | Returns a list of all addresses in the wallet. |
| WalletSign | Miner operations | Signs the given bytes using the specified address. |
| WalletSignMessage | Miner operations | Signs the given message using the specified address. |
The tool to check that methods in the table above match those in the specification is also included: check-methods-sync.sh.
./resources/frc-0104/check-methods-sync.shThis interface is contrived from the existing Lotus RPC V1 interface. Due to the large number of methods presently supported, we have limited the scope of this specification to include methods with a well established use case. We have observed the requirements of Lotus-miner, Curio, Web3Mine and Boost to inform which methods are actively used. We have also considered insights from Glif based on the usage of their RPC service.
We target two groups of users: end users and node operators. These each have their own requirements for particular data and access control. While no attempt is made in this FRC to specify considerations for these groups (e.g., access control), these groups are used to justify the need for particular methods.
The methods in the API are divided into these categories:
-
State Queries
- These primarily concern the chain state, actor state or network (p2p) state. They are read-only methods that are necessary to expose all elements of global state.
-
Miner Operations
- A small set of specialized methods miner applications require to participate in consensus.
-
Transactions
- Necessary methods to submit new transaction for inclusion in the chain, including inspecting the mempool and estimating gas parameters.
-
Node Operation
- Several methods are included to enable remote management and inspection of a node instance via RPC. These have been restricted to commonly used, implementation-agnostic methods.
Eth methods have been excluded as they are inferred from the Ethereum JSON-RPC specification, with some changes. We intend for these differences to specified in a later FIP.
It is expected that changes to this API will be made at some point in the future. All non-breaking changes should be introduced within the spec repo, at the discretion of node implementation teams (presently Lotus, Forest, Venus). Rough consensus should be followed, with implementation teams that adhere to the specification as the primary decision makers.
Breaking changes must be submitted as an FIP before being introduced to the latest spec.
In all cases valid semantic versioning must be used.
The methods in this specification are a subset of methods from the existing defacto specification (ie. Lotus API), which all implementation follow, thus backwards compatibility is maintained.
[TODO]
We may want to consider three types of testing:
- Schema conformance (request/response structure in OpenRPC document is valid)
- Implementation conformance (implementation adheres to schemas)
- Functional conformance (request returns expected response values)
Forest team is working on externalizing their RPC Snapshots Testing Suite, which will help to ensure API conformance.
Node implementers must consider the security implications of all methods and ensure sufficient access controls are implemented. This is beyond the scope of this FIP.
Hint: Major node implementations currently define 4 kinds of methods permissions: read, write, sign and admin. Example: Lotus RPC V1 interface.
Not relevant to this FRC.
A better API, one that is well defined and carefully constructed, will ease the development of many types of applications. This FRC is only sufficient in ensuring the API is well defined, additional work is needed in future to refine the API.
- Lotus
- Implementation of Common Node API methods
- OpenRPC document available
- Automated testing of conformance to schema
- Common Node API methods conform to schema
- Forest
- Implementation of Common Node API methods
- OpenRPC document available
- Automated testing of conformance to schema
- Common Node API methods conform to schema
- Venus
- Implementation of Common Node API methods
- OpenRPC document available
- Automated testing of conformance to schema
- Common Node API methods conform to schema
Copyright and related rights waived via CC0.