A framework that enables dynamic LINK payments on Direct Request (Any API), syncing the price with the network gas and token conditions. It targets NodeOps that seek being competitive on their Direct Request operations.
Direct Request (aka Any API) is a Chainlink product that allows requests of any API & service from a blockchain where the request is triggered by an on-chain event. A high level overview includes the following steps:
- A NodeOp adds a
directrequest
Job on its node. Adirectrequest
job specification (job spec) is a TOML file that defines, in dot notation, a set of tasks to be executed upon anOracleRequest
event emitted by theOracle
orOperator
contract (aka Oracle). These tasks determine how to request APIs, process their response, and submit the result on-chain. Some API integrations (e.g. request requirements, processing the response, etc.) are too complex for their built-in core tasks (e.g.http
,jsonparse
) and an External Adapter (EA must be used. An EA is an integration-specific server that bridges a job run with an API and provides enhanced request and response processing capabilities. Both jobs and EAs can be implemented by anyone (NodeOp, client, anonymous dev, etc.) but to be used they are required to be available in the Chainlink node first. - The NodeOp shares the Job details with the customer who implements the Consumer contract (Consumer) using the
ChainlinkClient
library. The main role of the Consumer is to build aChainlink Request
, send it, then fulfill it. The request is a Chainlink.Request object that contains the unique job details and the API request parameters (in Solidity) CBOR encoded (from Concise Binary Object Representation). - To request the job, the Consumer transfers LINK (via the
LinkToken.transferAndCall
method) to the Oracle and attaches theChainlink.Request
data as payload. The Oracle emits theOracleRequest
event (including the payload) after receiving the LINK payment. - Then, the Chainlink node subscribed to the event triggers a job run. Along with the job execution, the event data is decoded, the request parameters are processed and used to request the APIs (via either
http
task orbridge
task). Finally, the responses are processed and the result is submitted on-chain back to the Consumer via the Oracle thus fulfilling the request.
Resources:
- Chainlink docs - Direct Request Jobs (Direct Request from NodeOp point of view)
- Chainlink docs - Any API (Direct Request from Consumer point of view)
- Chainlink docs - Chainlink Arquitecture, Basic Request Model
- LinkPool docs - Lifecycle of a Chainlink Request
In the current model the LINK payment is statically defined in the TOML job spec (in the minContractPaymentLinkJuels
field), which has the following implications:
- The LINK amount is fixed on the off-chain side; no dynamism.
- Consumers don't have access to the exact amount and NodeOps must let them know about any change on it.
- Consumer transfers to Operator the LINK amount before the job runs and without knowing the outcome.
- NodeOps must calculate the LINK payment amount factoring in at least:
- The gas units incurred by the fulfillment tx (result-size-dependant).
- The gas price (in GASTKN) and the LINK price in the network.
- However, there are other factors too, e.g. changes on the result-size, frequency of gas spikes/network usage, gas bumps done by the Chainlink Node to assure tx inclusion, the L1 tx fees on L2s, etc. It is reasonable then that NodeOps set the LINK payment amount high enough to offset any potential losses.
- Adding a Direct Request job implies having to spend time calculating the LINK amount.
- NodeOps that manage multiple jobs (on multiple nodes and networks) struggle to keep each
minContractPaymentLinkJuels
up to date. Also take into account that the amount is probably changed via GUI.
- The Direct Request model adopted the same major improvements VRF v2 had? Pay-as-you-go based on the gas used on request fulfillment plus some profit margin set by the node operator (paid in LINK leveraging the Chainlink Price Feeds). On-demand callback
gasLimit
set by the requester. And a versatile subscription model that eases funding the requests. - NodeOps didn't have to worry anymore about token prices, network conditions, and just focus on profit margin? What about if the "last mile" of adding a job wasn't that time consuming and inaccurate?
- NodeOps had a framework to manage all of this, and consumers could verify it on-chain?
Well, these were the motivations behind DRCoordinator.
A framework composed of contracts (on-chain) and job spec management tools (off-chain), that enable Consumer to pay NodeOp only as much LINK is required to cover the gas costs incurred by the data delivery, plus some profit margin set by the NodeOp.
This is a high level overview of the Direct Request Model with DRCoordinator:
NodeOps have to deploy and set up first a DRCoordinator:
- Deploy, set up and verify a DRCoordinator using the
drcoordinator:deploy
Hardhat task.- NB: By default it will attempt to fetch the LINK / TKN Price Feed on the network and it will error if it is not found. In this case NodeOps will require to deploy in Multi Price Feed mode (See Price Feed Contract Addresses for choosing the right Price Feeds).
- Amend any non-immutable config after deployment using the
drcoordinator:set-config
Hardhat task. - NodeOps can check the DRCoordiantor storage detail using the
drcoordinator:detail
Hardhat task.
NodeOps have to add a DRCoordinator-friendly TOML job spec (image no 1), which only requires to:
- Set the
minContractPaymentLinkJuels
field to 0 Juels. Make sure to set first the node env varMINIMUM_CONTRACT_PAYMENT_LINK_JUELS
to 0 as well. - Add the DRCoordinator address in
requesters
to prevent the job being spammed (due to 0 Juels payment). - Add an extra data encode as
(bytes32 requestId, bytes data)
(viaethabiencode
orethabiencode2
tasks) before encoding the data for thefulfillOracleRequest2
tx.
NodeOps have to:
- Create the
Spec
(seeSpecLibrary.sol
) of the TOML spec added above (image no 2 & 3) and upload it in the DRCoordinator storage viaDRCoordinator.setSpec()
(image no 4).
- NodeOps should create the equivalent JSON Spec and upload it using the
drcoordinator:import-file
Hardhat task.
- Use
DRCoordinator.addSpecAuthorizedConsumers()
if on-chain whitelisting of consumers is desired. - Share/communicate the
Spec
details (via its key) so the Consumer devs can monitor theSpec
and act upon any change on it, e.g.fee
,payment
, etc.
Devs have to:
- Make Consumer inherit from
DRCoordinatorClient.sol
(an equivalent ofChainlinkClient.sol
for DRCoordinator requests). This library only builds theChainlink.Request
and then sends it to DRCoordinator (viaDRCoordinator.requestData()
), which is responsible for extending it and ultimately sending it to Operator. - Request a
Spec
by passing the Operator address, the maximum amount of gas willing to spend, the maximum amount of LINK willing to pay and theChainlink.Request
(which includes theSpec.specId
asid
and the request parameters CBOR encoded) (image no 5).
Devs can time the request with any of these strategies if gas prices are a concern:
- Call
DRCoordinator.calculateMaxPaymentAmount()
. - Call
DRCoordinator.calculateSpotPaymentAmount()
. - Call
DRCoordinator.getFeedData()
.
NB: Make sure Consumer has LINK balance in DRCoordinator.
When Consumer calls DRCoordinator.requestData()
DRCoordinator does (image no 5):
- Validates the arguments.
- Calculates MAX LINK payment amount, which is the amount of LINK Consumer would pay if all the
callbackGasLimit
was used fulfilling the request (txgasLimit
) (image no 6). - Checks that the Consumer balance can afford MAX LINK payment and that Consumer is willing to pay the amount.
- Calculates the LINK payment amount (REQUEST LINK payment) to be hold in escrow by Operator. The payment can be either a flat amount or a percentage (permyriad) of MAX LINK payment. The
paymentType
andpayment
are set in theSpec
by NodeOp. - Updates Consumer balancee.
- Stores essential data from Consumer,
Chainlink.Request
andSpec
in aFulfillConfig
(by request ID) struct to be used upon fulfillment. - Extends the Consumer
Chainlink.Request
and sends it to Operator (paying the REQUEST LINK amount) (image no 7), which emits theOracleRequest
event (image no 8).
6. Requesting the Data Provider(s) API(s), processing the response(s) and submitting the result on-chain
NB: all these steps follow the standard Chainlink Direct Request Model.
- The Chainlink node subscribed to the event triggers a
directrequest
job run. - The
OracleRequest
event data is decoded and the log and request parameters are processed and (9) used to request the Data Povider(s) API(s) (image no 9). - The API(s) response(s) (image no 10) are processed and the result is submitted on-chain back to DRCoordinator via
Operator.fulfillOracleRequest2()
(image no 11 & 12).
- NB: forwarding the response twice (i.e. Operator -> DRCoordinator -> Consumer) requires to encode the result as
bytes
twice (viaethabiencode
orethabiencode2
)./ - NB: the
gasLimit
parameter of theethtx
task has set the amount defined by Consumer when calledDRCoordinator.requestData()
plusGAS_AFTER_PAYMENT_CALCULATION
(50_000
gas units).
- Validates the request and its caller.
- Loads the request configuration (
FulfillConfig
) and attempts to fulfill the request by calling the Consumer callback method passing the response data (image no 13 & 14). - Calculates SPOT LINK payment, which is the equivalent gas amount used fulfilling the request in LINK, minus the REQUEST LINK payment, plus the fulfillment fee (image no 15). The fee can be either a flat amount of a percentage (permyriad) of SPOT LINK payment. The
feeType
andfee
are set in theSpec
by NodeOp. - Checks that the Consumer balance can afford SPOT LINK payment and that Consumer is willing to pay the amount. It is worth mentioning that DRCoordinator can refund Consumer if REQUEST LINK payment was greater than SPOT LINK payment and DRCoordinator's balance is greater or equal than SPOT payment. Tuning the
Spec.payment
andSpec.fee
should make this particular case very rare. - Updates Consumer and DRCoordinator balances.
-
Chainlink contracts (used for NodeOps tasks, and DRCoordinator tests)
- Hardhat: compile and run the smart contracts on a local development network
- TypeChain: generate TypeScript types for smart contracts
- Ethers: renowned Ethereum library and wallet implementation
- Waffle: tooling for writing comprehensive smart contract tests
- Solhint: linter
- Solcover: code coverage
- Prettier Plugin Solidity: code formatter
- Chainlink Smart Contracts: Chainlink smart contracts and their ABIs
- LINK
- AVAX Fuji
- BSC Testnet
- ETH Kovan
- ETH Rinkeby
- ETH Goerli faucet
- FTM Opera
- MATIC Mumbai
- OPT Goerli 1
- OPT Goerli 2
- OPT Goerli 3
- RSK
- XDAI
- ARB Goerli
- ARB Mainnet
- ARB Rinkeby
- AVAX Fuji
- AVAX Mainnet
- BSC Mainnet
- BSC Tesnet
- ETH Goerli
- ETH Kovan
- ETH Mainnet
- ETH Rinkeby
- FTM Mainnet
- FTM Testnet
- HECO Mainnet
- HECO Testnet
- KLAYTN Baobab
- MATIC Mainnet
- MATIC Mumbai
- METIS Mainnet
- MOONBEAM Mainnet
- MOONBEAM Moonriver
- ONE Mainnet
- OPT Goerli
- OPT Kovan
- OPT Mainnet
- POA Sokol
- RSK Mainnet
- XDAI Mainnet
NB: look for public RPCs on the official documentation of the aimed network.