support decimals #12812

shileiwill · 2024-04-15T03:40:06Z

No description provided.

github-actions · 2024-04-15T03:40:26Z

I see you added a changeset file but it does not contain a tag. Please edit the text include at least one of the following tags:

#nops : For any feature that is NOP facing and needs to be in the official Release Notes for the release.
#added : For any new functionality added.
#changed : For any change to the existing functionality. 
#removed : For any functionality/config that is removed.
#updated : For any functionality that is updated.
#deprecation_notice : For any upcoming deprecation functionality.
#breaking_change : For any functionality that requires manual action for the node to boot.
#db_update : For any feature that introduces updates to database schema.
#wip : For any change that is not ready yet and external communication about it should be held off till it is feature complete.
#bugfix - For bug fixes.
#internal - For changesets that need to be excluded from the final changelog.

github-actions · 2024-04-15T03:44:52Z

Go solidity wrappers are out-of-date, regenerate them via the make wrappers-all command

shileiwill · 2024-04-15T03:45:03Z

contracts/src/v0.8/automation/dev/v2_3/AutomationRegistryBase2_3.sol

-  uint256 internal constant ACCOUNTING_FIXED_GAS_OVERHEAD = 51_000; // Fixed overhead per tx
-  uint256 internal constant ACCOUNTING_PER_UPKEEP_GAS_OVERHEAD = 9_000; // Overhead per upkeep performed in batch
+  uint256 internal constant ACCOUNTING_FIXED_GAS_OVERHEAD = 51_000 + 30000; // Fixed overhead per tx
+  uint256 internal constant ACCOUNTING_PER_UPKEEP_GAS_OVERHEAD = 9_000 + 2000; // Overhead per upkeep performed in batch


Hey @RyanRHall : since we modified the transmit path, the gas cost is changed. Without any changes in those constants, the typescript unit tests complains:

4) AutomationRegistry2_3 #transmit When the upkeep is funded Gas benchmarking log upkeeps [ @skip-coverage ] When f=10 calculates gas overhead appropriately within a margin: Gas overhead calculated is too high, decrease account gas variables (ACCOUNTING_FIXED_GAS_OVERHEAD/ACCOUNTING_PER_SIGNER_GAS_OVERHEAD) by at least 24874

I saw that you also updated those constants in a previous PR, do you recommend updating those numbers based on the unit tests? Appreciate your insights~

can we try struct packing differently and then see if we still need to increase the gas overhead? see other comment for struct packing suggestion :)

shileiwill · 2024-04-15T23:23:54Z

contracts/src/v0.8/automation/dev/v2_3/AutomationRegistryBase2_3.sol

      // if the user can't cover the gas fee, then direct all of the payment to the transmitter and distribute no premium to the DON
-      payment = balance;
+      payment18Decimals = balance18Decimals;
+      // TODO to confirm: both priceUSD and linkUSD use 18 decimals, so here is safe?


To be confirmed.

RyanRHall · 2024-04-16T15:02:48Z

contracts/src/v0.8/automation/dev/v2_3/AutomationRegistryBase2_3.sol

+    uint8 decimals;
+    // 3rd word only read during cancellation and payment


this is a great case for careful struct packing! Before this PR, only the 1st word is read on the hot path, but now we read the 1st and 3rd words on the hot path (which is probably partly responsible for the increase in gas overhead).

The first word is using 216 (32 + 24 + 160) bits, meaning there are still 40 (256 - 216) bits available! We totally have space in the first word for this field, and by putting it there, we can save an SLOAD :)

Once again, I learnt the power of unit testing. after reordering the struct, gas is under control

RyanRHall · 2024-04-16T15:09:10Z

contracts/src/v0.8/automation/dev/v2_3/AutomationRegistryBase2_3.sol

@@ -379,7 +379,8 @@ abstract contract AutomationRegistryBase2_3 is ConfirmedOwner {
    uint256 fallbackPrice;
    // 2nd word only read if stale
    uint96 minSpend;
-    // 3rd word only read during cancellation
+    uint8 decimals;


in the function _setBillingConfig() can we add a require statement that checks the decimals value against the decimals() value returned by the token? This way we can be a little safer against mis-configuration.

Good call, I like the idea.

For the record, as per ERC20 standard, for decimals(), This method can be used to improve usability, but interfaces and other contracts MUST NOT expect these values to be present. And Openzeppelin used 18 and Etherscan uses 0 when the token doesn’t specify it’s decimals.

So unfortunately I dont think we are not guaranteed to get the decimals from a given token. Accordingly, the Openzepplin IERC20 interface doesnt have decimals().

I guess we will need to rely on the human input to get the decimal places.

I added a check to ensure the decimals is in range of [0,24], just trying to avoid human input errors. 99.9% of the tokens should be in this range. let me know if this check is necessary or too noisy.

RyanRHall · 2024-04-16T15:17:05Z

contracts/src/v0.8/automation/dev/v2_3/AutomationRegistryBase2_3.sol

-  uint256 internal constant ACCOUNTING_FIXED_GAS_OVERHEAD = 51_000; // Fixed overhead per tx
-  uint256 internal constant ACCOUNTING_PER_UPKEEP_GAS_OVERHEAD = 9_000; // Overhead per upkeep performed in batch
+  uint256 internal constant ACCOUNTING_FIXED_GAS_OVERHEAD = 51_000 + 30000; // Fixed overhead per tx
+  uint256 internal constant ACCOUNTING_PER_UPKEEP_GAS_OVERHEAD = 9_000 + 2000; // Overhead per upkeep performed in batch


can we try struct packing differently and then see if we still need to increase the gas overhead? see other comment for struct packing suggestion :)

RyanRHall · 2024-04-16T15:36:49Z

contracts/src/v0.8/automation/dev/v2_3/AutomationRegistryBase2_3.sol

@@ -975,29 +979,43 @@ abstract contract AutomationRegistryBase2_3 is ConfirmedOwner {

    PaymentReceipt memory receipt = _calculatePaymentAmount(hotVars, paymentParams);


in the PaymentReceipt let's say:

gasCharge / premium are in the billing token, using the billing token's native decimals

gasReimbursementJuels / premiumJuels use LINK, which always uses 18 decimals

RyanRHall · 2024-04-16T15:41:38Z

contracts/src/v0.8/automation/dev/v2_3/AutomationRegistryBase2_3.sol

+    // this balance uses the token's decimal places
+    uint96 balanceXDecimals = upkeep.balance;
+    // adjust to 18 decimal places to be comparable with the following payment
+    uint96 balance18Decimals = SafeCast.toUint96(
+      balanceXDecimals * 10 ** (18 - paymentParams.billingTokenParams.decimals)
+    );
+
+    // payment always has 18 decimals
+    uint96 payment18Decimals = receipt.gasCharge + receipt.premium;

    // this shouldn't happen, but in rare edge cases, we charge the full balance in case the user
    // can't cover the amount owed
-    if (balance < receipt.gasCharge) {
+    if (balance18Decimals < receipt.gasCharge) {
      // if the user can't cover the gas fee, then direct all of the payment to the transmitter and distribute no premium to the DON
-      payment = balance;
+      payment18Decimals = balance18Decimals;
+      // TODO to confirm: both priceUSD and linkUSD use 18 decimals, so here is safe?


I think we might get a cleaner solution if we try to handle the decimals problem in _calculatePaymentAmount() instead of _handlePayment() (see other comment about PaymentReceipt). I think the _handlePayment() code can remain as-is if we just tweak the PaymentReceipt struct so that it always uses the same decimals as the billing token :)

RyanRHall · 2024-04-16T15:42:18Z

contracts/src/v0.8/automation/dev/v2_3/AutomationRegistryBase2_3.sol

+    uint96 balanceXDecimals = upkeep.balance;
+    // adjust to 18 decimal places to be comparable with the following payment
+    uint96 balance18Decimals = SafeCast.toUint96(
+      balanceXDecimals * 10 ** (18 - paymentParams.billingTokenParams.decimals)


this will underflow and revert if the billing token uses >18 decimals (no idea how common this is, but I think we might as well support it!)

jmank88 · 2024-04-17T00:02:29Z

contracts/.changeset/heavy-horses-greet.md

Probably only need one changeset file 😆

RyanRHall

Can we add a foundry test for getMinBalance() that compares the min balance for tokens with different numbers of decimals? (but all the same other params)? This will test that decimals is being handled correctly by our payment calculation code path.

contracts/src/v0.8/automation/dev/test/BaseTest.t.sol

RyanRHall · 2024-04-17T18:39:28Z

contracts/src/v0.8/automation/dev/v2_3/AutomationRegistryBase2_3.sol

+      // most ERC20 tokens are 18 decimals, we support tokens with up to 24 decimals
+      if (config.decimals > 24) {
+        revert InvalidToken();
+      }


why only up to 24?

I checked several mainnet tokens and found that all of them have a decimals() function, so I think we can require(billingToken.decimals() == billingConfig.decimals) wdyt?

Here can we also make a requirement that billindConfig.aggregator.decimals() == 8?

why only up to 24?

24 is just a number, we can change.

Here can we also make a requirement that billindConfig.aggregator.decimals() == 8?

I am pretty confident that this decimals() can be done!

I checked several mainnet tokens and found that all of them have a decimals() function, so I think we can require(billingToken.decimals() == billingConfig.decimals)

Compiler will not be happy with this. our billingToken is of type IERC20 which doesnt have decimals().

Sorry, typo in the above message, the billingConfig.aggregator decimals should be 8 not 18

Compiler will not be happy with this. our billingToken is of type IERC20 which doesnt have decimals().

Can we change from IERC20 to something that supports the decimals() function? How about this guy?

We can alias the import so it's a little cleaner...

import {IERC20Metadata as IERC20} from ...

I will explore this IERC20Metadata.

There is definitely way to get the decimals. If OZ's interfaces dont provide that, we can make a low level call from the token address and fetch the decimals if it exists.

address(token).call(abi.encodeWithSignature("decimals()"))...

RyanRHall · 2024-04-17T18:43:51Z

contracts/src/v0.8/automation/dev/v2_3/AutomationRegistryBase2_3.sol

@@ -660,6 +663,7 @@ abstract contract AutomationRegistryBase2_3 is ConfirmedOwner {
    HotVars memory hotVars,
    PaymentParams memory paymentParams
  ) internal view returns (PaymentReceipt memory receipt) {
+    uint8 decimals = paymentParams.billingTokenParams.decimals;


it's probably cheaper to cast this as a uint256 here (otherwise, the evm has to convert it implicitly every time it is used in a calculation below)

RyanRHall · 2024-04-17T19:11:21Z

contracts/src/v0.8/automation/dev/v2_3/AutomationRegistryBase2_3.sol

+    if (decimals < 18) {
+      receipt.premium = SafeCast.toUint96(premium18Decimals / (10 ** (18 - decimals)));
+    } else if (decimals > 18) {
+      receipt.premium = SafeCast.toUint96(premium18Decimals * (10 ** (decimals - 18)));


because we're doing multiplication (line 696) after division (line 691) I think we're potentially losing some precision. Could we instead do something like this...

uint256 numeratorScalingFactor = decimals > 18 ? 10 ** (decimals - 18) : 1; uint256 denominatorScalingFactor = decimals < 18 ? 10 ** (18 - decimals) : 1; receipt.premium = SafeCast.toUint96( (premiumHexaicosaUSD * numeratorScalingFactor) / (paymentParams.billingTokenParams.priceUSD * denominatorScalingFactor ) );

wdyt?

github-actions · 2024-04-18T04:56:50Z

Go solidity wrappers are out-of-date, regenerate them via the make wrappers-all command

contracts/src/v0.8/automation/dev/v2_3/AutomationRegistryBase2_3.sol

RyanRHall · 2024-04-18T12:50:11Z

contracts/src/v0.8/automation/dev/v2_3/AutomationRegistryBase2_3.sol

+      // most ERC20 tokens are 18 decimals, we support tokens with up to 24 decimals
+      if (config.decimals > 24) {
+        revert InvalidToken();
+      }


Compiler will not be happy with this. our billingToken is of type IERC20 which doesnt have decimals().

Can we change from IERC20 to something that supports the decimals() function? How about this guy?

We can alias the import so it's a little cleaner...

import {IERC20Metadata as IERC20} from ...

cl-sonarqube-production · 2024-04-18T19:29:15Z

Quality Gate passed

Issues
0 New issues
0 Fixed issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
0.0% Duplication on New Code

See analysis details on SonarQube

RyanRHall

Really well done 🎉

RyanRHall · 2024-04-18T19:42:32Z

contracts/src/v0.8/automation/dev/test/AutomationRegistry2_3.t.sol

@@ -612,23 +635,51 @@ contract SetConfig is SetUp {
    );
  }

+  function testSetConfigRevertDueToInvalidDecimals() public {


this is awesome 🙌 love me some good defensive coding

RyanRHall · 2024-04-18T19:43:23Z

contracts/src/v0.8/automation/dev/test/AutomationRegistry2_3.t.sol


    uint256[] memory minRegistrationFees = new uint256[](billingTokens.length);
-    minRegistrationFees[0] = 100000000000000000000; // 100 USD
+    minRegistrationFees[0] = 100e18; // 100 USD


RyanRHall · 2024-04-18T19:56:23Z

contracts/src/v0.8/automation/dev/v2_3/AutomationRegistryBase2_3.sol

+      // most ERC20 tokens are 18 decimals, priceFeed must be 8 decimals
+      if (config.decimals != token.decimals() || config.priceFeed.decimals() != 8) {
+        revert InvalidToken();
+      }


🙌 love it

shileiwill temporarily deployed to sdlc April 15, 2024 03:40 — with GitHub Actions Inactive

shileiwill commented Apr 15, 2024

View reviewed changes

shileiwill force-pushed the decimals branch from bf0ad2c to 709badd Compare April 15, 2024 19:02

shileiwill temporarily deployed to sdlc April 15, 2024 19:03 — with GitHub Actions Inactive

shileiwill commented Apr 15, 2024

View reviewed changes

RyanRHall requested changes Apr 16, 2024

View reviewed changes

shileiwill force-pushed the decimals branch from 709badd to 2d3ffea Compare April 16, 2024 23:51

shileiwill temporarily deployed to sdlc April 16, 2024 23:51 — with GitHub Actions Inactive

shileiwill force-pushed the decimals branch from 2d3ffea to 7d1042a Compare April 17, 2024 00:00

shileiwill temporarily deployed to sdlc April 17, 2024 00:00 — with GitHub Actions Inactive

shileiwill marked this pull request as ready for review April 17, 2024 00:01

shileiwill requested review from a team as code owners April 17, 2024 00:01

jmank88 reviewed Apr 17, 2024

View reviewed changes

support decimals

f9d6306

shileiwill force-pushed the decimals branch from 7d1042a to f9d6306 Compare April 17, 2024 03:42

shileiwill temporarily deployed to sdlc April 17, 2024 03:42 — with GitHub Actions Inactive

shileiwill requested a review from RyanRHall April 17, 2024 17:05

RyanRHall requested changes Apr 17, 2024

View reviewed changes

contracts/src/v0.8/automation/dev/test/BaseTest.t.sol Show resolved Hide resolved

RyanRHall requested changes Apr 17, 2024

View reviewed changes

shileiwill temporarily deployed to sdlc April 18, 2024 04:52 — with GitHub Actions Inactive

address comments

a6642aa

shileiwill force-pushed the decimals branch from d1167a2 to a6642aa Compare April 18, 2024 05:26

shileiwill temporarily deployed to sdlc April 18, 2024 05:26 — with GitHub Actions Inactive

shileiwill commented Apr 18, 2024

View reviewed changes

contracts/src/v0.8/automation/dev/v2_3/AutomationRegistryBase2_3.sol Show resolved Hide resolved

RyanRHall requested changes Apr 18, 2024

View reviewed changes

shileiwill temporarily deployed to sdlc April 18, 2024 19:01 — with GitHub Actions Inactive

use IERC20Metadata

e9c06b9

shileiwill force-pushed the decimals branch from 6d816df to e9c06b9 Compare April 18, 2024 19:13

shileiwill temporarily deployed to sdlc April 18, 2024 19:13 — with GitHub Actions Inactive

Merge branch 'develop' into decimals

5a9f84a

shileiwill temporarily deployed to sdlc April 18, 2024 19:14 — with GitHub Actions Inactive

RyanRHall approved these changes Apr 18, 2024

View reviewed changes

shileiwill added this pull request to the merge queue Apr 18, 2024

Merged via the queue into develop with commit 5b33a32 Apr 18, 2024
109 checks passed

shileiwill deleted the decimals branch April 18, 2024 20:21

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

support decimals #12812

support decimals #12812

shileiwill commented Apr 15, 2024

github-actions bot commented Apr 15, 2024

github-actions bot commented Apr 15, 2024

shileiwill Apr 15, 2024

RyanRHall Apr 16, 2024

shileiwill Apr 15, 2024

RyanRHall Apr 16, 2024

shileiwill Apr 16, 2024

RyanRHall Apr 16, 2024

shileiwill Apr 16, 2024 •

edited

shileiwill Apr 16, 2024

RyanRHall Apr 16, 2024

RyanRHall Apr 16, 2024

RyanRHall Apr 16, 2024

RyanRHall Apr 16, 2024

jmank88 Apr 17, 2024

RyanRHall left a comment

RyanRHall Apr 17, 2024

RyanRHall Apr 17, 2024

RyanRHall Apr 17, 2024 •

edited

shileiwill Apr 17, 2024 •

edited by RyanRHall

RyanRHall Apr 17, 2024

RyanRHall Apr 18, 2024

shileiwill Apr 18, 2024

RyanRHall Apr 17, 2024

RyanRHall Apr 17, 2024

github-actions bot commented Apr 18, 2024

RyanRHall Apr 18, 2024

cl-sonarqube-production bot commented Apr 18, 2024

RyanRHall left a comment

RyanRHall Apr 18, 2024

RyanRHall Apr 18, 2024

RyanRHall Apr 18, 2024

		uint8 decimals;
		// 3rd word only read during cancellation and payment

		@@ -975,29 +979,43 @@ abstract contract AutomationRegistryBase2_3 is ConfirmedOwner {

		PaymentReceipt memory receipt = _calculatePaymentAmount(hotVars, paymentParams);

support decimals #12812

support decimals #12812

Conversation

shileiwill commented Apr 15, 2024

github-actions bot commented Apr 15, 2024

github-actions bot commented Apr 15, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

shileiwill Apr 16, 2024 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

RyanRHall left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

RyanRHall Apr 17, 2024 • edited

Choose a reason for hiding this comment

shileiwill Apr 17, 2024 • edited by RyanRHall

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

github-actions bot commented Apr 18, 2024

Choose a reason for hiding this comment

cl-sonarqube-production bot commented Apr 18, 2024

Quality Gate passed

RyanRHall left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

shileiwill Apr 16, 2024 •

edited

RyanRHall Apr 17, 2024 •

edited

shileiwill Apr 17, 2024 •

edited by RyanRHall