Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MultiAggregateRateLimiter - non-evm and multi-lane remote token support #939

Merged
merged 14 commits into from
Jun 7, 2024

Conversation

elatoskinas
Copy link
Collaborator

@elatoskinas elatoskinas commented May 29, 2024

Motivation

Follow up of #916 to integrate non-EVM token support, and convert rateLimitedTokens configuration to be per-chain. This converts the multi-ARL to be truly multi-lane and untied from EVM2EVM to EVM2Any, Any2EVM

Solution

  • Add chainSelector to rate limit tokens state
  • Convert addresses from bytes to bytes32 (for now assuming that we can fit token addresses in bytes32 for all non-EVM chains)

Copy link
Contributor

Go solidity wrappers are out-of-date, regenerate them via the make wrappers-all command

@elatoskinas elatoskinas added the do not merge Do not merge at this time label May 29, 2024
Copy link
Contributor

Go solidity wrappers are out-of-date, regenerate them via the make wrappers-all command

Copy link
Contributor

github-actions bot commented May 29, 2024

LCOV of commit d322f60 during Solidity Foundry #5042

Summary coverage rate:
  lines......: 98.7% (1492 of 1511 lines)
  functions..: 96.5% (301 of 312 functions)
  branches...: 92.8% (646 of 696 branches)

Files changed coverage rate: n/a

Base automatically changed from feat/multi-offramp-arl-hook to ccip-develop June 3, 2024 08:06
Copy link
Contributor

github-actions bot commented Jun 3, 2024

Go solidity wrappers are out-of-date, regenerate them via the make wrappers-all command

@elatoskinas elatoskinas removed the do not merge Do not merge at this time label Jun 3, 2024
@elatoskinas elatoskinas marked this pull request as ready for review June 3, 2024 16:27
@elatoskinas elatoskinas requested review from makramkd and a team as code owners June 3, 2024 16:27
Copy link
Contributor

github-actions bot commented Jun 3, 2024

I see you updated files related to core. Please run pnpm changeset in the root directory to add a changeset as well as in the text include at least one of the following tags:

  • #added For any new functionality added.
  • #breaking_change For any functionality that requires manual action for the node to boot.
  • #bugfix For bug fixes.
  • #changed For any change to the existing functionality.
  • #db_update For any feature that introduces updates to database schema.
  • #deprecation_notice For any upcoming deprecation functionality.
  • #internal For changesets that need to be excluded from the final changelog.
  • #nops For any feature that is NOP facing and needs to be in the official Release Notes for the release.
  • #removed For any functionality/config that is removed.
  • #updated For any functionality that is updated.
  • #wip For any change that is not ready yet and external communication about it should be held off till it is feature complete.

/// @notice RateLimitToken struct containing both the local and remote token addresses
struct RateLimitTokenArgs {
LocalRateLimitToken localTokenArgs; // Local token update args scoped to one remote chain
bytes32 remoteToken; // Token on the remote chain (for OnRamp - dest, of OffRamp - source)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

theoretically possible this would be bytes, but we could upgrade worst case

Copy link
Collaborator Author

@elatoskinas elatoskinas Jun 4, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, we should revisit this when we get to non-EVM, since right now we have not identified addresses with > 32 bytes

/// @dev Tokens that should be included in Aggregate Rate Limiting (from local chain (this chain) -> remote),
/// grouped per-remote chain.
mapping(uint64 remoteChainSelector => EnumerableMapAddresses.AddressToBytes32Map) internal
s_rateLimitedTokensLocalToRemote;

/// @dev Set of callers that can call the validation functions (this is required since the validations modify state)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

did we consider deriving these from the router like the token pool does? Simpler config wise and I think it'd also give you the property that the on/offramp can only call their respective methods

Copy link
Collaborator Author

@elatoskinas elatoskinas Jun 4, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with the config simplicity + there would be an added level of misconfiguration since we would check the chain selectors together with the ramps. However, it is a ~10% increase in gas due to the external call:

Function Name min avg median max # calls
onIncomingMessage (authorizedCallers) 27390 50521 54146 60411 14
onIncomingMessage (router) 33118 56238 59862 66127 14

I recon the added config redundancy is worth it to save costs in this case, especially since this cost is per-message? Given it is easy to set this config (we can just query the Router ramps in a tooling script and set them to the authorized callers) - should we keep the current approach?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it also a 10% increase if the msg came from the router, and therefore the router is hot?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah you're right, I believe it would indeed be the case for the OffRamp, since we check OnlyOffRamp in the same router anyway.

However, the increase could still be there for the OnRamp if there are no token transfers (otherwise there would also be a hot-path through the TokenPool in most cases)

Another aspect is when we have 2 routers at the same time (one test router, and one prod router), it becomes much harder to configure the multi-ARL, since we would typically not re-deploy the multi-ARL. We had this complexity for the Nonce Manager as well, and it looks like the easiest option would be to explicitly maintain the authorized callers set ( I recon we can later refactor out this functionality to an abstract contract that will be re-used by both the multi-ARL and Nonce Manager )

Copy link
Collaborator Author

@elatoskinas elatoskinas Jun 5, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess the OnRamp no-tokens case can be optimized - so the gas cost issue can be mitigated:

  function onOutgoingMessage(Client.EVM2AnyMessage memory message, uint64 destChainSelector) external {
    if (message.tokenAmounts.length == 0) {
      return;
    }
    isOnRamp(msg.sender);
  }

I'm still in favour of the current approach though due to the complexity above

uint64 remoteChainSelector = message.sourceChainSelector;
RateLimiter.TokenBucket storage tokenBucket = _getTokenBucket(remoteChainSelector, false);

// Skip rate limiting if it is disabled
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this would also be the default behaviour right? E.g. say we have a rate limited token, which then is added to a new lane. By default it'd be excluded from the rate limit until we actively enable it with updateRateLimitTokens. Seems potentially safer the other way i.e. default we revert until rate limit set for the new lane

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes this would be the default behaviour. The change here is mostly a gas optimization - if the rate limiter is disabled, we do not need to loop over all tokens and calculate the total value.

Seems potentially safer the other way i.e. default we revert until rate limit set for the new lane

This would not work because the message validator is called into by all lanes: see OffRamp. If we reverted with an unset rate-limit, we would be forced to set a rate limit config for all lanes. I think it's better to allow the flexibility to partially enable rate limits

@matYang matYang requested a review from RensR June 5, 2024 03:00
@@ -72,7 +79,7 @@ contract MultiAggregateRateLimiter is IMessageInterceptor, OwnerIsCreator {
address internal s_priceRegistry;

/// @notice Rate limiter token bucket states per chain, with separate buckets for incoming and outgoing lanes.
mapping(uint64 remoteChainSelector => RateLimiterBuckets buckets) s_rateLimitersByChainSelector;
mapping(uint64 remoteChainSelector => RateLimiterBuckets buckets) internal s_rateLimitersByChainSelector;

/// @param rateLimiterConfigs The RateLimiter.Configs per chain containing the capacity and refill rate
/// of the bucket
Copy link
Collaborator

@RensR RensR Jun 5, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment on constructor: given this is a chain contract, which will always be deployed before the lanes, I think it's fine to remove the RateLimiterConfigArgs[] memory rateLimiterConfigs args there. I would think it will always be left empty. We could then rm the internal function

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

True, this is not required now since the contract is completely separate - removing this

@RensR
Copy link
Collaborator

RensR commented Jun 5, 2024

Inconsistent order in adds and removed in _applyAuthorizedCallerUpdates. We usually remove first, then add. Both in args and in logic

* feat: add EnumerableMapAddresses library

* refactor: remove redundant helpers
@elatoskinas elatoskinas force-pushed the feat/multi-offramp-arl-non-evm-support branch from 4b5036f to d322f60 Compare June 7, 2024 13:21
@elatoskinas elatoskinas merged commit c36cd86 into ccip-develop Jun 7, 2024
76 of 77 checks passed
@elatoskinas elatoskinas deleted the feat/multi-offramp-arl-non-evm-support branch June 7, 2024 14:02
asoliman92 pushed a commit that referenced this pull request Jul 31, 2024
…rt (#939)

## Motivation
Follow up of #916 to integrate non-EVM token support, and convert
`rateLimitedTokens` configuration to be per-chain. This converts the
multi-ARL to be truly multi-lane and untied from EVM2EVM to EVM2Any,
Any2EVM

## Solution
* Add `chainSelector` to rate limit tokens state
* Convert addresses from `bytes` to `bytes32` (for now assuming that we
can fit token addresses in `bytes32` for all non-EVM chains)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants