Skip to content

Release protocol/v5.0.0

Latest
Latest
Compare
Choose a tag to compare
@github-actions github-actions released this 30 May 15:56
protocol/v5.0.0
c092bf0
This commit was signed with the committer’s verified signature.
roy-dydx
GPG key ID: AD3541742C721B4A
Learn about vigilant mode.

Protocol v5.0.0 Release Notes

Highlights

This release includes changes in following major areas:

  • Isolated Markets
  • Batch order cancellation
  • Protocol-enshrined LP Vault
  • Slinky Sidecar/Vote Extension
  • Performance Improvements
  • Soft Open Interest Cap
  • Full Node Streaming

Detailed Changes

Isolated Market

Perpetuals have been updated to include a market type which is either CROSS or ISOLATED. All currently existing perpetuals are CROSS perpetuals and have no change to functionality.

For ISOLATED perpetuals, a subaccount can only open a position in an ISOLATED perpetual if it holds no other perpetual positions. Subaccounts with an open position in an ISOLATED perpetual have their collateral stored in an account that is specific for the ISOLATED perpetual. Each ISOLATED perpetual will have a separate insurance fund that is used to backstop liquidations and collect liquidation fees.

Withdrawal gating logic when a subaccount with negative total net collateral is detected has been updated to gate withdrawals separately between CROSS perpetuals and each ISOLATED perpetual. For example, if a subaccount with negative total net collateral is detected and it holds an ISOLATED perpetual position, withdrawals will only be gated for subaccounts with positions in that specific ISOLATED perpetual and withdrawals are not gated for subaccounts with positions in CROSS perpetuals or other ISOLATED perpetuals.

The maximum number of subaccounts per address has been increased from 128 to 128001 to account for needed a subaccount to hold perpetual positions for each ISOLATED perpetual.

Check out the blog post for more details on this new feature.

  • Move insurance fund into x/perpetuals ([#1106](#1106))
  • Add market type to the CreatePerpetual API ([#1118](#1118))
  • Use market specific insurance fund for cross or isolated markets ([#1132](#1132))
  • Add function to retrieve collateral pool addr for a subaccount ([#1142](#1142))
  • Move SA module address transfers to use perpetual based SA accounts ([#1146](#1146))
  • Add state migrations for isolated markets ([#1155](#1155))
  • Update upper limit of subaccount number constraint ([#1164](#1164))
  • Check isolated market constraints in UpdateSubaccount. ([#1158](#1158))
  • Update subaccount-to-subaccount transfers for collateral pools. ([#1178](#1178))
  • Add market type to indexer events. ([#1201](#1201))
  • Add logic to transfer collateral when open/close isolated perpetual position. ([#1200](#1200))
  • Update withdrawal gating to be per collateral pool. ([#1208](#1208))
  • v5.0.0 state migration negative tnc subaccount store. ([#1226](#1226))
  • Add query function for collateral pool address. ([#1256](#1256))
  • Add new error code to be returned when orders fail isolated subaccount checks. ([#1331](#1331))
  • Add V2 for perpetual market create event + handlers. ([#1395](#1395))
  • Fix IsIsolatedPerpetual function orientation ([#1403](#1403))
  • Fix consensus failure from unhandled error. ([#1418](#1418))

Protocol Vault

x/vault module is added to introduce the functionality of LP vaults, which will be available for testing purposes only. A vault is activated after after its USDC balance reaches activation_threshold_quote_quantums and automatically provides liquidity for its corresponding market by placing stateful orders on both sides of the book and refreshing them every block. When LP vaults are fully operational anyone can deposit into any vault and upon each deposit, a vault mints new shares for the depositor based on deposit amount and current vault equity. x/vault parameters are updatable via governance.

Note that withdrawal from vaults is not supported in v5.0.0.

  • add x/vault protos ([#1169](#1169))
  • add x/vault total shares store ([#1179](#1179))
  • construct simple vault orders ([#1206](#1206))
  • refresh all vault orders ([#1213](#1213))
  • store x/vault parameters in state, add query, and update via governance ([#1238](#1238))
  • Implement MsgDepositToVault ([#1220](#1220))
  • change vault state key to not use serialization ([#1248](#1248))
  • Track shares of owners in vaults ([#1253](#1253))
  • decommission vaults at the beginning of a block ([#1264](#1264))
  • add cli for deposit to vault ([#1276](#1276))
  • rename order_size_ppm to order_size_pct_ppm in vault params ([#1271](#1271))
  • implement full vault strategy ([#1262](#1262))
  • keep track of vault shares as integers ([#1267](#1267))
  • vaults do not quote if order size is 0 ([#1325](#1325))
  • Skip various order placement logic for vault orders ([#1332](#1332))
  • return error if shares to mint is rounded down to zero ([#1333](#1333))
  • initialize vault module params in v5.0.0 upgrade handler ([#1359](#1359))
  • add activation threshold on vaults ([#1344](#1344))
  • add OwnerShares and AllVaults query and cli ([#1362](#1362))
  • bounded quotes by oracle price ([#1390](#1390))
  • updated default vault parameters ([#1450](#1450))

BatchCancel

MsgBatchCancel added to the protocol. Allow for batches (max 100) of short term orders to be cancelled. Treated as 2 order cancels by rate limiter.

  • MsgBatchCancel Protos ([#1094](#1094))
  • Validatebasic for new MsgBatchCancel ([#1101](#1101))
  • msgBatchCancel success + failure fields ([#1130](#1130))
  • Clob MsgBatchCancel functionality ([#1110](#1110))
  • Batch Cancel CLI + additional validation ([#1204](#1204))
  • Batch Cancel Rate Limit Metrics ([#1233](#1233))

Soft Open Interest Caps

Prevent oracle attacks and insurance fund losses from liquidation by scaling up the Initial Margin Fraction (IMF) after Open Interest (OI) exceeds a configurable cap. The effective IMF scales using below logic:

  • Each market has a Lower Cap and Upper Cap denominated in USDC.
  • Each market already has a Base IMF.
  • At any point in time, for each market, define
    • Open Notional = Open Interest * Oracle Price
    • Scaling Factor = (Open Notional - Lower Cap) / (Upper Cap - Lower Cap)
    • IMF Increase = Scaling Factor * (1 - Base IMF)
  • Then a market’s Effective IMF = Min(Base IMF + Max(IMF Increase, 0), 1.0). The effective IMF is the base IMF while the OI < lower cap, and increases linearly until OI = Upper Cap, at which point the IMF stays at 1.0 (requiring 1:1 collateral for trading).
  • Importantly, the market's Maintenance Margin Fraction (MMF) should not change.

The software default Soft Open Interest Caps configures are:

  • Large-Cap: IMF is never scaled
  • Mid-Cap: lower_cap = 20 million USDC, upper_cap = 50 million USDC
  • Long-Tail: lower_cap = 5 million USDC, upper_cap = 10 million USDC
  • Safety: lower_cap = 2 million USDC, upper_cap = 5 million USDC

Relevant pull requests:

  • OIMF protos and genesis values ([#1125](#1125))
  • Modify GetInitialMarginQuoteQuantums to reflect OIMF ([#1159](#1159))
  • Calculate current OI and pass to GetMarginRequirements calculation ([#1161](#1161))
  • Evaluate subaccounts update end state with OIMF ([#1172](#1172))
  • OIMF: Simplify interface for GetDeltaOpenInterestFromUpdates ([#1227](#1227))
  • Add upgrade handler for OIMF caps ([#1232](#1232))
  • Update OI after fill and added unit tests ([#1231](#1231))
  • Update LiquidityTierUpsertEvent to add OI caps ([#1242](#1242))
  • Add upgrade handler to initialize OI during upgrade handler ([#1302](#1302))
  • Update OI caps to new values ([#1351](#1351))
  • update caps in genesis file ([#1355](#1355))
  • update DB and APIs to add OI caps info ([#1305](#1305))
  • Add open interest update event and handlers ([#1352](#1352))
  • Add QueryAllLiquidityTiers to CLI ([#1434](#1434))
  • Refactor perpetual upgrade handlers in v5.0.0 ([#1442](#1442))
  • Update iavl with WorkingHash fix ([#1484](#1484))

Order Rate Limits

Rate Limits for short term orders and cancels were combined. Previously, rate limits were set to be 200 short term place orders over 1 block and 200 short term cancel orders over 1 block. To relax rate limits and allow for burst placing of orders, place and cancel order rates were combined and moved to a 5-block window. Max is now (200 + 200) * 5 = 2000 combined short term place and cancel orders over 5 blocks.

  • Combine place, cancel, batch cancel rate limiters ([#1165](#1165))
  • Fix upgrade rate limit config logic, add helpful logs ([#1212](#1212))

Full Node Streaming

A new feature that allows the full node to stream data directly to subscribers via GRPC without needing to go through indexer Websocket. Full node streaming currently supports L3 order book data, with complete order activities needed to construct a local orderbook.

Note that only orderbook data is streamed, and fills are not streamed in this current version. Streaming of fills will be added to v5.0.1

  • stream offchain updates through stream manager ([#1138](#1138))
  • add command line flag for full node streaming ([#1145](#1145))
  • construct the initial orderbook snapshot ([#1147](#1147))
  • separate indexer and grpc streaming events ([#1209](#1209))
  • only send response when there is at least one update ([#1216](#1216))
  • send order update when short term order state fill amounts a… ([#1241](#1241))
  • send fill amount updates for reverted operations in prepare check state ([#1240](#1240))
  • add block number + stage to grpc updates ([#1252](#1252))
  • avoid state reads when sending updates ([#1261](#1261))
  • send updates for both normal order matches and liquidation ([#1280](#1280))
  • decouple grpc and non validating full node flags ([#1400](#1400))

Performance Improvements

These changes improve the performance of the protocol without significant changes to the functionality.

  • Migrate over to cosmos version now that performance improvement had been released. ([#1007](#1007))
  • Push CheckTx concurrency into protocol ([#1030](#1030))
  • Cache current collateralization and margin requirements per subaccount ([#1064](#1064))
  • Increase maximum number of connections ([#1080](#1080))
  • Reduce the cost of adding an indexer event. ([#1078](#1078))
  • Update Cosmos SDK and add parallel query tests. ([#1086](#1086))
  • Optimize string building loop to write to a single buffer instead of to multiple buffers that are concatenated.
    ([#1119](#1119))
  • Create ante handlers which parallelize processing per account until a global lock is necessary. ([#1108](#1108))
  • Upgrade cosmos fork to use libsecp256k1 from go-ethereum ([#1210](#1210))
  • Store pruneable orders in a key-per-order format rather than key-per-height ([#1230](#1230))
  • Fix bugs in pruneable order migration ([#1250](#1250))
  • Skip add to book collat check upon placement for conditional orders ([#1369](#1369))

Slinky Sidecar/Vote Extension

Adding Oracle Prices to Vote Extensions allow oracle prices to be more accurate and for price updates to be deterministic and non-consensus blocking. [Skip’s Slinky Sidecar](https://docs.dydx.exchange/validators/running_a_validator#slinky-sidecar) separates oracle prices to be run separately from the dYdX Validator binary. The Sidecar is adds support for Raydium oracle price feeds.

  • feat: Slinky sidecar integration ([#1109](#1109))
  • feat: VoteExtension Slinky logic ([#1139](#1139))
  • feat: Proposal logic for Slinky ([#1135](#1135))
  • feat: Enforce MarketParam.Pair uniqueness constraint ([#1193](#1193))
  • feat: Slinky full integration PR ([#1141](#1141))
  • bug: Handle paginated querying to prices-keeper in slinky + pricefeed daemon ([#1177](#1177))
  • Remove smoothed prices ([#1215](#1215))
  • perf: optimize slinky_adapter via in-memory cache [SKI-13] ([#1182](#1182))
  • Bump slinky version to v0.3.1 ([#1275](#1275))
  • Add v5.0.0 enable vote extensions upgrade handler ([#1308](#1308))
  • fix: Fix v5 upgrade VE handler ([#1468](#1468))

Bug fixes

  • [bugfix][protocol] fix settled funding event emission ([#1051](#1051))
  • Fix --home flag being ignored in cosmos v0.50 ([#1053](#1053))
  • Fix unnecessary home directory creation issues ([#1073](#1073))
  • Upgrade iavl to v1.0.1 to prune orphan nodes async ([#1081](#1081))
  • Upgrade client/v2 to fix autocli signing issues ([#1090](#1090))
  • Don't attempt to write the store during simulated transactions. ([#1122](#1122))
  • Disable fast nodes and update iavl to address bugs ([#1173](#1173))
  • Fix lib.ErrorLogWithError ([#1306](#1306))
  • Upgrade IAVL to fix issues saving to disk ([#1357](#1357))
  • Fix stateful order removal event ordering. ([#1456](#1456))
  • Stop passing in app PreBlocker to slinky VE handler ([#1491](#1491))

Miscellaneous

  • [feature] Add request_id to all code paths using lib.UnwrapSDKContext ([#1058](#1058))
  • Metric for when subaccounts are created ([#1071](#1071))
  • deprecate pessimistic add-to-book collateralization check ([#1107](#1107))
  • Update deleveraging handler for DeleveragingEventV1 ([#1121](#1121))
  • Add v5 upgrade handler and set up container upgrade test ([#1153](#1153))
  • Emit metrics gated through execution mode ([#1157](#1157))
  • Update go version to 1.22 in [README.md](http://readme.md/) ([#1196](#1196))
  • Add query for PendingSendPacket ([#1176](#1176))
  • Upgrade cosmos fork to v0.50.5 ([#1246](#1246))
  • Move price cache population to Preblocker ([#1417](#1417))
  • Move clob hydration to preblocker ([#1412](#1412))
  • Upgrade to newly rebased cometbft fork ([#1438](#1438))
  • Increase conditional order trigger multiplier to 25 ([#1460](#1460))
  • Add a cli to convert module name to address ([#1462](#1462))

Indexer v5.0.0 Release Notes

Isolated Markets

  • Add market type to the CreatePerpetual API ([#1118](#1118))
  • Add parentSubaccountNumber endpoint for fills ([#1205](#1205))
  • Add market type to indexer events. ([#1201](#1201))
  • Implement GET /address/parentSubaccountNumber API ([#1229](#1229))
  • Implement /transfers/parentSubaccountNumber indexer API ([#1218](#1218))
  • Add API to get asset positions for parent subaccount ([#1243](#1243))
  • Update indexer to ingest and return market type for perpetuals ([#1266](#1266))
  • Add V2 for perpetual market create event + handlers. (backport #1395) ([#1402](#1402))
  • Add parentSubaccountNumber endpoint for historical-pnl (backport #1476) ([#1485](#1485))
  • Add parent subaccount websocket (backport #1463) ([#1486](#1486))
  • Fix subaccount validation to account for updated subaccount limit. (backport #1513) ([#1515](#1515))

Soft Open Interest Caps

Performance Improvements

  • Batch Process Vulcan messages ([#1018](#1018))
  • Optimize funding index update query ([#1061](#1061))
  • Add Indexes to candles and orders to optimize queries ([#1060](#1060))
  • add index that includes createdAt to fills to address read replica lag ([#1137](#1137))
  • Use read replica rather than main instance for aggregate-trading-rewards query ([#1190](#1190))
  • pass initial kafka timestamp from Ender -> Vulcan for better e2e order latency tracking ([#1211](#1211))
  • Enable rate limiting in socks by default ([#1272](#1272))
  • Indexer track e2e latency ([#1237](#1237))

Compliance

Implement persistent geo-blocking and close-only mode in Indexer

  • remove geoblocking from Indexer endpoints ([#1029](#1029))
  • add compliance status table ([#1032](#1032))
  • add roundtable task for CLOSE_ONLY -> BLOCKED compliance status transition ([#1034](#1034))
  • new compliance logic for comlink address-related endpoints ([#1048](#1048))
  • implement new geo-blocking strategy for socks ([#1049](#1049))
  • implement GET /compliance/screen/{address} endpoint ([#1087](#1087))
  • implement POST /compliance/setStatus for dev/staging environments ([#1088](#1088))
  • Implement skeleton for post /compliance/geoblock endpoint ([#1111](#1111))
  • implement post /compliance/geoblock ([#1129](#1129))
  • Fix Compliance types and validation (backport #1411) ([#1413](#1413))
  • Add updatedAt to compliance v2 API responses (backport #1360) ([#1480](#1480))
  • Update upsert querybuilder and compliance table upsert function (backport #1477) ([#1478](#1478))
  • implement logic for FIRST_STRIKE_CLOSE_ONLY mode (backport #1372) ([#1479](#1479))

Batch Cancel

  • msgBatchCancel success + failure fields ([#1130](#1130))

Bug Fixes

  • Fix Aggregate Historical Trading rewards and always use generated id in tables ([#1022](#1022))
  • Add missing index ([#1039](#1039))
  • Try-catch update functions in cache-update loops. ([#1042](#1042))
  • move funding handler from unordered_handlers to ordered_handlers ([#1097](#1097))
  • Ensure that all matching status are returned in list orders endpoint ([#1100](#1100))
  • Fix stateful order removal event ordering. (backport #1456) ([#1459](#1459))

Miscellaneous

  • Add print-block cmd to scripts README ([#1096](#1096))
  • add DELEVERAGING event type to print-block script ([#1120](#1120))
  • Add script to print candle time boundaries ([#1126](#1126))
  • Update Api documentation to have correct required fields ([#1167](#1167))
  • Add flag to send subaccount websocket message for long term orders in vulcan ([#1224](#1224))
  • Add flag to send subaccount websocket message from Ender for long term orders ([#1223](#1223))
  • Add BEC to initial subaccount orders socks channel (backport #1500) ([#1504](#1504))
Assets 4