Feat/add provider curve #4
Conversation
fix: non-determinitic from id
…t and apply metadata
feat: implement price aggregation with geckoterminal
add: add provider, curve
WalkthroughThis change introduces support for two new providers: the Curve DeFi API and the Bitget exchange WebSocket. It adds provider configurations, handler implementations, utility functions, and message parsing logic for both. The Curve API handler is integrated into the API query handler factory, with comprehensive tests and metadata structures included. For Bitget, WebSocket handler logic, message formats, parsing, and default configurations are implemented and registered in the WebSocket query handler factory. Additionally, a utility for converting float64 to big.Float is added. Changes
Sequence Diagram(s)sequenceDiagram
participant User
participant ProvidersConfig
participant CurveAPIHandler
participant BitgetWSHandler
participant APIQueryHandlerFactory
participant WSQueryHandlerFactory
User->>ProvidersConfig: Loads provider list
ProvidersConfig->>CurveAPIHandler: Initializes Curve API config
ProvidersConfig->>BitgetWSHandler: Initializes Bitget WS config
User->>APIQueryHandlerFactory: Requests API handler for Curve
APIQueryHandlerFactory->>CurveAPIHandler: Instantiates handler
User->>WSQueryHandlerFactory: Requests WS handler for Bitget
WSQueryHandlerFactory->>BitgetWSHandler: Instantiates handler
User->>CurveAPIHandler: CreateURL, ParseResponse
User->>BitgetWSHandler: CreateMessages, HandleMessage, HeartBeatMessages
Poem
📜 Recent review detailsConfiguration used: CodeRabbit UI 📒 Files selected for processing (1)
🚧 Files skipped from review as they are similar to previous changes (1)
🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 6
🧹 Nitpick comments (14)
providers/websockets/bitget/utils.go (1)
3-6: Reorder imports according to style conventions.Go's standard convention is to group imports with standard library first, then a blank line, followed by third-party imports.
import ( - "github.com/skip-mev/connect/v2/oracle/config" "time" + + "github.com/skip-mev/connect/v2/oracle/config" )🧰 Tools
🪛 golangci-lint (1.64.8)
4-4: File is not properly formatted
(gofumpt)
5-5: File is not properly formatted
(gofumpt)
providers/apis/defi/curve/utils.go (3)
31-39: Example JSON comment should end with a period.Minor style issue - comments should end with a period for consistency.
//{ // "data": { // "address": "0xCd5fE23C85820F7B72D0926FC9b05b43E359b7ee", // "usd_price": 1674.1742629502855, // "last_updated": "2025-04-16T06:04:23" // } - //} + //}.🧰 Tools
🪛 golangci-lint (1.64.8)
32-32: Comment should end in a period
(godot)
40-42: RenameCurveResponsetoResponseto avoid stutter.The type name stutters with the package name. In Go, when the package name is already
curve, the type should be called justResponse.- CurveResponse struct { + Response struct { Data CurveData `json:"data"` }🧰 Tools
🪛 golangci-lint (1.64.8)
[warning] 40-40: exported: type name will be used as curve.CurveResponse by other packages, and that stutters; consider calling this Response
(revive)
44-48: RenameCurveDatatoDatato avoid stutter.The type name stutters with the package name. In Go, when the package name is already
curve, the type should be called justData.- CurveData struct { + Data struct { Address string `json:"address"` UsdPrice float64 `json:"usd_price"` LastUpdated string `json:"last_updated"` }🧰 Tools
🪛 golangci-lint (1.64.8)
[warning] 44-44: exported: type name will be used as curve.CurveData by other packages, and that stutters; consider calling this Data
(revive)
providers/apis/defi/curve/types.go (1)
3-8: RenameCurveMetadatatoMetadatato avoid stuttering.The type name stutters with the package name. In Go, when the package name is already
curve, the type should be called justMetadata.-type CurveMetadata struct { +type Metadata struct { Network string `json:"network"` PoolID string `json:"pool_id"` BaseTokenAddress string `json:"base_token_address"` QuoteTokenAddress string `json:"quote_token_address"` }🧰 Tools
🪛 golangci-lint (1.64.8)
[warning] 3-3: exported: type name will be used as curve.CurveMetadata by other packages, and that stutters; consider calling this Metadata
(revive)
providers/factories/oracle/api.go (1)
6-7: Fix formatting issue flagged by static analysis.There's a minor formatting issue detected by gofumpt. The imports should be properly grouped and ordered.
- "go.uber.org/zap" - "net/http" + "net/http" + "go.uber.org/zap"🧰 Tools
🪛 golangci-lint (1.64.8)
6-6: File is not properly formatted
(gofumpt)
providers/websockets/bitget/parse.go (1)
13-44: Early return may skip processing other tickers.The function logic is generally well-structured, but there's a potential issue with the error handling. When a price conversion error is encountered for a ticker (lines 31-38), the function returns immediately, potentially skipping the processing of other tickers in the same message.
Consider modifying the error handling to continue processing other tickers even when one fails, rather than returning early.
func (h *WebSocketHandler) parseTickerUpdate( resp TickerUpdateMessage, ) (types.PriceResponse, error) { var ( resolved = make(types.ResolvedPrices) unresolved = make(types.UnResolvedPrices) ) if !strings.Contains(resp.Arg.Channel, string(TickerChannel)) { return types.NewPriceResponse(resolved, unresolved), fmt.Errorf("invalid channel %s", resp.Arg.Channel) } for _, data := range resp.Data { ticker, ok := h.cache.FromOffChainTicker(data.InstId) if !ok { return types.NewPriceResponse(resolved, unresolved), fmt.Errorf("unknown ticker %s", data.InstId) } price, err := math.Float64StringToBigFloat(data.LastPr) if err != nil { wErr := fmt.Errorf("failed to convert price to big.Float: %w", err) unresolved[ticker] = providertypes.UnresolvedResult{ ErrorWithCode: providertypes.NewErrorWithCode(wErr, providertypes.ErrorFailedToParsePrice), } - return types.NewPriceResponse(resolved, unresolved), nil + continue } resolved[ticker] = types.NewPriceResult(price, time.Now().UTC()) } return types.NewPriceResponse(resolved, unresolved), nil }providers/apis/defi/curve/api_handler_test.go (2)
182-191: Add t.Helper() to test helper function.This helper function should follow Go test convention by starting with t.Helper() to improve error reporting.
func createTickerWithMetadata(t *testing.T, base, quote string, metadata CurveMetadata) types.ProviderTicker { + t.Helper() metadataBytes, err := json.Marshal(metadata) require.NoError(t, err) return types.DefaultProviderTicker{ OffChainTicker: base + "/" + quote, JSON: string(metadataBytes), } }🧰 Tools
🪛 golangci-lint (1.64.8)
182-182: Comment should end in a period
(godot)
183-183: test helper function should start from t.Helper()
(thelper)
5-10: Fix import formatting.The imports section has inconsistent spacing between import groups.
import ( "encoding/json" "github.com/skip-mev/connect/v2/providers/base/testutils" providertypes "github.com/skip-mev/connect/v2/providers/types" "net/http" "testing" "time" - "github.com/skip-mev/connect/v2/oracle/config" "github.com/skip-mev/connect/v2/oracle/types" "github.com/stretchr/testify/require" )🧰 Tools
🪛 golangci-lint (1.64.8)
5-5: File is not properly formatted
(gofumpt)
10-10: File is not properly formatted
(gofumpt)
providers/websockets/bitget/messages.go (4)
48-52: Field naming inconsistency in ArgsData struct.The field
InstIdshould follow Go naming conventions and be namedInstID.type ArgsData struct { InstType string `json:"instType"` Channel string `json:"channel"` - InstId string `json:"instId"` + InstID string `json:"instId"` }🧰 Tools
🪛 golangci-lint (1.64.8)
[warning] 51-51: var-naming: struct field InstId should be InstID
(revive)
68-72: Update field reference if InstId is renamed.If you rename the
InstIdfield as suggested above, ensure this reference is also updated.args[j] = ArgsData{ InstType: "SPOT", Channel: string(TickerChannel), - InstId: tickers[start+j], + InstID: tickers[start+j], }
97-104: Incomplete comment for SubscriptionResponse.The comment shows the JSON structure but is cut off without closing brackets.
//{ // "event": "subscribe", // "arg": { // "instType": "SPOT", // "channel": "ticker", // "instId": "BTCUSDT" //} +//}
144-160: Field naming inconsistency in TickerUpdateData struct.Similar to ArgsData, the field
InstIdshould follow Go naming conventions and be namedInstID.type TickerUpdateData struct { - InstId string `json:"instId"` + InstID string `json:"instId"` LastPr string `json:"lastPr"` Open24H string `json:"open24h"` High24H string `json:"high24h"` Low24H string `json:"low24h"` Change24H string `json:"change24h"` BidPr string `json:"bidPr"` AskPr string `json:"askPr"` BidSz string `json:"bidSz"` AskSz string `json:"askSz"` BaseVolume string `json:"baseVolume"` QuoteVolume string `json:"quoteVolume"` OpenUtc string `json:"openUtc"` ChangeUtc24H string `json:"changeUtc24h"` Ts int64 `json:"ts,string"` // this is quoted string in JSON }🧰 Tools
🪛 golangci-lint (1.64.8)
[warning] 145-145: var-naming: struct field InstId should be InstID
(revive)
providers/websockets/bitget/ws_data_handler.go (1)
53-53: Remove commented out code.There's a commented out line that should be removed for code cleanliness.
var ( resp types.PriceResponse subscribeResp SubscriptionResponse - //baseResp BaseResponse update TickerUpdateMessage )🧰 Tools
🪛 golangci-lint (1.64.8)
53-53: commentFormatting: put a space between
//and comment text(gocritic)
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (12)
cmd/constants/providers.go(4 hunks)pkg/math/math.go(1 hunks)providers/apis/defi/curve/api_handler.go(1 hunks)providers/apis/defi/curve/api_handler_test.go(1 hunks)providers/apis/defi/curve/types.go(1 hunks)providers/apis/defi/curve/utils.go(1 hunks)providers/factories/oracle/api.go(2 hunks)providers/factories/oracle/websocket.go(2 hunks)providers/websockets/bitget/messages.go(1 hunks)providers/websockets/bitget/parse.go(1 hunks)providers/websockets/bitget/utils.go(1 hunks)providers/websockets/bitget/ws_data_handler.go(1 hunks)
🧰 Additional context used
🧬 Code Graph Analysis (5)
providers/factories/oracle/websocket.go (2)
providers/websockets/bitget/utils.go (1)
Name(9-9)providers/websockets/bitget/ws_data_handler.go (1)
NewWebSocketDataHandler(24-45)
providers/apis/defi/curve/api_handler_test.go (7)
oracle/config/api.go (1)
APIConfig(9-48)providers/apis/defi/curve/utils.go (2)
DefaultAPIConfig(20-29)Name(14-14)providers/apis/defi/curve/api_handler.go (1)
NewAPIHandler(24-43)oracle/types/provider.go (2)
ProviderTicker(15-22)DefaultProviderTicker(27-30)providers/apis/defi/curve/types.go (1)
CurveMetadata(3-8)providers/base/testutils/http.go (1)
CreateResponseFromJSON(23-26)providers/types/errors.go (1)
ErrorWithCode(75-78)
providers/websockets/bitget/utils.go (2)
oracle/config/websocket.go (8)
WebSocketConfig(72-144)DefaultReconnectionTimeout(15-15)DefaultReadBufferSize(24-24)DefaultWriteBufferSize(29-29)DefaultEnableCompression(36-36)DefaultReadTimeout(40-40)DefaultMaxReadErrorCount(59-59)DefaultMaxSubscriptionsPerBatch(68-68)providers/websockets/okx/utils.go (3)
ReadTimeout(45-45)WriteInterval(32-32)MaxSubscriptionsPerBatch(42-42)
providers/websockets/bitget/ws_data_handler.go (6)
oracle/types/oracle.go (2)
PriceWebSocketDataHandler(62-62)PriceResponse(73-73)oracle/config/websocket.go (1)
WebSocketConfig(72-144)oracle/types/provider.go (3)
ProviderTickers(33-37)NewProviderTickers(66-76)ProviderTicker(15-22)providers/websockets/bitget/utils.go (1)
Name(9-9)providers/base/websocket/handlers/ws_conn_handler.go (1)
WebsocketEncodedMessage(16-16)providers/websockets/bitget/messages.go (4)
SubscriptionResponse(105-108)TickerUpdateMessage(139-142)OperationPing(23-23)OperationSubscribe(19-19)
providers/apis/defi/curve/api_handler.go (8)
oracle/types/oracle.go (7)
PriceAPIDataHandler(43-43)PriceResponse(73-73)NewPriceResponseWithErr(96-96)ResolvedPrices(76-76)UnResolvedPrices(79-79)NewPriceResult(87-87)NewPriceResponse(93-93)oracle/config/api.go (1)
APIConfig(9-48)oracle/types/provider.go (3)
ProviderTickers(33-37)NewProviderTickers(66-76)ProviderTicker(15-22)providers/apis/defi/curve/utils.go (3)
Name(14-14)URL(17-17)CurveResponse(40-42)providers/apis/defi/curve/types.go (1)
CurveMetadata(3-8)providers/types/errors.go (4)
NewErrorWithCode(90-95)ErrorFailedToDecode(23-23)ErrorNoResponse(18-18)ErrorWithCode(75-78)pkg/math/math.go (1)
Float64ToBigFloat(170-174)providers/types/response.go (1)
UnresolvedResult(62-64)
🪛 golangci-lint (1.64.8)
providers/factories/oracle/api.go
6-6: File is not properly formatted
(gofumpt)
9-9: File is not properly formatted
(gofumpt)
providers/apis/defi/curve/types.go
[warning] 3-3: exported: type name will be used as curve.CurveMetadata by other packages, and that stutters; consider calling this Metadata
(revive)
providers/apis/defi/curve/utils.go
[warning] 40-40: exported: type name will be used as curve.CurveResponse by other packages, and that stutters; consider calling this Response
(revive)
[warning] 44-44: exported: type name will be used as curve.CurveData by other packages, and that stutters; consider calling this Data
(revive)
32-32: Comment should end in a period
(godot)
providers/apis/defi/curve/api_handler_test.go
182-182: Comment should end in a period
(godot)
5-5: File is not properly formatted
(gofumpt)
10-10: File is not properly formatted
(gofumpt)
183-183: test helper function should start from t.Helper()
(thelper)
providers/websockets/bitget/utils.go
4-4: File is not properly formatted
(gofumpt)
5-5: File is not properly formatted
(gofumpt)
providers/websockets/bitget/ws_data_handler.go
53-53: commentFormatting: put a space between // and comment text
(gocritic)
providers/websockets/bitget/messages.go
[warning] 51-51: var-naming: struct field InstId should be InstID
(revive)
[warning] 145-145: var-naming: struct field InstId should be InstID
(revive)
⏰ Context from checks skipped due to timeout of 90000ms (1)
- GitHub Check: Analyze (go)
🔇 Additional comments (28)
providers/websockets/bitget/utils.go (2)
8-14: LGTM! Constants are well-defined.The constants are appropriately named and documented, with clear purposes for the Bitget WebSocket provider.
16-34: LGTM! Configuration is comprehensive.The WebSocket configuration is well-structured with appropriate default values and Bitget-specific settings where needed. The
DefaultPingIntervalof 15 seconds aligns with WebSocket best practices for maintaining connections.providers/apis/defi/curve/utils.go (2)
9-18: LGTM! Well-documented API specification.The API constants are well-defined with helpful documentation links and examples that clearly indicate the source and format of the API.
20-29: LGTM! API configuration looks appropriate.The timeout and interval settings look reasonable for an external API. The 500ms timeout and 20-second interval provide a good balance between responsiveness and avoiding excessive requests.
pkg/math/math.go (1)
169-174: LGTM! Well-implemented conversion utility.This function provides a clean, straightforward way to convert float64 values to big.Float, consistent with the other conversion functions in the file. The implementation is correct and follows the established pattern.
providers/factories/oracle/websocket.go (2)
17-17: Import added for the new Bitget WebSocket provider.The import for the Bitget WebSocket package has been added appropriately, maintaining alphabetical ordering.
104-105: Bitget WebSocket provider integration looks good.The implementation follows the standard pattern used for other WebSocket providers in the factory function. The Bitget handler is correctly instantiated with the required logger and configuration.
providers/factories/oracle/api.go (2)
17-17: Import added for the new Curve DeFi API provider.The import for the Curve DeFi API package has been added appropriately.
101-102: Curve DeFi API provider integration looks good.The implementation follows the standard pattern used for other API providers in the factory function. The Curve API handler is correctly instantiated with the required configuration.
cmd/constants/providers.go (4)
12-12: Import added for the new Curve DeFi API provider.The import for the Curve DeFi API package has been added appropriately.
24-24: Import added for the new Bitget WebSocket provider.The import for the Bitget WebSocket package has been added appropriately.
61-65: Curve DeFi API provider configuration looks good.The Curve provider configuration is correctly added to the DEFI providers section and follows the same structure as other provider entries, with Name, API config, and Type properly specified.
169-173: Bitget WebSocket provider configuration looks good.The Bitget WebSocket provider configuration is correctly added to the Exchange WebSocket providers section and follows the same structure as other provider entries, with Name, WebSocket config, and Type properly specified.
providers/websockets/bitget/parse.go (2)
21-23: Channel validation is correct.The channel validation check correctly ensures that only ticker channel messages are processed.
31-31: Good use of Float64StringToBigFloat utility.The conversion from string to big.Float using the math utility function is appropriate for handling high-precision financial data.
providers/apis/defi/curve/api_handler_test.go (2)
16-56: Well-structured test cases for API handler initialization.The test suite effectively covers both successful initialization and failure cases (wrong API name, disabled API). Good use of table-driven tests to systematically verify handler creation behavior.
58-127: Comprehensive URL creation tests.The test cases thoroughly validate URL creation logic with appropriate checks for empty tickers and required metadata fields (Network, BaseTokenAddress). The assertions properly verify both error cases and successful URL formation.
providers/apis/defi/curve/api_handler.go (3)
24-43: Good validation in constructor.The constructor properly validates the API configuration with clear error messages, checking that the name matches, the API is enabled, and the basic configuration is valid.
68-120: ParseResponse implementation with correct error handling.The method appropriately handles various error cases and constructs a well-formed price response with resolved and unresolved tickers.
45-66:❓ Verification inconclusive
CreateURL only uses the first ticker.
The method takes a slice of tickers but only processes the first one. This approach assumes all tickers in the batch use the same network, which might be a limitation.
Verify whether multiple tickers with different networks should be supported or if the current implementation is intentional. If supporting multiple tickers is needed, consider:
🏁 Script executed:
#!/bin/bash # Check if there are other similar API handlers that process multiple tickers echo "Checking for similar API handlers that process multiple tickers..." grep -r "CreateURL" --include="*.go" ./providers/apis/ | grep -v curve | grep -A 10 "tickers \[\]types.ProviderTicker"Length of output: 261
Clarify multi-ticker support in CreateURL
The handler’s signature accepts a slice of tickers but only ever uses the first element, which assumes all provided tickers share the same network. This could lead to incorrect URLs if multiple tickers on different networks are passed.
Please confirm which behavior is intended:
- This method will only ever receive a single ticker (in which case consider updating the signature to accept a single
ProviderTicker), or- It should support multiple tickers (and networks), in which case you’ll need to iterate over the slice (or group by network) and build URLs accordingly.
File needing attention:
- providers/apis/defi/curve/api_handler.go (lines 45–66)
providers/websockets/bitget/messages.go (3)
12-27: Well-organized type definitions and constants.The code defines clear types and constants for operations and channels, making the intention explicit and the code readable.
32-42: Useful JSON structure documentation.The code includes helpful comments showing the expected JSON structure for subscription requests, which aids understanding.
54-89: Efficient batch processing for subscription requests.Good implementation of batched subscription requests respecting the max subscriptions per batch limit. The error handling is thorough, including checking for empty tickers and marshaling errors.
providers/websockets/bitget/ws_data_handler.go (5)
16-22: Clear struct definition with proper fields.The WebSocketHandler struct contains all necessary fields for handling WebSocket connections, including a logger, configuration, and cache.
24-45: Robust constructor with thorough validation.The NewWebSocketDataHandler function properly validates the WebSocket configuration, ensuring the name matches, the WebSocket is enabled, and the configuration is valid.
47-83: HandleMessage method with appropriate error handling.The method adequately handles different message types (ping, subscription, ticker updates) with proper error handling and logging.
🧰 Tools
🪛 golangci-lint (1.64.8)
53-53: commentFormatting: put a space between
//and comment text(gocritic)
102-108: Correctly implemented Copy method.The Copy method creates a new instance with a fresh ticker cache while preserving the logger and configuration, which is important for thread safety.
72-76:❌ Incorrect review comment
Missing reference to parseTickerUpdate function.
The method calls
h.parseTickerUpdate, but this function is not defined in this file. Ensure this function exists in another file within the package.
🏁 Script executed:
#!/bin/bash # Check if parseTickerUpdate is defined in another file echo "Checking for parseTickerUpdate definition..." grep -r "func.*parseTickerUpdate" --include="*.go" ./providers/websockets/bitget/Length of output: 267
parseTickerUpdate is already defined in the package
The methodh.parseTickerUpdateis implemented inproviders/websockets/bitget/parse.go, so no missing reference exists.Likely an incorrect or invalid review comment.
Summary by CodeRabbit