feat/Add Defira connector #5422

navijation · 2022-06-19T22:24:45Z

Before submitting this PR, please make sure:

Your code builds clean without any errors or warnings
You are using approved title ("feat/", "fix/", "docs/", "refactor/")

A description of the changes proposed in the pull request:

Add Defira gateway connector and manual curl tests
Add schema and validation tests for Defira config file
Fix bug in Harmony chain which messed up gas estimates
Add WONE74 token used by Defira testnet to sushiswap testnet file to allow manual tests to pass
Add dependency on fixed commit of https://github.com/zuzu-cat/defira-sdk for Defira pairs and trades

Tests performed by the developer:

Get sell / buy price on Defira mainnet using USDC and FIRA pair (should be roughly 2.365 FIRA for USDC)
Perform sell / buy trade on Defira testnet using WONE74 (the token used to bootstrap testnet DEX) and 1ETH
Added unit tests for Defira routes, core, and schema validation

Tips for QA testing:

Make sure that token config file is setup to accomodate Defira properly (default is Sushiswap tokens; add more tokens
as needed or switch to the Defira testnet tokens JSON in harmony.yaml)

Add Defira gateway scaffold w/ correct ABI

…nectors-api feat/Add Defira to `GET /connectors` API

Feat/Add Defira mainnet tokens to config

Add correct Defira testnet DEX address

Rename Defira connector, fix testnet router address

Add @types/node-fetch dev dep

…hema Feat/Add Defira config schema and init code hash

arnelhbot · 2022-07-14T03:29:28Z

PR Update: Upon testing we have noticed that an Unhandled error in background task: 'harmony' error happens when we try to connect an existing wallet with Mainnet and Testnet.

2022-07-14 04:25:29 | error | 	unhandledRejection: Client network socket disconnected before secure TLS connection was established
Error: Client network socket disconnected before secure TLS connection was established
    at connResetException (node:internal/errors:683:14)
    at TLSSocket.onConnectEnd (node:_tls_wrap:1577:19)
    at TLSSocket.emit (node:events:406:35)
    at endReadableNT (node:internal/streams/readable:1329:12)
    at processTicksAndRejections (node:internal/process/task_queues:83:21) | Error: Client network socket disconnected before secure TLS connection was established
    at connResetException (node:internal/errors:683:14)
    at TLSSocket.onConnectEnd (node:_tls_wrap:1577:19)
    at TLSSocket.emit (node:events:406:35)
    at endReadableNT (node:internal/streams/readable:1329:12)
    at processTicksAndRejections (node:internal/process/task_queues:83:21)

Steps to reproduce:

Delete docker instance in .hummingbot-gateway
Stop and Delete docker instance Hummingbot-gateway-xxxxxx
Start client and call gateway create and enter Infura API
Enter gateway connect and select Defira/mainnet
Enter gateway connect and select Defira/testnet
When prompted to connect with existing wallet, type yes

logs_gateway_app.log.2022-07-14.zip

navijation · 2022-07-15T17:00:03Z

@arnelhbot Are you sure this is related to my PR? I didn't make any changes to the Harmony chain aside from the gas price calculation and the getSpender stuff.

rapcmia · 2022-07-18T07:17:21Z

@arnelhbot Are you sure this is related to my PR? I didn't make any changes to the Harmony chain aside from the gas price calculation and the getSpender stuff.

Hi @navneethjayendran yes, looks like it has something to do with Harmony chain and outside of the PR. We are investigating this atm since this is the first PR connector for the Harmony chain.

arnelhbot · 2022-07-18T12:33:47Z

PR Update: Up testing we noticed that when starting the AMM_Arbitrage strategy connected to Harmony mainnet wallet, we receive an Unhandled error in background task.

2022-07-19 01:30:26 | error | 	The spender param is not a valid Ethereum address (0x followed by 40 hexidecimal characters). | Error: The spender param is not a valid Ethereum address (0x followed by 40 hexidecimal characters).
    at throwIfErrorsExist (/usr/src/app/dist/src/services/validators.js:38:15)
    at /usr/src/app/dist/src/services/validators.js:85:40
    at /usr/src/app/dist/src/evm/evm.routes.js:32:61
    at Generator.next (<anonymous>)
    at /usr/src/app/dist/src/evm/evm.routes.js:8:71
    at new Promise (<anonymous>)
    at __awaiter (/usr/src/app/dist/src/evm/evm.routes.js:4:12)
    at /usr/src/app/dist/src/evm/evm.routes.js:31:90
    at /usr/src/app/dist/src/services/error-handler.js:37:28
    at Layer.handle [as handle_request] (/usr/src/app/node_modules/express/lib/router/layer.js:95:5)

Steps to reproduce:

Connect Harmony mainnet wallet
Create AMM_Arbitrage strategy
Start strategy and observe log pane for error

logs_gateway_app.log.2022-07-18.zip

Faouzijedidi1 · 2022-07-18T12:58:05Z

@navneethjayendran To fix the current error that @arnelhbot faced. You need to add defira to the validated spenders in the harmony.validators.ts lines 40-46. Similar to how sushiswap and viperswap are validated add defira there. This File

navijation · 2022-07-18T23:49:01Z

@navneethjayendran To fix the current error that @arnelhbot faced. You need to add defira to the validated spenders in the harmony.validators.ts lines 40-46. Similar to how sushiswap and viperswap are validated add defira there. This File

Thanks for the reminder. I forgot to test this step E2E. Added unit test coverage.

arnelhbot · 2022-07-21T13:58:13Z

PR Update: Upon testing, we have noticed we are still getting the same The spender param is not a valid Ethereum address error. We are also seeing a new error missing response when running bot and waiting for allowance

2022-07-22 02:48:54 | error | 	missing response (requestBody="{\"method\":\"eth_getBalance\",\"params\":[\"0x08940dc9b5a19fab9319b77c61dda7b8067e6843\",\"latest\"],\"id\":45,\"jsonrpc\":\"2.0\"}", requestMethod="POST", serverError={"code":"ECONNRESET","path":null,"host":"api.s0.t.hmny.io","port":443}, url="https://api.s0.t.hmny.io/", code=SERVER_ERROR, version=web/5.6.1) | Error: missing response (requestBody="{\"method\":\"eth_getBalance\",\"params\":[\"0x08940dc9b5a19fab9319b77c61dda7b8067e6843\",\"latest\"],\"id\":45,\"jsonrpc\":\"2.0\"}", requestMethod="POST", serverError={"code":"ECONNRESET","path":null,"host":"api.s0.t.hmny.io","port":443}, url="https://api.s0.t.hmny.io/", code=SERVER_ERROR, version=web/5.6.1)
    at Logger.makeError (/usr/src/app/node_modules/@ethersproject/logger/lib/index.js:233:21)
    at Logger.throwError (/usr/src/app/node_modules/@ethersproject/logger/lib/index.js:242:20)
    at /usr/src/app/node_modules/@ethersproject/web/lib/index.js:252:36
    at step (/usr/src/app/node_modules/@ethersproject/web/lib/index.js:33:23)
    at Object.throw (/usr/src/app/node_modules/@ethersproject/web/lib/index.js:14:53)
    at rejected (/usr/src/app/node_modules/@ethersproject/web/lib/index.js:6:65)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)

Steps to reproduce:

Connect Harmony mainnet wallet
Create AMM_Arbitrage strategy
Start strategy and observe log pane for error

logs_gateway_app.log.2022-07-21.zip

navijation · 2022-07-27T03:05:04Z

@Faouzijedidi1 It seems like the Harmony chain integration is problematic since things are failing on operations that have nothing to do with the connector, e.g. reusing an existing wallet. Can we meet and discuss the best solution to these issues?

arnelhbot · 2022-08-02T07:51:32Z

Upon testing new updates, we are getting a async_utils - Unhandled error in background task message once we starting the bot.

2022-08-02 07:39:29,419 - 1466638 - hummingbot.core.utils.async_utils - ERROR - Unhandled error in background task: Error on POST https://localhost:21701/evm/allowances Error: {'message': 'The spender param is not a valid Ethereum address (0x followed by 40 hexidecimal characters).', 'httpErrorCode': 404, 'errorCode': -1, 'stack': 'Error: The spender param is not a valid Ethereum address (0x followed by 40 hexidecimal characters).\n    at throwIfErrorsExist (/usr/src/app/dist/src/services/validators.js:38:15)\n    at /usr/src/app/dist/src/services/validators.js:85:40\n    at /usr/src/app/dist/src/evm/evm.routes.js:32:61\n    at Generator.next (<anonymous>)\n    at /usr/src/app/dist/src/evm/evm.routes.js:8:71\n    at new Promise (<anonymous>)\n    at __awaiter (/usr/src/app/dist/src/evm/evm.routes.js:4:12)\n    at /usr/src/app/dist/src/evm/evm.routes.js:31:90\n    at /usr/src/app/dist/src/services/error-handler.js:37:28\n    at Layer.handle [as handle_request] (/usr/src/app/node_modules/express/lib/router/layer.js:95:5)'}
Traceback (most recent call last):
  File "/home/ubuntu/Hummingbot/pull_request/5422/hummingbot/core/utils/async_utils.py", line 9, in safe_wrapper
    return await c
  File "/home/ubuntu/Hummingbot/pull_request/5422/hummingbot/connector/gateway_EVM_AMM.py", line 290, in auto_approve
    await self.update_allowances()
  File "/home/ubuntu/Hummingbot/pull_request/5422/hummingbot/connector/gateway_EVM_AMM.py", line 340, in update_allowances
    self._allowances = await self.get_allowances()
  File "/home/ubuntu/Hummingbot/pull_request/5422/hummingbot/connector/gateway_EVM_AMM.py", line 348, in get_allowances
    resp: Dict[str, Any] = await self._get_gateway_instance().get_allowances(
  File "/home/ubuntu/Hummingbot/pull_request/5422/hummingbot/core/gateway/gateway_http_client.py", line 341, in get_allowances
    return await self.api_request("post", "evm/allowances", {
  File "/home/ubuntu/Hummingbot/pull_request/5422/hummingbot/core/gateway/gateway_http_client.py", line 194, in api_request
    raise e
  File "/home/ubuntu/Hummingbot/pull_request/5422/hummingbot/core/gateway/gateway_http_client.py", line 182, in api_request
    raise ValueError(f"Error on {method.upper()} {url} Error: {parsed_response}")

Steps to reproduce:

Cloned Feature branch with no issues
Manually created Docker image in gateway folder
Generated-cert and gateway create with no issues
Successfully connected Defira mainnet/testnet
Configured Harmony NodeURL https://api.harmony.one
Created simple AMM_Arbitrage strategy and started bot
Observe log pane for error

Defira_conf_amm_arb_1.log

navijation · 2022-08-02T15:18:06Z

@arnelhbot I see the problem. There are red herring validators for the Harmony chain that don't actually do anything. I edited the spender validator there instead of the actually used Ethereum validators. It is now fixed and I am able to do AMM_ARB with Defira against itself.

arnelhbot

Test performed:

Cloned Feature branch with no issues
Created Docker image in gateway folder
Generated-cert and gateway create with no issues
Successfully connected Defira mainnet/testnet
Created simple AMM_Arbitrage strategy and started bot
Confirmed arbitrage opportunity and filled order
Txhash visible in ledger
History confirmed displaying order filled

Bring in Defikingdoms changes

arnelhbot

Test performed:

Cloned Feature branch with no issues
Created Docker image in gateway folder
Generated-cert and gateway create with no issues
Successfully connected Defira mainnet/testnet
DefiKingdoms visible as a connector
Created simple AMM_Arbitrage strategy and started bot
Confirmed arbitrage opportunity and filled order
Txhash visible in ledger
History confirmed displaying order filled

navijation and others added 30 commits May 10, 2022 21:09

Add Defira gateway scaffold w/ correct ABI

7b169a1

Revert some spurious renaming

120f269

Use Harmony as chain

9070edb

Merge pull request #1 from NavneethJayendran/gateway-connector-scaffold

b4a89b9

Add Defira gateway scaffold w/ correct ABI

Fix gateway build

8988be0

feat remove v3 files and edit defira config

b584f8e

Add Defira to GET /connectors

b448266

Merge pull request #2 from NavneethJayendran/defira/add-defira-to-con…

3cebbc8

…nectors-api feat/Add Defira to `GET /connectors` API

Add starter Tranquil token list

00b5b83

Add logo URIs from Sushiswap tokens list

f7983a5

Merge pull request #3 from NavneethJayendran/feat/add-tokens-json

7273314

Feat/Add Defira mainnet tokens to config

Update Defira address

04d156a

Add correct Defira testnet DEX address

6f57718

Merge pull request #4 from NavneethJayendran/add-defira-testnet-address

51e20a2

Add correct Defira testnet DEX address

Rename Defira connector, fix testnet router address

a29b064

Add newline

ba582f9

Remove one newline

9c3a3a7

Merge pull request #5 from NavneethJayendran/add-defira-testnet-address

b5c1710

Rename Defira connector, fix testnet router address

Add defira-sdk dep

0909651

Revert to functioning code with TODOs

7858375

Add @types/node-fetch dep

5fecef0

Merge pull request #10 from NavneethJayendran/types-node-fetch-dep

f2f7986

Add @types/node-fetch dev dep

Add @types/node-fetch dep

25170bc

Add Defira gateway config schema

ddc3372

defira.yaml -> defira.yml

4648ebd

Fix JSON schema and YAML access

4d88353

Fix reference to renamed method

d9651e9

Merge pull request #9 from NavneethJayendran/nav/add-defira-config-sc…

7c6cf77

…hema Feat/Add Defira config schema and init code hash

Add @types/node-fetch dep

e6256fa

Add Defira gateway config schema

80de234

navijation and others added 3 commits July 15, 2022 10:06

Sed out Harmony RPC URL in test workflows

8446952

Merge branch 'development' into defira-connector

e99ce7b

Fix merge related issues

875196f

Update spender validator to support defira

886fb5f

Merge branch 'development' into defira-connector

1a32159

navijation added 5 commits July 31, 2022 11:00

Merge branch 'development' into defira-connector

b3f9e7d

Fix transactionGasLimit bug in ethereum-base

e35a9b8

gasLimit -> gasLimitEstimate

5e966cc

Fix lingering gasLimit -> gasLimitEstimate

5be52a2

Add Harmony native token to client map

c1229aa

Fix actual getSpender validation

632ca68

Merge branch 'development' into defira-connector

19ecdb9

arnelhbot previously approved these changes Aug 3, 2022

View reviewed changes

JeremyKono and others added 2 commits August 5, 2022 22:29

Merge branch 'development' into defira-connector

e16b952

Merge branch 'development' into defira-connector

d135db1

Bring in Defikingdoms changes

navijation dismissed arnelhbot’s stale review via d135db1 August 5, 2022 15:32

arnelhbot approved these changes Aug 9, 2022

View reviewed changes

Merge branch 'development' into defira-connector

99ae46d

Faouzijedidi1 approved these changes Aug 10, 2022

View reviewed changes

JeremyKono merged commit 6a4f759 into hummingbot:development Aug 11, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat/Add Defira connector #5422

feat/Add Defira connector #5422

navijation commented Jun 19, 2022 •

edited

arnelhbot commented Jul 14, 2022

navijation commented Jul 15, 2022

rapcmia commented Jul 18, 2022 •

edited

arnelhbot commented Jul 18, 2022

Faouzijedidi1 commented Jul 18, 2022

navijation commented Jul 18, 2022

arnelhbot commented Jul 21, 2022 •

edited

navijation commented Jul 27, 2022

arnelhbot commented Aug 2, 2022

navijation commented Aug 2, 2022

arnelhbot left a comment •

edited

arnelhbot left a comment •

edited

feat/Add Defira connector #5422

feat/Add Defira connector #5422

Conversation

navijation commented Jun 19, 2022 • edited

arnelhbot commented Jul 14, 2022

navijation commented Jul 15, 2022

rapcmia commented Jul 18, 2022 • edited

arnelhbot commented Jul 18, 2022

Faouzijedidi1 commented Jul 18, 2022

navijation commented Jul 18, 2022

arnelhbot commented Jul 21, 2022 • edited

navijation commented Jul 27, 2022

arnelhbot commented Aug 2, 2022

navijation commented Aug 2, 2022

arnelhbot left a comment • edited

Choose a reason for hiding this comment

arnelhbot left a comment • edited

Choose a reason for hiding this comment

navijation commented Jun 19, 2022 •

edited

rapcmia commented Jul 18, 2022 •

edited

arnelhbot commented Jul 21, 2022 •

edited

arnelhbot left a comment •

edited

arnelhbot left a comment •

edited