feat: Design and implement monitoring for price discrepancies across exchanges [DEV-1300]#29
Conversation
Signed-off-by: jay-dee7 <jasdeepsingh.uppal@gmail.com>
Signed-off-by: jay-dee7 <jasdeepsingh.uppal@gmail.com>
Contributor
Author
|
@ankurdotb now using deployed version of |
ankurdotb
requested changes
Oct 10, 2022
Collaborator
ankurdotb
left a comment
ankurdotb
left a comment
There was a problem hiding this comment.
Needs secrets in deploy stage workflow
ankurdotb
changed the title
ankurdotb
added a commit
that referenced
this pull request
Oct 12, 2022
commit 2e5ba95 Author: Ankur Banerjee <ankurdotb@users.noreply.github.com> Date: Wed Oct 12 22:51:19 2022 +0100 ci: Decouple production deploy from staging commit a9fc8c2 Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Wed Oct 12 09:20:00 2022 +0100 chore(deps-dev): Bump wrangler from 2.1.9 to 2.1.11 (#34) Bumps [wrangler](https://github.com/cloudflare/wrangler2/tree/HEAD/packages/wrangler) from 2.1.9 to 2.1.11. - [Release notes](https://github.com/cloudflare/wrangler2/releases) - [Changelog](https://github.com/cloudflare/wrangler2/blob/main/packages/wrangler/CHANGELOG.md) - [Commits](https://github.com/cloudflare/wrangler2/commits/wrangler@2.1.11/packages/wrangler) --- updated-dependencies: - dependency-name: wrangler dependency-type: direct:development update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> commit 1e0b027 Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Wed Oct 12 08:56:16 2022 +0100 chore(deps): Bump itty-router from 2.6.5 to 2.6.6 (#30) Bumps [itty-router](https://github.com/kwhitley/itty-router) from 2.6.5 to 2.6.6. - [Release notes](https://github.com/kwhitley/itty-router/releases) - [Changelog](https://github.com/kwhitley/itty-router/blob/v2.x/CHANGELOG.md) - [Commits](https://github.com/kwhitley/itty-router/commits/v2.6.6) --- updated-dependencies: - dependency-name: itty-router dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> commit f2a10c3 Author: Benyam Seifu <45682486+benyam7@users.noreply.github.com> Date: Wed Oct 12 03:26:34 2022 +0300 feat: Design and implement monitoring for price discrepancies across exchanges [DEV-1300] (#29) * Price discrepancy API * Use direct value return * Function checks * fix: use forEach instead of for loop in Prices API * Update .gitignore * Dynamic CoinGecko marketplaces * chore: switch to env vars and minor changes * chore: add readme for prices endpoint * chore: change arbitrage threshold * Set GET handler for CoinGecko API * Correct spelling mistake * Rename to arbitrage * Rename Arbitrage import * Set CoinGecko ID in Wrangler * Set CoinGecko ID in binding * Update wrangler.toml * Remove CHEQ_COIN_ID from constants * Removed constants file * Update currency.ts * Rename to CHEQ_COIN_ID to COINGECKO_ID * Install CoinGecko API package * feat: move market monitor logic to DO functions * chore: update readme and deps * Updated documentation for arbitrage * Fix typo in marketMonitoring types * chore: switch to the deployed version of market monitoring API * chore: restructure response and add /all endpoint for data dump * fix: market monitoring endpoint * chore: change to camelCase * chore: remove access token for market API * chore: add WEBHOOK_URL secret to deploy workflow * Add webhook URL secret to staging deploy * Rename marketing monitoring API constants * Rename coinPriceHandler * Make constants match * Remove concurrency guard * Change staging cron schedule Signed-off-by: jay-dee7 <jasdeepsingh.uppal@gmail.com> Co-authored-by: nikola-cheqd <nikola@cheqd.io> Co-authored-by: jay-dee7 <jasdeepsingh.uppal@gmail.com> Co-authored-by: Ankur Banerjee <ankurdotb@users.noreply.github.com>
ankurdotb
added a commit
that referenced
this pull request
Nov 4, 2022
* Switch upstream GraphQL API endpoint * ci: Update upstream API endpoints * Simplified get accounts query so that it doesn't break * Fixed bad query * Improved error handling for graphql client * wip * Fixed circulating supply endpoint (still missing rewards and unbonding) * Switch upstream GraphQL API endpoint * ci: Update upstream API endpoints * Fixed bad query * Created new endpoint to update single account balance on KV for later usage * Added total balance as field on account balance cache * Update bigDipperApi.ts * Revert "Update bigDipperApi.ts" This reverts commit 50b08ee. * Refactoring for balance fetching from different sources * Fixed account balance helper * Started with new graphql query * Fixed update balance per account endpoint (temporary) * Fixed balance query checks * Fixed circulating supply endpoint to work with new query * Fixed update all balances endpoint to feed KV (temporary endpoint) * Better error handling so that API doesn't crash * Fixed all balances updater * Only updating missing balances (temp) * Reactivating cron * Removed single account balance updater temp endpoint * Improved error logging for graphgl (workers requires a Map for logging objs) * Removing duplicates from KV while procecssing * Removed temporary account removal from KV * Better logging * Fixed endpoint * Fixed endpoint * Added proper list of addresses to import (temp) * Implementing requested changes from PR review * Made group be random on cron * Simplified record on KV * Better naming and place for cron helper * Better error handling * Removed unused code * Better yet error handling * Fixed error issue returning bad response * Clean-up of addresses * Reprocessing addresses * Removed static account addresses * Removed unecessary endpoint for processing balance groups * Added check for null account on balance endpoint * Added updated at to KV record * Squashed commit of the following: commit 2e5ba95 Author: Ankur Banerjee <ankurdotb@users.noreply.github.com> Date: Wed Oct 12 22:51:19 2022 +0100 ci: Decouple production deploy from staging commit a9fc8c2 Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Wed Oct 12 09:20:00 2022 +0100 chore(deps-dev): Bump wrangler from 2.1.9 to 2.1.11 (#34) Bumps [wrangler](https://github.com/cloudflare/wrangler2/tree/HEAD/packages/wrangler) from 2.1.9 to 2.1.11. - [Release notes](https://github.com/cloudflare/wrangler2/releases) - [Changelog](https://github.com/cloudflare/wrangler2/blob/main/packages/wrangler/CHANGELOG.md) - [Commits](https://github.com/cloudflare/wrangler2/commits/wrangler@2.1.11/packages/wrangler) --- updated-dependencies: - dependency-name: wrangler dependency-type: direct:development update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> commit 1e0b027 Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Wed Oct 12 08:56:16 2022 +0100 chore(deps): Bump itty-router from 2.6.5 to 2.6.6 (#30) Bumps [itty-router](https://github.com/kwhitley/itty-router) from 2.6.5 to 2.6.6. - [Release notes](https://github.com/kwhitley/itty-router/releases) - [Changelog](https://github.com/kwhitley/itty-router/blob/v2.x/CHANGELOG.md) - [Commits](https://github.com/kwhitley/itty-router/commits/v2.6.6) --- updated-dependencies: - dependency-name: itty-router dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> commit f2a10c3 Author: Benyam Seifu <45682486+benyam7@users.noreply.github.com> Date: Wed Oct 12 03:26:34 2022 +0300 feat: Design and implement monitoring for price discrepancies across exchanges [DEV-1300] (#29) * Price discrepancy API * Use direct value return * Function checks * fix: use forEach instead of for loop in Prices API * Update .gitignore * Dynamic CoinGecko marketplaces * chore: switch to env vars and minor changes * chore: add readme for prices endpoint * chore: change arbitrage threshold * Set GET handler for CoinGecko API * Correct spelling mistake * Rename to arbitrage * Rename Arbitrage import * Set CoinGecko ID in Wrangler * Set CoinGecko ID in binding * Update wrangler.toml * Remove CHEQ_COIN_ID from constants * Removed constants file * Update currency.ts * Rename to CHEQ_COIN_ID to COINGECKO_ID * Install CoinGecko API package * feat: move market monitor logic to DO functions * chore: update readme and deps * Updated documentation for arbitrage * Fix typo in marketMonitoring types * chore: switch to the deployed version of market monitoring API * chore: restructure response and add /all endpoint for data dump * fix: market monitoring endpoint * chore: change to camelCase * chore: remove access token for market API * chore: add WEBHOOK_URL secret to deploy workflow * Add webhook URL secret to staging deploy * Rename marketing monitoring API constants * Rename coinPriceHandler * Make constants match * Remove concurrency guard * Change staging cron schedule Signed-off-by: jay-dee7 <jasdeepsingh.uppal@gmail.com> Co-authored-by: nikola-cheqd <nikola@cheqd.io> Co-authored-by: jay-dee7 <jasdeepsingh.uppal@gmail.com> Co-authored-by: Ankur Banerjee <ankurdotb@users.noreply.github.com> * Update wrangler.toml * Fixed broken trigger * fix total delegations count per validatior * fix totat delegators count * fix total staked coins * Update wrangler.toml * fix: total balance * Fix endpoint * fix total balance * fix circulating supply * chore refactor a bit * fix node api (include unbounding/delegated free balance for totalBalance * make total balance endpoint to consider unbounding balance * Squashed commit of the following: commit ff235a6 Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Tue Oct 18 11:08:46 2022 +0100 chore(deps-dev): Bump @cloudflare/workers-types from 3.16.0 to 3.17.0 (#36) Bumps [@cloudflare/workers-types](https://github.com/cloudflare/workers-types) from 3.16.0 to 3.17.0. - [Release notes](https://github.com/cloudflare/workers-types/releases) - [Changelog](https://github.com/cloudflare/workers-types/blob/master/CHANGELOG.md) - [Commits](cloudflare/workers-types@v3.16.0...v3.17.0) --- updated-dependencies: - dependency-name: "@cloudflare/workers-types" dependency-type: direct:development update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore store account balance infos in KV * fix total balance * trigger cron based on current * remove CSW KV and change trigger interval for staging * remove :delayed * fix wrangler.toml * fix total balance * decrease height by one * lowwer chances of getting total balance to be 0 * Fix typo in available balance * Fix typo in unbonding balance * Convert available balance to CHEQ * Update staging trigger to every hour * get total delegations balance from node api * total unbonding balance for delegator from node api * fix calculate unbonding balance * get account balance infos from node for totalBalance * update account balance infos from node api * add a logic to calculate vesting and vested(taken form wallet app) * update vesting and vested endpoints to get result from newly added logic * review changes * fix a bug when calculating avaliable balance * remove getting account details from gql * create staging KV for total delegators count * add helper methods to do caching * update total delegators count in cron * make total delegators count read from cache if avaliable * add preview for CSW KV for local dev * switch to node api for getting delegators count per validator * refacor total delegators count a bit * remove total delegators count kv * setup KV for active validators * add a logic to add and remove validators from kv * set up KV for total delegators * paginate and get all delegators for a validator * add a logic to add or remove delegators in KV * change total delegators to read from cache instead * set 10 as group for active validators * setup a trigger to update delegators to validators * add preview id for new KVs * change delegators count endpoint to use paginated response * update next_key after next fetch * clean up pagination key * paginate using offset when getting delegators per validator * remove newlines and spaces from query * paginate using offeset to get total delegated amount * paginate using offest when getting total unboding amount * don't export private functions * move active validators from param func block * call add and remove active validators on webtrigger * fix syntax error * fix error at: create_hashmap_of_validators_addresses_from_kv * Delete get_delegator_count_for_validator * Rename PAGINATION_LIMIT * Pagination limit binding * Standardise all node queries * Correct pagination limit * add logs when updating total delegators * get total supply from gql * Remove preview KV pairs * Restructure URL params * Fix missing pagination limit variable in staging * Fix typo in unbonding methods * Fix typo in originalVesting * Remove unused delayed vesting functions - Remove marked_as_delayed_vesting_account - Remove filter_marked_as_account_types * Temporarily log request URL for validator delegators * fix adding up total delegations and unbondings * use Map instead of object literal * Reduce staging cron to 1 hour * create a new KV for active validators on staging * get voting power for active validator * reuse active validators resp * store voting power as well as total delegator count for a validator in KV * add new validator to smallest validator group in KV * try getting delegators count for validator from kv first * cast all numeric constants to Number again. I decided to do this after noticing some requests like this: https://api.cheqd.net/cosmos/staking/v1beta1/validators/cheqdvaloper1akqyyul8q32nzxa2yzkenex0q2l4dx6udpjfdf/delegations?pagination.count_total=false&pagination.limit=50&pagination.offset=050 As you can see offset is set to "050", which should be the case of our Numeric constants being are considered as String. I highly doubt this was happing because we're wrapping our Numeric constants in quotes in toml file. * Remove total delegators Remove from KV bindings, helpers, handlers * Cleanup BigDipper API and types * Remove extra ValidatorDetailResponse type * Clean up balance helper function * Rename balanceGroup to circulating * Add RewardsResponse type * Rename circulating supply helper * Remove unused function in node helper * Remove unused import in vesting * Update delegatorCount.ts * refactor webhook triggers * refactor circulating * refactor delegators count * cleanup vesting * refactor helpers: balance * refactor helpers node * move group and address extraction to separate file * remove old kv as we dont need pre-populated kv * add limit optional param * fix adding new validator to kv logic * fix creating map logic * remove sending arbitrage as it is causing hook to fail * remove additional check when adding validator to KV * add additional logs * remove old pre populated validators kv for prod * Fix imports * Remove unncessary helper * Update balance.ts * Remove total delegators endpoint from README * Bump wrangler * Add cache cleanup scripts * catch all cron exceptions * remove voting count * Add circulating supply link * add more logs * Remove active validator group binding * add more logs * Update readme for API changes * extract delegator count * Update nodeApi.ts * Trying another logging method * Logging using map * try catch api call * remove unused type * add a additional cron for testing * run updating active validators kv on separate cron * put only one active validator on kv to lessen load * Disable market monitoring and circulating supply triggers * Set triggrer to 2 mins * Deactivate adding validators * Update .gitignore * Log delete stale * Update dispatch.yml * Update validators.ts * Delete validators helper * Update circulating supply groups * Activate circulating supply trigger * Update group prefix * Remove unused functions in nodeApi * Removed unused BigDipper API * Change market monitoring functions to PascalCase * Set node API methods to camelCase * Rename circulating supply * Delete delegatorCount.ts * Rename to isValidAddress * Rename to isValidAddress * Rename to isDelayedVestingAccount * Rename to convertToLargestDenom * Rename to convertToMainTokenDenom * Rename to fetchAccountBalance * Rename to calculateTotalUnbondingBalance * Rename to extractPrefixAndKey * Delete unused function * Rename to isContinuousVestingAccount * Rename to calculateVestedBalance * Simplify vesting account calculation * Update graphql.ts * Update node.ts * Change cron triggers * Reactivate markdown link check * Reactivate linter * Update pull-request.yml * Fix broken links * Bump to Node 18 * Removed unused handler * Throw error on circulating supply Co-authored-by: drgomesp <drgomesp@gmail.com> Co-authored-by: Benyam Seifu <45682486+benyam7@users.noreply.github.com>
cheqd-bot Bot
pushed a commit
that referenced
this pull request
Mar 13, 2025
## [2.0.0](1.0.0...2.0.0) (2025-03-13) ### ⚠ BREAKING CHANGES * Add support for non-circulating-addresses ### Features * Add analytics endpoints to fetch identity data [DEV-4774] ([#269](#269)) ([6ba0f6a](6ba0f6a)), closes [#270](#270) * Add support for non-circulating-addresses ([b775822](b775822)) * Circulating supply API and vesting API fixes [DEV-1388] ([#7](#7)) ([a3e4a12](a3e4a12)) * Create circulating supply API [DEV-1028] ([3a26551](3a26551)) * Delegator Count for Validators ([#8](#8)) ([63830b6](63830b6)) * Design and implement monitoring for price discrepancies across exchanges [DEV-1300] ([#29](#29)) ([f2a10c3](f2a10c3)) * Remove market monitoring API and update documentation for identity analytics [DEV-4788] ([#271](#271)) ([5d01d69](5d01d69)) * Staking rewards APIs ([#11](#11)) ([209d5f2](209d5f2)) * Store identity-related transactions into relational DB [DEV-4773] ([#261](#261)) ([6ea1e62](6ea1e62)) ### Bug Fixes * Circulating supply response ([#41](#41)) ([3b930d3](3b930d3)) * Drop active validators KV ([#42](#42)) ([bcadfc5](bcadfc5)) * Filter by denom when displaying total account balance [DEV-3904] ([#197](#197)) ([5e58dce](5e58dce)) * Fix circulating supply API endpoint [DEV-1821] ([#35](#35)) ([301769e](301769e)), closes [#38](#38) [#37](#37) * **graphql:** Broken upstream API fix [DEV-1726] ([#22](#22)) ([d1365d8](d1365d8)) * Removed mis-configs & added bindings ([29b2b66](29b2b66)) * Removed mis-configs & added bindings ([3e0092c](3e0092c)) * Update production KV ([e9ff2aa](e9ff2aa))
|
🎉 This PR is included in version 2.0.0 🎉 The release is available on GitHub release Your semantic-release bot 📦🚀 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR creates :
/arbitrage/allendpoint that returns current price of CHEQ token across different markets, a boolean flag to indicate if there is at least on arbitrage opportunity passing the threshold and a possible arbitrage opportunities./arbitrageendpoint that returns a array of possible arbitrage opportunities.It uses our monitor market api which is a digital ocean app that uses coin gecko api to get latest price for CHEQ token. market monitoring api does the logic for calculating possible arbitrage opportunities and returns data to be consumed by our cosmos custom data-api
It does also triggers a Webhook when the
market arbitrage thresholdis greater or equals to10. @ankurdotb time to setup the zap with webhook URL.