Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Resolve proposal parsing issues [DEV-1966] #69

Merged
merged 16 commits into from
Dec 7, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
129 changes: 129 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,129 @@
## Unreleased
### Changes

#### Parse Command
- ([\#492](https://github.com/forbole/bdjuno/pull/492)) Add parse command for periodic tasks: `x/bank` total supply, `x/distribution` community pool, `x/mint` inflation, `pricefeed` token price and price history, `x/staking` staking pool

#### Upgrade Module
- ([\#467](https://github.com/forbole/bdjuno/pull/467)) Store software upgrade plan and refresh data at upgrade height

#### Staking Module
- ([\#443](https://github.com/forbole/bdjuno/pull/443)) Remove tombstone status from staking module(already stored in slashing module)
- ([\#455](https://github.com/forbole/bdjuno/pull/455)) Added `unbonding_tokens` and `staked_not_bonded_tokens` values to staking pool table

#### Gov Module
- ([\#461](https://github.com/forbole/bdjuno/pull/461)) Parse `x/gov` genesis with `genesisDoc.InitialHeight` instead of the hard-coded height 1
- ([\#465](https://github.com/forbole/bdjuno/pull/465)) Get open proposal ids in deposit or voting period by block time instead of current time
- ([\#489](https://github.com/forbole/bdjuno/pull/489)) Remove block height foreign key from proposal_vote and proposal_deposit tables and add column timestamp
- ([\#499](https://github.com/forbole/bdjuno/pull/499)) Check if proposal has passed voting end time before marking it invalid

#### Daily refetch
- ([\#454](https://github.com/forbole/bdjuno/pull/454)) Added `daily refetch` module to refetch missing blocks every day

#### Hasura
- ([\#473](https://github.com/forbole/bdjuno/pull/473)) Improved Hasura permissions
- ([\#491](https://github.com/forbole/bdjuno/pull/491)) Add host address to Hasura actions

### Dependencies
- ([\#462](https://github.com/forbole/bdjuno/pull/462)) Updated Juno to `v3.4.0`

## Version v3.2.0
### Changes
#### Mint module
- ([\#432](https://github.com/forbole/bdjuno/pull/432)) Update inflation rate when mint param change proposal is passed

#### Gov module
- ([\#401](https://github.com/forbole/bdjuno/pull/401)) Update the proposal status to the latest in `bdjuno parse gov proposal [id]` command
- ([\#430](https://github.com/forbole/bdjuno/pull/430)) Update the proposals that have invalid status but can still be in voting or deposit periods

### Dependencies
- ([\#440](https://github.com/forbole/bdjuno/pull/440)) Updated Juno to `v3.3.0`

## Version v3.1.0
### Dependencies
- Updated Juno to `v3.2.0`

### Changes
#### Hasura
- ([\#395](https://github.com/forbole/bdjuno/pull/395)) Remove time label from Hasura Prometheus monitoring

#### Bank module
- ([\#410](https://github.com/forbole/bdjuno/pull/410)) Change total supply query from only 1 page to all pages

## Version v3.0.1
### Dependencies
- Updated Juno to `v3.1.1`

## Version v3.0.0
### Notes
This version introduces breaking changes to `transaction` and `message` PostgreSQL tables. It implements PostgreSQL table partitioning to fix slow data retrieval from database that stores large amount of transactions and messages. Read more details about [migrating to v3.0.0](https://docs.bigdipper.live/cosmos-based/parser/migrations/v2.0.0)

### New features
#### CLI
- ([\#356](https://github.com/forbole/bdjuno/pull/356)) Implemented `migrate` command to perform easy migration to higher BDJuno versions
- ([\#356](https://github.com/forbole/bdjuno/pull/356)) Updated `parse-genesis` command to parse genesis file without accessing the node

#### Database
- ([\#356](https://github.com/forbole/bdjuno/pull/356)) Added PostgreSQL table partition to `transaction` and `message` table
- ([\#356](https://github.com/forbole/bdjuno/pull/356)) Created new `messages_by_address` function

### Changes
#### Hasura
- ([\#377](https://github.com/forbole/bdjuno/pull/377)) Updated Hasura metadata
- ([\#381](https://github.com/forbole/bdjuno/pull/381)) Hasura actions are now a module

### Dependencies
- ([\#356](https://github.com/forbole/bdjuno/pull/356)) Updated Juno to `v3.0.0`

## Version v2.0.0
### Notes
This version introduces breaking changes to certain address-specific data that is no longer periodically parsed from the node and stored in the database. Instead, the data is now obtained directly from the node when needed using Hasura Actions. Read more details about [migrating to v2.0.0](https://docs.bigdipper.live/cosmos-based/parser/migrations/v2.0.0)

### New features
#### CLI
- ([\#257](https://github.com/forbole/bdjuno/pull/257)) Added `parse-genesis` command to parse the genesis file
- ([\#228](https://github.com/forbole/bdjuno/pull/228)) ([\#248](https://github.com/forbole/bdjuno/pull/248)) Added `fix` command:
- `auth`: fix vesting accounts details
- `blocks`: fix missing blocks and transactions from given start height
- `gov`: fix proposal with given proposal ID
- `staking`: fix validators info at the latest height

#### Hasura Actions
- ([\#329](https://github.com/forbole/bdjuno/pull/329)) Implemented Hasura Actions service to replace periodic queries. If you are using GraphQL queries on your application, you should updated the old queries to use the below new actions instead.
Here's a list of data acquired through Hasura Actions:
- Of a certain address/delegator:
- Account balance (`action_account_balance`)
- Delegation rewards (`action_delegation_reward`)
- Delegator withdraw address (`action_delegator_withdraw_address`)
- Delegations (`action_delegation`)
- Total delegations amount (`action_delegation_total`)
- Unbonding delegations (`action_unbonding_delegation`)
- Total unbonding delegations amount (`action_unbonding_delegation_total`)
- Redelegations (`action_redelegation`)
- Of a certain validator:
- Commission amount (`action_validator_commission_amount`)
- Delegations to this validator (`action_validator_delegations`)
- Redelegations from this validator (`action_validator_redelegations_from`)
- Unbonding delegations (`action_validator_unbonding_delegations`)
- ([\#352](https://github.com/forbole/bdjuno/pull/352)) Added prometheus monitoring to hasura actions

#### Local node support
- Added the support for `node.type = "local"` for parsing a static local node without the usage gRPC queries: [config reference](https://docs.bigdipper.live/cosmos-based/parser/config/config#node).

#### Modules
- ([\#232](https://github.com/forbole/bdjuno/pull/232)) Updated the `x/auth` module support to handle and store `vesting accounts` and `vesting periods` inside the database.
- ([\#276](https://github.com/forbole/bdjuno/pull/276)) Added the support for the `x/feegrant` module (v0.44.x)

### Changes

#### CLI
- ([\#351](https://github.com/forbole/bdjuno/pull/351)) Fixed version display for `bdjuno version` cmd

#### Database
- ([\#300](https://github.com/forbole/bdjuno/pull/300)) Changed `bonded_tokens` and `not_bonded_tokens` type inside `staking_pool` table to `TEXT` to avoid value overflow
- ([\#275](https://github.com/forbole/bdjuno/pull/275)) Added `tombstoned` column inside `validator_status` table
- ([\#232](https://github.com/forbole/bdjuno/pull/232)) Added `vesting_account` and `vesting_period` table
- ([\#276](https://github.com/forbole/bdjuno/pull/276)) Added `fee_grant_allowance` table (v0.44.x)

#### Modules
- ([\#353](https://github.com/forbole/bdjuno/pull/353)) Removed the support for the `history` module
1 change: 0 additions & 1 deletion database/gov.go
Original file line number Diff line number Diff line change
Expand Up @@ -403,7 +403,6 @@ WHERE proposal_validator_status_snapshot.height <= excluded.height`

// SaveSoftwareUpgradePlan allows to save the given software upgrade plan with its proposal id
func (db *Db) SaveSoftwareUpgradePlan(proposalID uint64, plan upgradetypes.Plan, height int64) error {

stmt := `
INSERT INTO software_upgrade_plan(proposal_id, plan_name, upgrade_height, info, height)
VALUES ($1, $2, $3, $4, $5)
Expand Down
23 changes: 11 additions & 12 deletions database/gov_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -223,10 +223,10 @@ func (suite *DbTestSuite) TestBigDipperDb_GetOpenProposalsIds() {
"proposalType1",
content2,
types.ProposalStatusInvalid,
time.Date(2020, 1, 2, 00, 00, 00, 000, time.UTC),
time.Date(2020, 1, 2, 01, 00, 00, 000, time.UTC),
time.Date(2020, 1, 2, 02, 00, 00, 000, time.UTC),
time.Date(2020, 1, 2, 03, 00, 00, 000, time.UTC),
time.Date(2020, 1, 2, 0o0, 0o0, 0o0, 0o00, time.UTC),
time.Date(2020, 1, 2, 0o1, 0o0, 0o0, 0o00, time.UTC),
time.Date(2020, 1, 2, 0o2, 0o0, 0o0, 0o00, time.UTC),
time.Date(2020, 1, 2, 0o3, 0o0, 0o0, 0o00, time.UTC),
proposer2.String(),
)

Expand Down Expand Up @@ -348,9 +348,9 @@ func (suite *DbTestSuite) TestBigDipperDb_SaveDeposits() {
depositor3 := suite.getAccount("cosmos1gyds87lg3m52hex9yqta2mtwzw89pfukx3jl7g")
amount3 := sdk.NewCoins(sdk.NewCoin("desmos", sdk.NewInt(50000)))

timestamp1 := time.Date(2020, 1, 1, 15, 00, 00, 000, time.UTC)
timestamp2 := time.Date(2020, 1, 1, 16, 00, 00, 000, time.UTC)
timestamp3 := time.Date(2020, 1, 1, 17, 00, 00, 000, time.UTC)
timestamp1 := time.Date(2020, 1, 1, 15, 0o0, 0o0, 0o00, time.UTC)
timestamp2 := time.Date(2020, 1, 1, 16, 0o0, 0o0, 0o00, time.UTC)
timestamp3 := time.Date(2020, 1, 1, 17, 0o0, 0o0, 0o00, time.UTC)

deposit := []types.Deposit{
types.NewDeposit(proposal.ProposalID, depositor.String(), amount, timestamp1, 10),
Expand Down Expand Up @@ -414,7 +414,7 @@ func (suite *DbTestSuite) TestBigDipperDb_SaveVote() {
proposal := suite.getProposalRow(1)
voter := suite.getAccount("cosmos1z4hfrxvlgl4s8u4n5ngjcw8kdqrcv43599amxs")

timestamp := time.Date(2020, 1, 1, 15, 00, 00, 000, time.UTC)
timestamp := time.Date(2020, 1, 1, 15, 0o0, 0o0, 0o00, time.UTC)

vote := types.NewVote(1, voter.String(), govtypes.OptionYes, timestamp, 1)
err := suite.database.SaveVote(vote)
Expand Down Expand Up @@ -838,7 +838,7 @@ func (suite *DbTestSuite) TestBigDipperDb_SaveSoftwareUpgradePlan() {

// ----------------------------------------------------------------------------------------------------------------
// Save software upgrade plan at height 10 with upgrade height at 100
var plan = upgradetypes.Plan{
plan := upgradetypes.Plan{
Name: "name",
Height: 100,
Info: "info",
Expand Down Expand Up @@ -917,7 +917,7 @@ func (suite *DbTestSuite) TestBigDipperDb_DeleteSoftwareUpgradePlan() {
_ = suite.getProposalRow(1)

// Save software upgrade plan at height 10 with upgrade height at 100
var plan = upgradetypes.Plan{
plan := upgradetypes.Plan{
Name: "name",
Height: 100,
Info: "info",
Expand All @@ -934,14 +934,13 @@ func (suite *DbTestSuite) TestBigDipperDb_DeleteSoftwareUpgradePlan() {
err = suite.database.Sqlx.Select(&rows, `SELECT * FROM software_upgrade_plan`)
suite.Require().NoError(err)
suite.Require().Len(rows, 0)

}

func (suite *DbTestSuite) TestBigDipperDb_CheckSoftwareUpgradePlan() {
_ = suite.getProposalRow(1)

// Save software upgrade plan at height 10 with upgrade height at 100
var plan = upgradetypes.Plan{
plan := upgradetypes.Plan{
Name: "name",
// the Height here is the upgrade height
Height: 100,
Expand Down
4 changes: 2 additions & 2 deletions database/types/upgrade.go
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
package types

type SoftwareUpgradePlanRow struct {
PlanName string `db:"plan_name"`
Info string `db:"info"`
ProposalID uint64 `db:"proposal_id"`
PlanName string `db:"plan_name"`
UpgradeHeight int64 `db:"upgrade_height"`
Info string `db:"info"`
Height int64 `db:"height"`
}

Expand Down
4 changes: 2 additions & 2 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,9 @@ module github.com/forbole/bdjuno/v3
go 1.18

require (
github.com/cheqd/cheqd-node v0.7.3
github.com/cheqd/cheqd-node v0.6.9
github.com/cosmos/cosmos-sdk v0.45.9
github.com/cosmos/gaia/v7 v7.1.0
github.com/cosmos/gaia/v7 v7.0.2
github.com/forbole/juno/v3 v3.4.0
github.com/go-co-op/gocron v1.18.0
github.com/gogo/protobuf v1.3.3
Expand Down
8 changes: 4 additions & 4 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -111,8 +111,8 @@ github.com/cespare/xxhash v1.1.0/go.mod h1:XrSqR1VqqWfGrhpAt58auRo0WTKS1nRRg3ghf
github.com/cespare/xxhash/v2 v2.1.1/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=
github.com/cespare/xxhash/v2 v2.1.2 h1:YRXhKfTDauu4ajMg1TPgFO5jnlC2HCbmLXMcTG5cbYE=
github.com/cespare/xxhash/v2 v2.1.2/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=
github.com/cheqd/cheqd-node v0.7.3 h1:ELllyb+4HCCEsBnAEruQeJD292jP2CcfTlmXDTZI2vA=
github.com/cheqd/cheqd-node v0.7.3/go.mod h1:QssvmpF/YRCxb8mkxAAZOuGq5wbnkYdQlDu0LILQNV0=
github.com/cheqd/cheqd-node v0.6.9 h1:Yv1Ga0J6EVibgesYEgSHWB7+5aKZit2ZNjBvb+1g13c=
github.com/cheqd/cheqd-node v0.6.9/go.mod h1:qleVcKI+Dgj2e1Dug1wRriOrjytlrz2YKh5BrSP0quE=
github.com/cheqd/cosmos-sdk v0.45.9-cheqd-tag h1:K1eIYxqaQ5aFSVAnPNOeU12U0B3mStYN0vrawMoUkCY=
github.com/cheqd/cosmos-sdk v0.45.9-cheqd-tag/go.mod h1:Z5M4TX7PsHNHlF/1XanI2DIpORQ+Q/st7oaeufEjnvU=
github.com/chzyer/logex v1.1.10/go.mod h1:+Ywpsq7O8HXn0nuIou7OrIPyXbp3wmkHB+jjWRnGsAI=
Expand Down Expand Up @@ -140,8 +140,8 @@ github.com/cosmos/btcutil v1.0.4 h1:n7C2ngKXo7UC9gNyMNLbzqz7Asuf+7Qv4gnX/rOdQ44=
github.com/cosmos/btcutil v1.0.4/go.mod h1:Ffqc8Hn6TJUdDgHBwIZLtrLQC1KdJ9jGJl/TvgUaxbU=
github.com/cosmos/cosmos-sdk/ics23/go v0.8.0 h1:iKclrn3YEOwk4jQHT2ulgzuXyxmzmPczUalMwW4XH9k=
github.com/cosmos/cosmos-sdk/ics23/go v0.8.0/go.mod h1:2a4dBq88TUoqoWAU5eu0lGvpFP3wWDPgdHPargtyw30=
github.com/cosmos/gaia/v7 v7.1.0 h1:vS8AlJaI+tcODc+lsxGDZVxfVR329x06fEbXXjedidw=
github.com/cosmos/gaia/v7 v7.1.0/go.mod h1:NYNOpx24P02rjsSdZxirTlYI+H9R/DCfpDmRm4PND1w=
github.com/cosmos/gaia/v7 v7.0.2 h1:tn4wZk6jt0uqDL7z/V2Bs6NVd+gtsX+4BsQ94wPB8xw=
github.com/cosmos/gaia/v7 v7.0.2/go.mod h1:RDJGBlI+gGnWjYAy6LFM/Dtor5ZSo0Z/BYm3HA+/RWE=
github.com/cosmos/go-bip39 v0.0.0-20180819234021-555e2067c45d/go.mod h1:tSxLoYXyBmiFeKpvmq4dzayMdCjCnu8uqmCysIGBT2Y=
github.com/cosmos/go-bip39 v1.0.0 h1:pcomnQdrdH22njcAatO0yWojsUnCO3y2tNoV1cb6hHY=
github.com/cosmos/go-bip39 v1.0.0/go.mod h1:RNJv0H/pOIVgxw6KS7QeX2a0Uo0aKUlfhZ4xuwvCdJw=
Expand Down
4 changes: 3 additions & 1 deletion modules/daily_refetch/handle_periodic_operations.go
Original file line number Diff line number Diff line change
Expand Up @@ -4,11 +4,13 @@ import (
"fmt"
"time"

parsecmdtypes "github.com/forbole/juno/v3/cmd/parse/types"
"github.com/forbole/juno/v3/parser"
"github.com/forbole/juno/v3/types/config"

"github.com/go-co-op/gocron"
"github.com/rs/zerolog/log"

parsecmdtypes "github.com/forbole/juno/v3/cmd/parse/types"
)

func (m *Module) RegisterPeriodicOperations(scheduler *gocron.Scheduler) error {
Expand Down
10 changes: 8 additions & 2 deletions modules/gov/handle_msg.go
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,8 @@ package gov

import (
"fmt"
"time"

"strconv"
"time"

"github.com/forbole/bdjuno/v3/types"

Expand Down Expand Up @@ -57,6 +56,13 @@ func (m *Module) handleMsgSubmitProposal(tx *juno.Tx, index int, msg *govtypes.M
return fmt.Errorf("error while getting proposal: %s", err)
}

// Unpack the content
var content govtypes.Content
err = m.cdc.UnpackAny(proposal.Content, &content)
if err != nil {
return fmt.Errorf("error while unpacking proposal content: %s", err)
}

// Store the proposal
proposalObj := types.NewProposal(
proposal.ProposalId,
Expand Down
3 changes: 2 additions & 1 deletion modules/staking/utils_staking_pool.go
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ func (m *Module) GetStakingPool(height int64) (*types.Pool, error) {
}

unbondingTokens := sdk.NewInt(0)

for _, validator := range validatorsList {
// get list of all unbonding delegations for each validator
unbondingDelegations := m.getTotalUnbondingDelegationsFromValidator(height, validator.GetOperator())
Expand All @@ -44,7 +45,7 @@ func (m *Module) GetStakingPool(height int64) (*types.Pool, error) {
func (m *Module) getTotalUnbondingDelegationsFromValidator(height int64, valOperatorAddress string) []stakingtypes.UnbondingDelegation {
var unbondingDelegations []stakingtypes.UnbondingDelegation
var nextKey []byte
var stop = false
stop := false
for !stop {
res, err := m.source.GetUnbondingDelegationsFromValidator(height,
valOperatorAddress,
Expand Down
2 changes: 1 addition & 1 deletion types/config/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,8 @@ import (

// Config represents the BDJuno configuration
type Config struct {
ActionsConfig *actions.Config `yaml:"actions"`
JunoConfig junoconfig.Config `yaml:"-,inline"`
ActionsConfig *actions.Config `yaml:"actions"`
}

// NewConfig returns a new Config instance
Expand Down