Skip to content

Commit

Permalink
fix: Resolve proposal parsing issues [DEV-1966] (#68)
Browse files Browse the repository at this point in the history
* Pull upstream changes

* Remove unwanted upstream files

* Fix typo

* Fix proposal parsing issue

* Run gofumpt

* Fix linting issues
  • Loading branch information
filipdjokic committed Dec 7, 2022
1 parent 31dacfc commit 0d0520e
Show file tree
Hide file tree
Showing 10 changed files with 207 additions and 71 deletions.
129 changes: 129 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,129 @@
## Unreleased
### Changes

#### Parse Command
- ([\#492](https://github.com/forbole/bdjuno/pull/492)) Add parse command for periodic tasks: `x/bank` total supply, `x/distribution` community pool, `x/mint` inflation, `pricefeed` token price and price history, `x/staking` staking pool

#### Upgrade Module
- ([\#467](https://github.com/forbole/bdjuno/pull/467)) Store software upgrade plan and refresh data at upgrade height

#### Staking Module
- ([\#443](https://github.com/forbole/bdjuno/pull/443)) Remove tombstone status from staking module(already stored in slashing module)
- ([\#455](https://github.com/forbole/bdjuno/pull/455)) Added `unbonding_tokens` and `staked_not_bonded_tokens` values to staking pool table

#### Gov Module
- ([\#461](https://github.com/forbole/bdjuno/pull/461)) Parse `x/gov` genesis with `genesisDoc.InitialHeight` instead of the hard-coded height 1
- ([\#465](https://github.com/forbole/bdjuno/pull/465)) Get open proposal ids in deposit or voting period by block time instead of current time
- ([\#489](https://github.com/forbole/bdjuno/pull/489)) Remove block height foreign key from proposal_vote and proposal_deposit tables and add column timestamp
- ([\#499](https://github.com/forbole/bdjuno/pull/499)) Check if proposal has passed voting end time before marking it invalid

#### Daily refetch
- ([\#454](https://github.com/forbole/bdjuno/pull/454)) Added `daily refetch` module to refetch missing blocks every day

#### Hasura
- ([\#473](https://github.com/forbole/bdjuno/pull/473)) Improved Hasura permissions
- ([\#491](https://github.com/forbole/bdjuno/pull/491)) Add host address to Hasura actions

### Dependencies
- ([\#462](https://github.com/forbole/bdjuno/pull/462)) Updated Juno to `v3.4.0`

## Version v3.2.0
### Changes
#### Mint module
- ([\#432](https://github.com/forbole/bdjuno/pull/432)) Update inflation rate when mint param change proposal is passed

#### Gov module
- ([\#401](https://github.com/forbole/bdjuno/pull/401)) Update the proposal status to the latest in `bdjuno parse gov proposal [id]` command
- ([\#430](https://github.com/forbole/bdjuno/pull/430)) Update the proposals that have invalid status but can still be in voting or deposit periods

### Dependencies
- ([\#440](https://github.com/forbole/bdjuno/pull/440)) Updated Juno to `v3.3.0`

## Version v3.1.0
### Dependencies
- Updated Juno to `v3.2.0`

### Changes
#### Hasura
- ([\#395](https://github.com/forbole/bdjuno/pull/395)) Remove time label from Hasura Prometheus monitoring

#### Bank module
- ([\#410](https://github.com/forbole/bdjuno/pull/410)) Change total supply query from only 1 page to all pages

## Version v3.0.1
### Dependencies
- Updated Juno to `v3.1.1`

## Version v3.0.0
### Notes
This version introduces breaking changes to `transaction` and `message` PostgreSQL tables. It implements PostgreSQL table partitioning to fix slow data retrieval from database that stores large amount of transactions and messages. Read more details about [migrating to v3.0.0](https://docs.bigdipper.live/cosmos-based/parser/migrations/v2.0.0)

### New features
#### CLI
- ([\#356](https://github.com/forbole/bdjuno/pull/356)) Implemented `migrate` command to perform easy migration to higher BDJuno versions
- ([\#356](https://github.com/forbole/bdjuno/pull/356)) Updated `parse-genesis` command to parse genesis file without accessing the node

#### Database
- ([\#356](https://github.com/forbole/bdjuno/pull/356)) Added PostgreSQL table partition to `transaction` and `message` table
- ([\#356](https://github.com/forbole/bdjuno/pull/356)) Created new `messages_by_address` function

### Changes
#### Hasura
- ([\#377](https://github.com/forbole/bdjuno/pull/377)) Updated Hasura metadata
- ([\#381](https://github.com/forbole/bdjuno/pull/381)) Hasura actions are now a module

### Dependencies
- ([\#356](https://github.com/forbole/bdjuno/pull/356)) Updated Juno to `v3.0.0`

## Version v2.0.0
### Notes
This version introduces breaking changes to certain address-specific data that is no longer periodically parsed from the node and stored in the database. Instead, the data is now obtained directly from the node when needed using Hasura Actions. Read more details about [migrating to v2.0.0](https://docs.bigdipper.live/cosmos-based/parser/migrations/v2.0.0)

### New features
#### CLI
- ([\#257](https://github.com/forbole/bdjuno/pull/257)) Added `parse-genesis` command to parse the genesis file
- ([\#228](https://github.com/forbole/bdjuno/pull/228)) ([\#248](https://github.com/forbole/bdjuno/pull/248)) Added `fix` command:
- `auth`: fix vesting accounts details
- `blocks`: fix missing blocks and transactions from given start height
- `gov`: fix proposal with given proposal ID
- `staking`: fix validators info at the latest height

#### Hasura Actions
- ([\#329](https://github.com/forbole/bdjuno/pull/329)) Implemented Hasura Actions service to replace periodic queries. If you are using GraphQL queries on your application, you should updated the old queries to use the below new actions instead.
Here's a list of data acquired through Hasura Actions:
- Of a certain address/delegator:
- Account balance (`action_account_balance`)
- Delegation rewards (`action_delegation_reward`)
- Delegator withdraw address (`action_delegator_withdraw_address`)
- Delegations (`action_delegation`)
- Total delegations amount (`action_delegation_total`)
- Unbonding delegations (`action_unbonding_delegation`)
- Total unbonding delegations amount (`action_unbonding_delegation_total`)
- Redelegations (`action_redelegation`)
- Of a certain validator:
- Commission amount (`action_validator_commission_amount`)
- Delegations to this validator (`action_validator_delegations`)
- Redelegations from this validator (`action_validator_redelegations_from`)
- Unbonding delegations (`action_validator_unbonding_delegations`)
- ([\#352](https://github.com/forbole/bdjuno/pull/352)) Added prometheus monitoring to hasura actions

#### Local node support
- Added the support for `node.type = "local"` for parsing a static local node without the usage gRPC queries: [config reference](https://docs.bigdipper.live/cosmos-based/parser/config/config#node).

#### Modules
- ([\#232](https://github.com/forbole/bdjuno/pull/232)) Updated the `x/auth` module support to handle and store `vesting accounts` and `vesting periods` inside the database.
- ([\#276](https://github.com/forbole/bdjuno/pull/276)) Added the support for the `x/feegrant` module (v0.44.x)

### Changes

#### CLI
- ([\#351](https://github.com/forbole/bdjuno/pull/351)) Fixed version display for `bdjuno version` cmd

#### Database
- ([\#300](https://github.com/forbole/bdjuno/pull/300)) Changed `bonded_tokens` and `not_bonded_tokens` type inside `staking_pool` table to `TEXT` to avoid value overflow
- ([\#275](https://github.com/forbole/bdjuno/pull/275)) Added `tombstoned` column inside `validator_status` table
- ([\#232](https://github.com/forbole/bdjuno/pull/232)) Added `vesting_account` and `vesting_period` table
- ([\#276](https://github.com/forbole/bdjuno/pull/276)) Added `fee_grant_allowance` table (v0.44.x)

#### Modules
- ([\#353](https://github.com/forbole/bdjuno/pull/353)) Removed the support for the `history` module
1 change: 0 additions & 1 deletion database/gov.go
Original file line number Diff line number Diff line change
Expand Up @@ -403,7 +403,6 @@ WHERE proposal_validator_status_snapshot.height <= excluded.height`

// SaveSoftwareUpgradePlan allows to save the given software upgrade plan with its proposal id
func (db *Db) SaveSoftwareUpgradePlan(proposalID uint64, plan upgradetypes.Plan, height int64) error {

stmt := `
INSERT INTO software_upgrade_plan(proposal_id, plan_name, upgrade_height, info, height)
VALUES ($1, $2, $3, $4, $5)
Expand Down
113 changes: 56 additions & 57 deletions database/gov_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -84,10 +84,10 @@ func (suite *DbTestSuite) getProposalRow(id int) types.Proposal {
proposalType,
govtypes.NewTextProposal(title, description),
govtypes.StatusPassed.String(),
time.Date(2020, 1, 1, 00, 00, 00, 000, time.UTC),
time.Date(2020, 1, 1, 01, 00, 00, 000, time.UTC),
time.Date(2020, 1, 1, 02, 00, 00, 000, time.UTC),
time.Date(2020, 1, 1, 03, 00, 00, 000, time.UTC),
time.Date(2020, 1, 1, 0o0, 0o0, 0o0, 0o00, time.UTC),
time.Date(2020, 1, 1, 0o1, 0o0, 0o0, 0o00, time.UTC),
time.Date(2020, 1, 1, 0o2, 0o0, 0o0, 0o00, time.UTC),
time.Date(2020, 1, 1, 0o3, 0o0, 0o0, 0o00, time.UTC),
proposer.String(),
)

Expand Down Expand Up @@ -123,10 +123,10 @@ func (suite *DbTestSuite) TestBigDipperDb_SaveProposals() {
"proposalType",
content1,
govtypes.StatusDepositPeriod.String(),
time.Date(2020, 1, 1, 00, 00, 00, 000, time.UTC),
time.Date(2020, 1, 1, 01, 00, 00, 000, time.UTC),
time.Date(2020, 1, 1, 02, 00, 00, 000, time.UTC),
time.Date(2020, 1, 1, 03, 00, 00, 000, time.UTC),
time.Date(2020, 1, 1, 0o0, 0o0, 0o0, 0o00, time.UTC),
time.Date(2020, 1, 1, 0o1, 0o0, 0o0, 0o00, time.UTC),
time.Date(2020, 1, 1, 0o2, 0o0, 0o0, 0o00, time.UTC),
time.Date(2020, 1, 1, 0o3, 0o0, 0o0, 0o00, time.UTC),
proposer1.String(),
),
types.NewProposal(
Expand All @@ -135,10 +135,10 @@ func (suite *DbTestSuite) TestBigDipperDb_SaveProposals() {
"proposalType1",
content2,
govtypes.StatusPassed.String(),
time.Date(2020, 1, 2, 00, 00, 00, 000, time.UTC),
time.Date(2020, 1, 2, 01, 00, 00, 000, time.UTC),
time.Date(2020, 1, 2, 02, 00, 00, 000, time.UTC),
time.Date(2020, 1, 2, 03, 00, 00, 000, time.UTC),
time.Date(2020, 1, 2, 0o0, 0o0, 0o0, 0o00, time.UTC),
time.Date(2020, 1, 2, 0o1, 0o0, 0o0, 0o00, time.UTC),
time.Date(2020, 1, 2, 0o2, 0o0, 0o0, 0o00, time.UTC),
time.Date(2020, 1, 2, 0o3, 0o0, 0o0, 0o00, time.UTC),
proposer2.String(),
),
}
Expand All @@ -158,10 +158,10 @@ func (suite *DbTestSuite) TestBigDipperDb_SaveProposals() {
"title",
"description",
suite.encodeProposalContent(content1),
time.Date(2020, 1, 1, 00, 00, 00, 000, time.UTC),
time.Date(2020, 1, 1, 01, 00, 00, 000, time.UTC),
time.Date(2020, 1, 1, 02, 00, 00, 000, time.UTC),
time.Date(2020, 1, 1, 03, 00, 00, 000, time.UTC),
time.Date(2020, 1, 1, 0o0, 0o0, 0o0, 0o00, time.UTC),
time.Date(2020, 1, 1, 0o1, 0o0, 0o0, 0o00, time.UTC),
time.Date(2020, 1, 1, 0o2, 0o0, 0o0, 0o00, time.UTC),
time.Date(2020, 1, 1, 0o3, 0o0, 0o0, 0o00, time.UTC),
proposer1.String(),
govtypes.StatusDepositPeriod.String(),
),
Expand All @@ -172,10 +172,10 @@ func (suite *DbTestSuite) TestBigDipperDb_SaveProposals() {
"title1",
"description1",
suite.encodeProposalContent(content2),
time.Date(2020, 1, 2, 00, 00, 00, 000, time.UTC),
time.Date(2020, 1, 2, 01, 00, 00, 000, time.UTC),
time.Date(2020, 1, 2, 02, 00, 00, 000, time.UTC),
time.Date(2020, 1, 2, 03, 00, 00, 000, time.UTC),
time.Date(2020, 1, 2, 0o0, 0o0, 0o0, 0o00, time.UTC),
time.Date(2020, 1, 2, 0o1, 0o0, 0o0, 0o00, time.UTC),
time.Date(2020, 1, 2, 0o2, 0o0, 0o0, 0o00, time.UTC),
time.Date(2020, 1, 2, 0o3, 0o0, 0o0, 0o00, time.UTC),
proposer2.String(),
govtypes.StatusPassed.String(),
),
Expand All @@ -194,10 +194,10 @@ func (suite *DbTestSuite) TestBigDipperDb_GetProposal() {
"proposalType",
content,
govtypes.StatusDepositPeriod.String(),
time.Date(2020, 1, 1, 00, 00, 00, 000, time.UTC),
time.Date(2020, 1, 1, 01, 00, 00, 000, time.UTC),
time.Date(2020, 1, 1, 02, 00, 00, 000, time.UTC),
time.Date(2020, 1, 1, 03, 00, 00, 000, time.UTC),
time.Date(2020, 1, 1, 0o0, 0o0, 0o0, 0o00, time.UTC),
time.Date(2020, 1, 1, 0o1, 0o0, 0o0, 0o00, time.UTC),
time.Date(2020, 1, 1, 0o2, 0o0, 0o0, 0o00, time.UTC),
time.Date(2020, 1, 1, 0o3, 0o0, 0o0, 0o00, time.UTC),
proposer.String(),
)
input := []types.Proposal{proposal}
Expand All @@ -223,10 +223,10 @@ func (suite *DbTestSuite) TestBigDipperDb_GetOpenProposalsIds() {
"proposalType1",
content2,
types.ProposalStatusInvalid,
time.Date(2020, 1, 2, 00, 00, 00, 000, time.UTC),
time.Date(2020, 1, 2, 01, 00, 00, 000, time.UTC),
time.Date(2020, 1, 2, 02, 00, 00, 000, time.UTC),
time.Date(2020, 1, 2, 03, 00, 00, 000, time.UTC),
time.Date(2020, 1, 2, 0o0, 0o0, 0o0, 0o00, time.UTC),
time.Date(2020, 1, 2, 0o1, 0o0, 0o0, 0o00, time.UTC),
time.Date(2020, 1, 2, 0o2, 0o0, 0o0, 0o00, time.UTC),
time.Date(2020, 1, 2, 0o3, 0o0, 0o0, 0o00, time.UTC),
proposer2.String(),
)

Expand All @@ -237,10 +237,10 @@ func (suite *DbTestSuite) TestBigDipperDb_GetOpenProposalsIds() {
"proposalType",
content1,
govtypes.StatusVotingPeriod.String(),
time.Date(2020, 1, 1, 00, 00, 00, 000, time.UTC),
time.Date(2020, 1, 1, 01, 00, 00, 000, time.UTC),
time.Date(2020, 1, 1, 02, 00, 00, 000, time.UTC),
time.Date(2020, 1, 1, 03, 00, 00, 000, time.UTC),
time.Date(2020, 1, 1, 0o0, 0o0, 0o0, 0o00, time.UTC),
time.Date(2020, 1, 1, 0o1, 0o0, 0o0, 0o00, time.UTC),
time.Date(2020, 1, 1, 0o2, 0o0, 0o0, 0o00, time.UTC),
time.Date(2020, 1, 1, 0o3, 0o0, 0o0, 0o00, time.UTC),
proposer1.String(),
),
types.NewProposal(
Expand All @@ -249,10 +249,10 @@ func (suite *DbTestSuite) TestBigDipperDb_GetOpenProposalsIds() {
"proposalType",
content1,
govtypes.StatusDepositPeriod.String(),
time.Date(2020, 1, 1, 00, 00, 00, 000, time.UTC),
time.Date(2020, 1, 1, 01, 00, 00, 000, time.UTC),
time.Date(2020, 1, 1, 02, 00, 00, 000, time.UTC),
time.Date(2020, 1, 1, 03, 00, 00, 000, time.UTC),
time.Date(2020, 1, 1, 0o0, 0o0, 0o0, 0o00, time.UTC),
time.Date(2020, 1, 1, 0o1, 0o0, 0o0, 0o00, time.UTC),
time.Date(2020, 1, 1, 0o2, 0o0, 0o0, 0o00, time.UTC),
time.Date(2020, 1, 1, 0o3, 0o0, 0o0, 0o00, time.UTC),
proposer1.String(),
),
types.NewProposal(
Expand All @@ -261,10 +261,10 @@ func (suite *DbTestSuite) TestBigDipperDb_GetOpenProposalsIds() {
"proposalType1",
content2,
govtypes.StatusPassed.String(),
time.Date(2020, 1, 2, 00, 00, 00, 000, time.UTC),
time.Date(2020, 1, 2, 01, 00, 00, 000, time.UTC),
time.Date(2020, 1, 2, 02, 00, 00, 000, time.UTC),
time.Date(2020, 1, 2, 03, 00, 00, 000, time.UTC),
time.Date(2020, 1, 2, 0o0, 0o0, 0o0, 0o00, time.UTC),
time.Date(2020, 1, 2, 0o1, 0o0, 0o0, 0o00, time.UTC),
time.Date(2020, 1, 2, 0o2, 0o0, 0o0, 0o00, time.UTC),
time.Date(2020, 1, 2, 0o3, 0o0, 0o0, 0o00, time.UTC),
proposer2.String(),
),
types.NewProposal(
Expand All @@ -273,10 +273,10 @@ func (suite *DbTestSuite) TestBigDipperDb_GetOpenProposalsIds() {
"proposalType1",
content2,
govtypes.StatusRejected.String(),
time.Date(2020, 1, 2, 00, 00, 00, 000, time.UTC),
time.Date(2020, 1, 2, 01, 00, 00, 000, time.UTC),
time.Date(2020, 1, 2, 02, 00, 00, 000, time.UTC),
time.Date(2020, 1, 2, 03, 00, 00, 000, time.UTC),
time.Date(2020, 1, 2, 0o0, 0o0, 0o0, 0o00, time.UTC),
time.Date(2020, 1, 2, 0o1, 0o0, 0o0, 0o00, time.UTC),
time.Date(2020, 1, 2, 0o2, 0o0, 0o0, 0o00, time.UTC),
time.Date(2020, 1, 2, 0o3, 0o0, 0o0, 0o00, time.UTC),
proposer2.String(),
),
invalidProposal,
Expand All @@ -299,8 +299,8 @@ func (suite *DbTestSuite) TestBigDipperDb_UpdateProposal() {
update := types.NewProposalUpdate(
proposal.ProposalID,
govtypes.StatusPassed.String(),
time.Date(2020, 1, 1, 00, 00, 00, 000, time.UTC),
time.Date(2020, 1, 1, 01, 00, 00, 000, time.UTC),
time.Date(2020, 1, 1, 0o0, 0o0, 0o0, 0o00, time.UTC),
time.Date(2020, 1, 1, 0o1, 0o0, 0o0, 0o00, time.UTC),
)

err = suite.database.UpdateProposal(update)
Expand All @@ -315,8 +315,8 @@ func (suite *DbTestSuite) TestBigDipperDb_UpdateProposal() {
suite.encodeProposalContent(proposal.Content),
proposal.SubmitTime,
proposal.DepositEndTime,
time.Date(2020, 1, 1, 00, 00, 00, 000, time.UTC),
time.Date(2020, 1, 1, 01, 00, 00, 000, time.UTC),
time.Date(2020, 1, 1, 0o0, 0o0, 0o0, 0o00, time.UTC),
time.Date(2020, 1, 1, 0o1, 0o0, 0o0, 0o00, time.UTC),
proposer.String(),
govtypes.StatusPassed.String(),
)
Expand Down Expand Up @@ -348,9 +348,9 @@ func (suite *DbTestSuite) TestBigDipperDb_SaveDeposits() {
depositor3 := suite.getAccount("cosmos1gyds87lg3m52hex9yqta2mtwzw89pfukx3jl7g")
amount3 := sdk.NewCoins(sdk.NewCoin("desmos", sdk.NewInt(50000)))

timestamp1 := time.Date(2020, 1, 1, 15, 00, 00, 000, time.UTC)
timestamp2 := time.Date(2020, 1, 1, 16, 00, 00, 000, time.UTC)
timestamp3 := time.Date(2020, 1, 1, 17, 00, 00, 000, time.UTC)
timestamp1 := time.Date(2020, 1, 1, 15, 0o0, 0o0, 0o00, time.UTC)
timestamp2 := time.Date(2020, 1, 1, 16, 0o0, 0o0, 0o00, time.UTC)
timestamp3 := time.Date(2020, 1, 1, 17, 0o0, 0o0, 0o00, time.UTC)

deposit := []types.Deposit{
types.NewDeposit(proposal.ProposalID, depositor.String(), amount, timestamp1, 10),
Expand Down Expand Up @@ -414,7 +414,7 @@ func (suite *DbTestSuite) TestBigDipperDb_SaveVote() {
proposal := suite.getProposalRow(1)
voter := suite.getAccount("cosmos1z4hfrxvlgl4s8u4n5ngjcw8kdqrcv43599amxs")

timestamp := time.Date(2020, 1, 1, 15, 00, 00, 000, time.UTC)
timestamp := time.Date(2020, 1, 1, 15, 0o0, 0o0, 0o00, time.UTC)

vote := types.NewVote(1, voter.String(), govtypes.OptionYes, timestamp, 1)
err := suite.database.SaveVote(vote)
Expand Down Expand Up @@ -639,7 +639,7 @@ func (suite *DbTestSuite) TestBigDipperDb_SaveProposalValidatorsStatusesSnapshot
// ----------------------------------------------------------------------------------------------------------------
// Save snapshots

var snapshots = []types.ProposalValidatorStatusSnapshot{
snapshots := []types.ProposalValidatorStatusSnapshot{
types.NewProposalValidatorStatusSnapshot(
1,
validator1.GetConsAddr(),
Expand Down Expand Up @@ -838,7 +838,7 @@ func (suite *DbTestSuite) TestBigDipperDb_SaveSoftwareUpgradePlan() {

// ----------------------------------------------------------------------------------------------------------------
// Save software upgrade plan at height 10 with upgrade height at 100
var plan = upgradetypes.Plan{
plan := upgradetypes.Plan{
Name: "name",
Height: 100,
Info: "info",
Expand Down Expand Up @@ -917,7 +917,7 @@ func (suite *DbTestSuite) TestBigDipperDb_DeleteSoftwareUpgradePlan() {
_ = suite.getProposalRow(1)

// Save software upgrade plan at height 10 with upgrade height at 100
var plan = upgradetypes.Plan{
plan := upgradetypes.Plan{
Name: "name",
Height: 100,
Info: "info",
Expand All @@ -934,14 +934,13 @@ func (suite *DbTestSuite) TestBigDipperDb_DeleteSoftwareUpgradePlan() {
err = suite.database.Sqlx.Select(&rows, `SELECT * FROM software_upgrade_plan`)
suite.Require().NoError(err)
suite.Require().Len(rows, 0)

}

func (suite *DbTestSuite) TestBigDipperDb_CheckSoftwareUpgradePlan() {
_ = suite.getProposalRow(1)

// Save software upgrade plan at height 10 with upgrade height at 100
var plan = upgradetypes.Plan{
plan := upgradetypes.Plan{
Name: "name",
// the Height here is the upgrade height
Height: 100,
Expand Down
Loading

0 comments on commit 0d0520e

Please sign in to comment.