Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: add refetch blocks and transactions fix command #248

Merged
merged 57 commits into from
Dec 15, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
57 commits
Select commit Hold shift + click to select a range
b629fd2
Register NewTxsCmd
MonikaCat Oct 20, 2021
c21bf20
Add txs.go
MonikaCat Oct 20, 2021
b71403f
register blocks fix
MonikaCat Oct 22, 2021
3244813
Added block fix
MonikaCat Oct 22, 2021
f6d9e50
Update insert
MonikaCat Oct 22, 2021
6ee8d84
Updated db saving method
MonikaCat Oct 25, 2021
91b20a6
Updated block refetching
MonikaCat Oct 25, 2021
64f995e
Add: Get starting height from config file, refetch only missing block…
MonikaCat Oct 25, 2021
5859c9c
Cleanup
MonikaCat Oct 25, 2021
33bd098
Merge branch 'v2/cosmos/stargate' of https://github.com/forbole/bdjun…
MonikaCat Oct 27, 2021
f0a1c2b
Add: UpdateTxInDatabase
MonikaCat Nov 8, 2021
a23ef8c
Update refreshTxs method
MonikaCat Nov 9, 2021
585e906
Rm handleBlock
MonikaCat Nov 10, 2021
a806ce1
Added distrModule
MonikaCat Nov 10, 2021
0892c56
Updated tx db saving values
MonikaCat Nov 12, 2021
5f6724d
Updated db tx values
MonikaCat Nov 15, 2021
60e11bd
Add store msgs in message table
MonikaCat Nov 15, 2021
ff62300
Rm error check from event search
MonikaCat Nov 15, 2021
cd09c81
Update txDetails type
MonikaCat Nov 15, 2021
b28f7da
Merge branch 'v2/cosmos/stargate' of https://github.com/forbole/bdjun…
MonikaCat Nov 15, 2021
334d784
Merge branch 'm/fix-missing-blocks' of https://github.com/forbole/bdj…
MonikaCat Nov 15, 2021
3e6c4ef
Update register gov module
MonikaCat Nov 15, 2021
ee44ca3
Store accounts first in saveUpToDateBalances
MonikaCat Nov 17, 2021
ae3b556
Check if delegations exists otherwise return nil
MonikaCat Nov 17, 2021
09bb440
Update command
MonikaCat Nov 17, 2021
30bd7fd
Added RemoveDuplicateAccountBalance method
MonikaCat Nov 18, 2021
fbc2fcd
Updated ErrDelegationNotFound msg
MonikaCat Nov 18, 2021
1bada96
Update handleMsgCreateValidator
MonikaCat Nov 19, 2021
c6344cb
Move ErrNotFound to utils/errors.go
MonikaCat Nov 19, 2021
5fc45fc
Updated delegation error handling
MonikaCat Nov 19, 2021
371912e
Merge branch 'v2/cosmos/stargate' of https://github.com/forbole/bdjun…
MonikaCat Nov 19, 2021
93aa091
Add expected modules
MonikaCat Nov 19, 2021
2f2bcf7
Update registrar.go
MonikaCat Nov 19, 2021
753212f
Cleanup, add HandleMessages
MonikaCat Nov 19, 2021
a69de74
Update consensus module registrar
MonikaCat Nov 22, 2021
74bc35f
Removed unused source
MonikaCat Nov 22, 2021
8e3da73
Moved UpdateTxs
MonikaCat Nov 22, 2021
0457ebb
Removed unused db param
MonikaCat Nov 22, 2021
98547fb
Update refetching desc
MonikaCat Nov 22, 2021
479633b
Update cdm desc
MonikaCat Nov 22, 2021
0682fb6
Removed RemoveDuplicateAccountBalance
MonikaCat Nov 22, 2021
bee7e28
Cleanup
MonikaCat Nov 22, 2021
e95d007
Rm unused modules from expected_modules.go
MonikaCat Nov 22, 2021
ffc8b33
lint
MonikaCat Nov 22, 2021
2a7c18a
Cleanup
MonikaCat Nov 23, 2021
d46a684
Update block, tx and msg storing methods
MonikaCat Dec 1, 2021
543cdb3
Update: read startHeight from config file
MonikaCat Dec 1, 2021
d7192a6
Merge branch 'v2/cosmos/stargate' of https://github.com/forbole/bdjun…
MonikaCat Dec 1, 2021
08881ee
Update registered staking module
MonikaCat Dec 1, 2021
8788376
Rv RemoveDuplicateAccountBalance
MonikaCat Dec 6, 2021
6806a11
Rm error code check
MonikaCat Dec 6, 2021
6f01b71
Merge branch 'v2/cosmos/stargate' of https://github.com/forbole/bdjun…
MonikaCat Dec 6, 2021
8195ad4
Add worker to process missing blocks
MonikaCat Dec 13, 2021
7030101
Updated registrar
MonikaCat Dec 13, 2021
12db700
Merge branch 'v2/cosmos/stargate' of https://github.com/forbole/bdjun…
MonikaCat Dec 13, 2021
6228bc7
Revert err msg
MonikaCat Dec 15, 2021
378cf53
Merge branch 'v2/cosmos/stargate' into m/fix-missing-blocks
mergify[bot] Dec 15, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
45 changes: 45 additions & 0 deletions cmd/fix/blocks/blocks.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
package blocks

import (
"fmt"

"github.com/forbole/juno/v2/cmd/parse"

"github.com/forbole/juno/v2/parser"
"github.com/forbole/juno/v2/types/config"
"github.com/spf13/cobra"
)

// blocksCmd returns a Cobra command that allows to fix missing blocks in database
func blocksCmd(parseConfig *parse.Config) *cobra.Command {
MonikaCat marked this conversation as resolved.
Show resolved Hide resolved
return &cobra.Command{
Use: "refetch",
Short: "Fix missing blocks and transactions in database from the start height",
RunE: func(cmd *cobra.Command, args []string) error {
parseCtx, err := parse.GetParsingContext(parseConfig)
if err != nil {
return err
}

workerCtx := parser.NewContext(parseCtx.EncodingConfig.Marshaler, nil, parseCtx.Node, parseCtx.Database, parseCtx.Logger, parseCtx.Modules)
worker := parser.NewWorker(0, workerCtx)

// Get latest height
height, err := parseCtx.Node.LatestHeight()
if err != nil {
return fmt.Errorf("error while getting chain latest block height: %s", err)
}

k := config.Cfg.Parser.StartHeight
fmt.Printf("Refetching missing blocks and transactions from height %d ... \n", k)
for ; k <= height; k++ {
err := worker.Process(k)
if err != nil {
return fmt.Errorf("error while re-fetching block %d: %s", k, err)
}
}

return nil
},
}
}
20 changes: 20 additions & 0 deletions cmd/fix/blocks/cmd.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
package blocks

import (
"github.com/forbole/juno/v2/cmd/parse"
"github.com/spf13/cobra"
)

// NewBlocksCmd returns the Cobra command that allows to fix all the things related to blocks
func NewBlocksCmd(parseConfig *parse.Config) *cobra.Command {
cmd := &cobra.Command{
Use: "blocks",
Short: "Fix things related to blocks and transactions",
}

cmd.AddCommand(
blocksCmd(parseConfig),
)

return cmd
}
2 changes: 2 additions & 0 deletions cmd/fix/fix.go
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ import (
"github.com/spf13/cobra"

fixauth "github.com/forbole/bdjuno/v2/cmd/fix/auth"
fixblocks "github.com/forbole/bdjuno/v2/cmd/fix/blocks"
fixgov "github.com/forbole/bdjuno/v2/cmd/fix/gov"
fixstaking "github.com/forbole/bdjuno/v2/cmd/fix/staking"
)
Expand All @@ -19,6 +20,7 @@ func NewFixCmd(parseCfg *parse.Config) *cobra.Command {

cmd.AddCommand(
fixauth.NewAuthCmd(parseCfg),
fixblocks.NewBlocksCmd(parseCfg),
fixgov.NewGovCmd(parseCfg),
fixstaking.NewStakingCmd(parseCfg),
)
Expand Down
2 changes: 1 addition & 1 deletion go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ go 1.16

require (
github.com/cosmos/cosmos-sdk v0.42.9
github.com/forbole/juno/v2 v2.0.0-20211206070432-9314a5057955
github.com/forbole/juno/v2 v2.0.0-20211213105313-d167239d2dad
github.com/go-co-op/gocron v1.11.0
github.com/gogo/protobuf v1.3.3
github.com/jmoiron/sqlx v1.2.1-0.20200324155115-ee514944af4b
Expand Down
4 changes: 2 additions & 2 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -207,8 +207,8 @@ github.com/fatih/color v1.9.0/go.mod h1:eQcE1qtQxscV5RaZvpXrrb8Drkc3/DdQ+uUYCNjL
github.com/fatih/color v1.13.0/go.mod h1:kLAiJbzzSOZDVNGyDpeOxJ47H46qBXwg5ILebYFFOfk=
github.com/felixge/httpsnoop v1.0.1 h1:lvB5Jl89CsZtGIWuTcDM1E/vkVs49/Ml7JJe07l8SPQ=
github.com/felixge/httpsnoop v1.0.1/go.mod h1:m8KPJKqk1gH5J9DgRY2ASl2lWCfGKXixSwevea8zH2U=
github.com/forbole/juno/v2 v2.0.0-20211206070432-9314a5057955 h1:a8XH7gpsjUgmXjPkeU2C1b3yRppPxc8nVKWqK4vXAGI=
github.com/forbole/juno/v2 v2.0.0-20211206070432-9314a5057955/go.mod h1:nOZEHr5+sZDXEDpV2E+NHkmH0Zvf4prkt5+UVKwScQ8=
github.com/forbole/juno/v2 v2.0.0-20211213105313-d167239d2dad h1:0rhtVBf124+gEd7lZtohkR5FshoJgnldaVswpeHftDI=
github.com/forbole/juno/v2 v2.0.0-20211213105313-d167239d2dad/go.mod h1:nOZEHr5+sZDXEDpV2E+NHkmH0Zvf4prkt5+UVKwScQ8=
github.com/forbole/tendermint v0.34.13-0.20210820072129-a2a4af55563d h1:pUqGUgTUU24ibHeloQeg1F2pFbgQllddsuZ+x+CcUzw=
github.com/forbole/tendermint v0.34.13-0.20210820072129-a2a4af55563d/go.mod h1:aeHL7alPh4uTBIJQ8mgFEE8VwJLXI1VD3rVOmH2Mcy0=
github.com/fortytw2/leaktest v1.3.0 h1:u8491cBMTQ8ft8aeV+adlcytMZylmA5nnwwkRZjI8vw=
Expand Down
6 changes: 4 additions & 2 deletions modules/registrar.go
Original file line number Diff line number Diff line change
Expand Up @@ -107,11 +107,13 @@ func (r *Registrar) BuildModules(ctx registrar.Context) jmodules.Modules {

authModule := auth.NewModule(r.parser, cdc, db)
bankModule := bank.NewModule(r.parser, sources.BankSource, cdc, db)
consensusModule := consensus.NewModule(db)
distrModule := distribution.NewModule(ctx.JunoConfig, sources.DistrSource, bankModule, cdc, db)
historyModule := history.NewModule(ctx.JunoConfig.Chain, r.parser, cdc, db)
mintModule := mint.NewModule(sources.MintSource, cdc, db)
slashingModule := slashing.NewModule(sources.SlashingSource, nil, cdc, db)
stakingModule := staking.NewModule(sources.StakingSource, bankModule, distrModule, historyModule, slashingModule, cdc, db)
govModule := gov.NewModule(sources.GovSource, authModule, bankModule, distrModule, mintModule, slashingModule, stakingModule, cdc, db)

return []jmodules.Module{
messages.NewModule(r.parser, cdc, ctx.Database),
Expand All @@ -120,9 +122,9 @@ func (r *Registrar) BuildModules(ctx registrar.Context) jmodules.Modules {

authModule,
bankModule,
consensus.NewModule(db),
consensusModule,
distrModule,
gov.NewModule(sources.GovSource, authModule, bankModule, distrModule, mintModule, slashingModule, stakingModule, cdc, db),
govModule,
historyModule,
mint.NewModule(sources.MintSource, cdc, db),
modules.NewModule(ctx.JunoConfig.Chain, db),
Expand Down
3 changes: 1 addition & 2 deletions modules/staking/utils_delegations.go
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,8 @@ package staking
import (
"fmt"

"google.golang.org/grpc/codes"

stakingtypes "github.com/cosmos/cosmos-sdk/x/staking/types"
"google.golang.org/grpc/codes"

"github.com/forbole/bdjuno/v2/types"
)
Expand Down