Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: Refactor TxSearch #14758

Merged
merged 41 commits into from
Feb 24, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
41 commits
Select commit Hold shift + click to select a range
ae3e190
updates
alexanderbez Jan 24, 2023
a47cfb7
updates
alexanderbez Jan 24, 2023
8df20c4
updates
alexanderbez Jan 24, 2023
82dc102
updates
alexanderbez Jan 25, 2023
f743d0b
updates
alexanderbez Jan 25, 2023
0df2abd
updates
alexanderbez Jan 25, 2023
20579ab
updates
alexanderbez Jan 25, 2023
3bdab1f
updates
alexanderbez Jan 25, 2023
756f722
updates
alexanderbez Jan 25, 2023
0afa78f
updates
alexanderbez Jan 25, 2023
97989b0
Merge branch 'main' into bez/14727-refactor-tx-search
alexanderbez Jan 25, 2023
bee3970
updates
alexanderbez Jan 25, 2023
a96463f
updates
alexanderbez Jan 26, 2023
a933e22
updates
alexanderbez Jan 26, 2023
62652ee
Merge branch 'main' into bez/14727-refactor-tx-search
alexanderbez Jan 26, 2023
ae2e900
Merge branch 'main' into bez/14727-refactor-tx-search
alexanderbez Jan 26, 2023
0f1025a
Merge branch 'main' into bez/14727-refactor-tx-search
tac0turtle Feb 20, 2023
76a2a03
fix merge
tac0turtle Feb 20, 2023
f938b7d
address comments
tac0turtle Feb 20, 2023
bcabf4b
Merge branch 'main' into bez/14727-refactor-tx-search
tac0turtle Feb 20, 2023
b88decb
revert internal flags
tac0turtle Feb 20, 2023
5ff6d30
delete extra lines
tac0turtle Feb 20, 2023
5c83612
Merge branch 'main' into bez/14727-refactor-tx-search
tac0turtle Feb 20, 2023
573fe32
Merge branch 'main' into bez/14727-refactor-tx-search
tac0turtle Feb 21, 2023
eb4de62
remove unneeded testcase
tac0turtle Feb 21, 2023
6c3c2d0
Merge branch 'main' into bez/14727-refactor-tx-search
tac0turtle Feb 21, 2023
7977997
Merge branch 'main' into bez/14727-refactor-tx-search
julienrbrt Feb 23, 2023
1b7d3b9
surface issue by remove cometbft replace
julienrbrt Feb 23, 2023
81c63c7
Merge branch 'main' into bez/14727-refactor-tx-search
alexanderbez Feb 23, 2023
2cab281
Merge branch 'main' into bez/14727-refactor-tx-search
alexanderbez Feb 23, 2023
3a6dc4d
updates
alexanderbez Feb 23, 2023
c08fdf0
Merge branch 'main' into bez/14727-refactor-tx-search
alexanderbez Feb 23, 2023
bf69105
updates
alexanderbez Feb 23, 2023
e86ea5e
Merge branch 'main' into bez/14727-refactor-tx-search
alexanderbez Feb 23, 2023
e4d15e6
Merge branch 'main' into bez/14727-refactor-tx-search
alexanderbez Feb 24, 2023
afbaf98
updates
julienrbrt Feb 24, 2023
ec5d9e1
Merge branch 'main' into bez/14727-refactor-tx-search
julienrbrt Feb 24, 2023
4187baf
updates
julienrbrt Feb 24, 2023
4180ed1
same pagination
julienrbrt Feb 24, 2023
7cdec41
updates
julienrbrt Feb 24, 2023
79a989a
Merge branch 'main' into bez/14727-refactor-tx-search
julienrbrt Feb 24, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,6 @@ Ref: https://keepachangelog.com/en/1.0.0/
* (query) [#14468](https://github.com/cosmos/cosmos-sdk/pull/14468) Implement pagination for collections.
* (x/bank) [#14045](https://github.com/cosmos/cosmos-sdk/pull/14045) Add CLI command `spendable-balances`, which also accepts the flag `--denom`.
* (x/slashing, x/staking) [#14363](https://github.com/cosmos/cosmos-sdk/pull/14363) Add the infraction a validator commited type as an argument to a `SlashWithInfractionReason` keeper method.
* (client) [#13867](https://github.com/cosmos/cosmos-sdk/pull/13867/) Wire AutoCLI commands with SimApp.
* (x/distribution) [#14322](https://github.com/cosmos/cosmos-sdk/pull/14322) Introduce a new gRPC message handler, `DepositValidatorRewardsPool`, that allows explicit funding of a validator's reward pool.
* (x/evidence) [#13740](https://github.com/cosmos/cosmos-sdk/pull/13740) Add new proto field `hash` of type `string` to `QueryEvidenceRequest` which helps to decode the hash properly while using query API.
* (core) [#13306](https://github.com/cosmos/cosmos-sdk/pull/13306) Add a `FormatCoins` function to in `core/coins` to format sdk Coins following the Value Renderers spec.
Expand Down Expand Up @@ -82,6 +81,7 @@ Ref: https://keepachangelog.com/en/1.0.0/

### Improvements

* (x/auth) [#14758](https://github.com/cosmos/cosmos-sdk/pull/14758) Allow transaction event queries to directly passed to Tendermint, which will allow for full query operator support, e.g. `>`.
* (server) [#15041](https://github.com/cosmos/cosmos-sdk/pull/15041) Remove unnecessary sleeps from gRPC and API server initiation. The servers will start and accept requests as soon as they're ready.
* (x/staking) [#14864](https://github.com/cosmos/cosmos-sdk/pull/14864) `create-validator` CLI command now takes a json file as an arg instead of having a bunch of required flags to it.
* (cli) [#14659](https://github.com/cosmos/cosmos-sdk/pull/14659) Added ability to query blocks by either height/hash `simd q block --type=height|hash <height|hash>`.
Expand Down Expand Up @@ -182,6 +182,10 @@ Ref: https://keepachangelog.com/en/1.0.0/

### API Breaking Changes

* (x/auth) [#14758](https://github.com/cosmos/cosmos-sdk/pull/14758) Refactor transaction searching:
* Refactor `QueryTxsByEvents` to accept a `query` of type `string` instead of `events` of type `[]string`
* Pass `prove=false` to Tendermint's `TxSearch` RPC method
* Refactor CLI methods to accept `--query` flag instead of `--events`
* (server) [#15041](https://github.com/cosmos/cosmos-sdk/pull/15041) Refactor how gRPC and API servers are started to remove unnecessary sleeps:
* Remove `ServerStartTime` constant.
* Rename `WaitForQuitSignals` to `ListenForQuitSignals`. Note, this function is no longer blocking. Thus the caller is expected to provide a `context.CancelFunc` which indicates that when a signal is caught, that any spawned processes can gracefully exit.
Expand Down
495 changes: 288 additions & 207 deletions api/cosmos/tx/v1beta1/service.pulsar.go

Large diffs are not rendered by default.

36 changes: 26 additions & 10 deletions proto/cosmos/tx/v1beta1/service.proto
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,8 @@ service Service {
// GetBlockWithTxs fetches a block with decoded txs.
//
// Since: cosmos-sdk 0.45.2
rpc GetBlockWithTxs(GetBlockWithTxsRequest) returns (GetBlockWithTxsResponse) {
rpc GetBlockWithTxs(GetBlockWithTxsRequest)
returns (GetBlockWithTxsResponse) {
option (google.api.http).get = "/cosmos/tx/v1beta1/txs/block/{height}";
}
// TxDecode decodes the transaction.
Expand Down Expand Up @@ -82,22 +83,35 @@ service Service {
// RPC method.
message GetTxsEventRequest {
// events is the list of transaction event type.
repeated string events = 1;
// Deprecated post v0.47.x: use query instead, which should contain a valid
// events query.
repeated string events = 1 [deprecated = true];

// pagination defines a pagination for the request.
// Deprecated post v0.46.x: use page and limit instead.
cosmos.base.query.v1beta1.PageRequest pagination = 2 [deprecated = true];

OrderBy order_by = 3;
// page is the page number to query, starts at 1. If not provided, will default to first page.

// page is the page number to query, starts at 1. If not provided, will
// default to first page.
uint64 page = 4;

// limit is the total number of results to be returned in the result page.
// If left empty it will default to a value to be set by each app.
uint64 limit = 5;

// query defines the transaction event query that is proxied to Tendermint's
// TxSearch RPC method. The query must be valid.
//
// Since Cosmos SDK 0.48
string query = 6;
tac0turtle marked this conversation as resolved.
Show resolved Hide resolved
}

// OrderBy defines the sorting order
enum OrderBy {
// ORDER_BY_UNSPECIFIED specifies an unknown sorting order. OrderBy defaults to ASC in this case.
// ORDER_BY_UNSPECIFIED specifies an unknown sorting order. OrderBy defaults
// to ASC in this case.
ORDER_BY_UNSPECIFIED = 0;
// ORDER_BY_ASC defines ascending order
ORDER_BY_ASC = 1;
Expand Down Expand Up @@ -127,18 +141,19 @@ message BroadcastTxRequest {
BroadcastMode mode = 2;
}

// BroadcastMode specifies the broadcast mode for the TxService.Broadcast RPC method.
// BroadcastMode specifies the broadcast mode for the TxService.Broadcast RPC
// method.
enum BroadcastMode {
// zero-value for mode ordering
BROADCAST_MODE_UNSPECIFIED = 0;
// DEPRECATED: use BROADCAST_MODE_SYNC instead,
// BROADCAST_MODE_BLOCK is not supported by the SDK from v0.47.x onwards.
BROADCAST_MODE_BLOCK = 1 [deprecated = true];
// BROADCAST_MODE_SYNC defines a tx broadcasting mode where the client waits for
// a CheckTx execution response only.
// BROADCAST_MODE_SYNC defines a tx broadcasting mode where the client waits
// for a CheckTx execution response only.
BROADCAST_MODE_SYNC = 2;
// BROADCAST_MODE_ASYNC defines a tx broadcasting mode where the client returns
// immediately.
// BROADCAST_MODE_ASYNC defines a tx broadcasting mode where the client
// returns immediately.
BROADCAST_MODE_ASYNC = 3;
}

Expand Down Expand Up @@ -196,7 +211,8 @@ message GetBlockWithTxsRequest {
cosmos.base.query.v1beta1.PageRequest pagination = 2;
}

// GetBlockWithTxsResponse is the response type for the Service.GetBlockWithTxs method.
// GetBlockWithTxsResponse is the response type for the Service.GetBlockWithTxs
// method.
//
// Since: cosmos-sdk 0.45.2
message GetBlockWithTxsResponse {
Expand Down
2 changes: 1 addition & 1 deletion server/grpc/server_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -159,7 +159,7 @@ func (s *IntegrationTestSuite) TestGRPCServer_GetTxsEvent() {
_, err := txServiceClient.GetTxsEvent(
context.Background(),
&txtypes.GetTxsEventRequest{
Events: []string{"message.action='send'"},
Query: "message.action='send'",
},
)
s.Require().NoError(err)
Expand Down
2 changes: 0 additions & 2 deletions tests/e2e/auth/cli_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,7 @@ import (
"testing"

"cosmossdk.io/simapp"

"github.com/cosmos/cosmos-sdk/testutil/network"

"github.com/stretchr/testify/suite"
)

Expand Down
12 changes: 8 additions & 4 deletions tests/e2e/auth/suite.go
Original file line number Diff line number Diff line change
Expand Up @@ -708,17 +708,21 @@ func (s *E2ETestSuite) TestCLIQueryTxsCmdByEvents() {
{
"fee event happy case",
[]string{
fmt.Sprintf("--events=tx.fee=%s",
sdk.NewCoins(sdk.NewCoin(s.cfg.BondDenom, sdk.NewInt(10))).String()),
fmt.Sprintf(
"--query=tx.fee='%s'",
sdk.NewCoins(sdk.NewCoin(s.cfg.BondDenom, sdk.NewInt(10))).String(),
),
fmt.Sprintf("--%s=json", flags.FlagOutput),
},
false,
},
{
"no matching fee event",
[]string{
fmt.Sprintf("--events=tx.fee=%s",
sdk.NewCoins(sdk.NewCoin(s.cfg.BondDenom, sdk.NewInt(0))).String()),
fmt.Sprintf(
"--query=tx.fee='%s'",
sdk.NewCoins(sdk.NewCoin(s.cfg.BondDenom, sdk.NewInt(0))).String(),
),
fmt.Sprintf("--%s=json", flags.FlagOutput),
},
true,
Expand Down
114 changes: 41 additions & 73 deletions tests/e2e/tx/service_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -7,12 +7,12 @@ import (
"strings"
"testing"

"github.com/stretchr/testify/require"
"github.com/stretchr/testify/suite"

errorsmod "cosmossdk.io/errors"

"cosmossdk.io/simapp"

"github.com/cosmos/cosmos-sdk/client"
"github.com/cosmos/cosmos-sdk/client/flags"
clienttx "github.com/cosmos/cosmos-sdk/client/tx"
Expand Down Expand Up @@ -136,7 +136,7 @@ func (s *E2ETestSuite) TestQueryBySig() {
b64Sig := base64.StdEncoding.EncodeToString(sig.Signature)
sigFormatted := fmt.Sprintf("%s.%s='%s'", sdk.EventTypeTx, sdk.AttributeKeySignature, b64Sig)
res, err := s.queryClient.GetTxsEvent(context.Background(), &tx.GetTxsEventRequest{
Events: []string{sigFormatted},
Query: sigFormatted,
OrderBy: 0,
Page: 0,
Limit: 10,
Expand All @@ -145,53 +145,6 @@ func (s *E2ETestSuite) TestQueryBySig() {
s.Require().Len(res.Txs, 1)
s.Require().Len(res.Txs[0].Signatures, 1)
s.Require().Equal(res.Txs[0].Signatures[0], sig.Signature)

// bad format should error
_, err = s.queryClient.GetTxsEvent(context.Background(), &tx.GetTxsEventRequest{Events: []string{"tx.foo.bar='baz'"}})
s.Require().ErrorContains(err, "invalid event;")
}

func TestEventRegex(t *testing.T) {
t.Parallel()

testCases := []struct {
name string
event string
match bool
}{
{
name: "valid: with quotes",
event: "tx.message='something'",
match: true,
},
{
name: "valid: with underscores",
event: "claim_reward.message_action='something'",
match: true,
},
{
name: "valid: no quotes",
event: "tx.message=something",
match: true,
},
{
name: "invalid: too many separators",
event: "tx.message.foo='bar'",
match: false,
},
{
name: "valid: symbols ok",
event: "tx.signature='foobar/baz123=='",
match: true,
},
}

for _, tc := range testCases {
t.Run(tc.name, func(t *testing.T) {
match := authtx.EventRegex.Match([]byte(tc.event))
require.Equal(t, tc.match, match)
})
}
}

func (s E2ETestSuite) TestSimulateTx_GRPC() {
Expand Down Expand Up @@ -294,48 +247,62 @@ func (s E2ETestSuite) TestGetTxEvents_GRPC() {
{
"nil request",
nil,
true, "request cannot be nil", 0,
true,
"request cannot be nil",
0,
},
{
"empty request",
&tx.GetTxsEventRequest{},
true, "must declare at least one event to search", 0,
true,
"query cannot be empty",
0,
},
{
"request with dummy event",
&tx.GetTxsEventRequest{Events: []string{"foobar"}},
true, "event foobar should be of the format: {eventType}.{eventAttribute}={value}", 0,
&tx.GetTxsEventRequest{Query: "foobar"},
true,
"failed to search for txs",
0,
},
{
"request with order-by",
&tx.GetTxsEventRequest{
Events: []string{bankMsgSendEventAction},
Query: bankMsgSendEventAction,
OrderBy: tx.OrderBy_ORDER_BY_ASC,
},
false, "", 3,
false,
"",
3,
},
{
"without pagination",
&tx.GetTxsEventRequest{
Events: []string{bankMsgSendEventAction},
Query: bankMsgSendEventAction,
},
false, "", 3,
false,
"",
3,
},
{
"with pagination",
&tx.GetTxsEventRequest{
Events: []string{bankMsgSendEventAction},
Page: 2,
Limit: 2,
Query: bankMsgSendEventAction,
Page: 2,
Limit: 2,
},
false, "", 1,
false,
"",
1,
},
{
"with multi events",
&tx.GetTxsEventRequest{
Events: []string{bankMsgSendEventAction, "message.module='bank'"},
Query: fmt.Sprintf("%s AND message.module='bank'", bankMsgSendEventAction),
},
false, "", 3,
false,
"",
3,
},
}
for _, tc := range testCases {
Expand All @@ -350,6 +317,7 @@ func (s E2ETestSuite) TestGetTxEvents_GRPC() {
s.Require().GreaterOrEqual(len(grpcRes.Txs), 1)
s.Require().Equal("foobar", grpcRes.Txs[0].Body.Memo)
s.Require().Equal(len(grpcRes.Txs), tc.expLen)

// Make sure fields are populated.
// ref: https://github.com/cosmos/cosmos-sdk/issues/8680
// ref: https://github.com/cosmos/cosmos-sdk/issues/8681
Expand All @@ -373,47 +341,47 @@ func (s E2ETestSuite) TestGetTxEvents_GRPCGateway() {
"empty params",
fmt.Sprintf("%s/cosmos/tx/v1beta1/txs", val.APIAddress),
true,
"must declare at least one event to search", 0,
"query cannot be empty", 0,
},
{
"without pagination",
fmt.Sprintf("%s/cosmos/tx/v1beta1/txs?events=%s", val.APIAddress, bankMsgSendEventAction),
fmt.Sprintf("%s/cosmos/tx/v1beta1/txs?query=%s", val.APIAddress, bankMsgSendEventAction),
false,
"", 3,
},
{
"with pagination",
fmt.Sprintf("%s/cosmos/tx/v1beta1/txs?events=%s&page=%d&limit=%d", val.APIAddress, bankMsgSendEventAction, 2, 2),
fmt.Sprintf("%s/cosmos/tx/v1beta1/txs?query=%s&page=%d&limit=%d", val.APIAddress, bankMsgSendEventAction, 2, 2),
false,
"", 1,
},
{
"valid request: order by asc",
fmt.Sprintf("%s/cosmos/tx/v1beta1/txs?events=%s&events=%s&order_by=ORDER_BY_ASC", val.APIAddress, bankMsgSendEventAction, "message.module='bank'"),
fmt.Sprintf("%s/cosmos/tx/v1beta1/txs?query=%s&query=%s&order_by=ORDER_BY_ASC", val.APIAddress, bankMsgSendEventAction, "message.module='bank'"),
false,
"", 3,
},
{
"valid request: order by desc",
fmt.Sprintf("%s/cosmos/tx/v1beta1/txs?events=%s&events=%s&order_by=ORDER_BY_DESC", val.APIAddress, bankMsgSendEventAction, "message.module='bank'"),
fmt.Sprintf("%s/cosmos/tx/v1beta1/txs?query=%s&query=%s&order_by=ORDER_BY_DESC", val.APIAddress, bankMsgSendEventAction, "message.module='bank'"),
false,
"", 3,
},
{
"invalid request: invalid order by",
fmt.Sprintf("%s/cosmos/tx/v1beta1/txs?events=%s&events=%s&order_by=invalid_order", val.APIAddress, bankMsgSendEventAction, "message.module='bank'"),
fmt.Sprintf("%s/cosmos/tx/v1beta1/txs?query=%s&query=%s&order_by=invalid_order", val.APIAddress, bankMsgSendEventAction, "message.module='bank'"),
true,
"is not a valid tx.OrderBy", 0,
},
{
"expect pass with multiple-events",
fmt.Sprintf("%s/cosmos/tx/v1beta1/txs?events=%s&events=%s", val.APIAddress, bankMsgSendEventAction, "message.module='bank'"),
fmt.Sprintf("%s/cosmos/tx/v1beta1/txs?query=%s&query=%s", val.APIAddress, bankMsgSendEventAction, "message.module='bank'"),
false,
"", 3,
},
{
"expect pass with escape event",
fmt.Sprintf("%s/cosmos/tx/v1beta1/txs?events=%s", val.APIAddress, "message.action%3D'/cosmos.bank.v1beta1.MsgSend'"),
fmt.Sprintf("%s/cosmos/tx/v1beta1/txs?query=%s", val.APIAddress, "message.action%3D'/cosmos.bank.v1beta1.MsgSend'"),
false,
"", 3,
},
Expand All @@ -427,11 +395,11 @@ func (s E2ETestSuite) TestGetTxEvents_GRPCGateway() {
} else {
var result tx.GetTxsEventResponse
err = val.ClientCtx.Codec.UnmarshalJSON(res, &result)
s.Require().NoError(err)
s.Require().NoError(err, "failed to unmarshal JSON: %s", res)
s.Require().GreaterOrEqual(len(result.Txs), 1)
s.Require().Equal("foobar", result.Txs[0].Body.Memo)
s.Require().NotZero(result.TxResponses[0].Height)
s.Require().Equal(len(result.Txs), tc.expLen)
s.Require().Equal(tc.expLen, len(result.Txs))
}
})
}
Expand Down
2 changes: 1 addition & 1 deletion tests/go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -198,7 +198,7 @@ replace (
cosmossdk.io/x/nft => ../x/nft
cosmossdk.io/x/tx => ../x/tx
cosmossdk.io/x/upgrade => ../x/upgrade
// This can be deleted after the CometBFT PR is merged
julienrbrt marked this conversation as resolved.
Show resolved Hide resolved
// TODO: This can be deleted after the CometBFT PR is merged.
github.com/cometbft/cometbft => github.com/cometbft/cometbft v0.0.0-20230203130311-387422ac220d
)

Expand Down
Loading