Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Neutron launch fixes and optimizations #1185

Merged
merged 8 commits into from
May 24, 2023
Merged

Neutron launch fixes and optimizations #1185

merged 8 commits into from
May 24, 2023

Conversation

agouin
Copy link
Member

@agouin agouin commented May 11, 2023

Ordered channels:

  • Allow multiple messages per tx on ordered channels.
  • Sequences must be consecutive from lowest sequence.
  • for next MsgRecvPacket, must start with channel NextSeqRecv

Queries

  • Concurrent QuerySendPacket calls for flush.
  • Fix write_acknowledgement queries which do not have port_id in events.

Max Messages parameter

  • max msgs flag should be uint64, no need to parse string.
  • PathProcessor was not consuming the max messages param.

Logging

  • zap.Object logging does not work on zaplogfmt.NewEncoder(config), so change auto logging to console encoder.

relayer/strategies.go Fixed Show fixed Hide fixed
@agouin agouin changed the title pipe max msgs through path processor Neutron launch fixes and optimizations May 12, 2023
@agouin agouin marked this pull request as ready for review May 12, 2023 00:53
relayer/strategies.go Outdated Show resolved Hide resolved
relayer/processor/path_processor.go Outdated Show resolved Hide resolved
relayer/processor/path_processor_internal.go Outdated Show resolved Hide resolved
relayer/processor/path_processor_internal.go Show resolved Hide resolved
relayer/processor/path_processor_internal.go Outdated Show resolved Hide resolved
Copy link
Member

@jtieri jtieri left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great improvements here!! 🚢

@agouin agouin merged commit 23d1e5c into main May 24, 2023
8 checks passed
@agouin agouin deleted the andrew/max_msgs branch May 24, 2023 21:48
@7AC 7AC mentioned this pull request May 30, 2023
viveksharmapoudel added a commit to icon-project/ibc-relay that referenced this pull request Jun 30, 2023
* Flush query should include begin and end block events (cosmos#1125)

* Include begin and end block events

* disable flushing when termination condition is set

* Still flush for FlushLifecycle

* Add sort for flush logging to avoid confusion

* pre_init messages (cosmos#1131)

* Wire initial messages into path processor caches so that retry logic will occur

* Fix counterparty keys

* Remove debug log

* fix default coin type (cosmos#1134)

* fix slip44 default

* Add test case

* build: bump to Go 1.20 + bump deps (cosmos#1132)

* build: bump to Go 1.20 + bump deps

This bumps the Go version to 1.20 and also bumps the SDK version to 0.47.0 and ibc-go to v7.0.0

* chore: update GH workflows to use Go 1.20 + update interchaintest deps

* chore: update missing deps in go.sum + use 1.20 in dockerfiles

* chore: bump to `setup-go/v4` and remove caching step

* chore: bump to `checkout/v3` and remove caching step

* chore: bump 1.20.2

* Fix flushing acks (cosmos#1139)

* Fix ordered channel closure (cosmos#1142)

* Fix ordered channel closure

* Increase timeout for scenarios test

* Fix tracking of init messages when IDs aren't the same

* bump interchaintest to include explicit port bindings

* Fix flush termination condition (cosmos#1141)

* bump to main sha (cosmos#1143)

* Pre-filter flush channels (cosmos#1146)

* Add channel close correlation (cosmos#1145)

* Add channel close correlation

* Switch to pre-close key

* make tx channel-close cli command work, add test coverage

* more sweet code removals

* update comment

* Fix flush on ordered channels (cosmos#1150)

* Fix flush on ordered channels

* Queue all packets at nextseqrecv or above

* Now that we have periodic flushing, skip blocks if they can't be queried (cosmos#1154)

* Lock config file for all write operations (cosmos#1156)

* Lock config file for all write operations

* Fix linter errs

* tidy

* more tidy

* Penumbra support v2 (cosmos#1144)

* Penumbra buf go gen

* Use go prefix override

* wip: penumbra relayer provider

remove copied-over cosmos provider tests, rename processor cosmos -> penumbra

rename ccp -> pcp

reformat into new relayer dir structure

update penumbra types to point to buf.build

building again

* fix penumbra

* fix: implement MsgSubmitMisbehaviour

* fix: remove unnecessary proto file + fix msg type cast

* chore: add removal of penumbra client protos in protocgen

* working penumbra relayer functionality

wip: unbase64?

wip: multiple messages per penumbra tx

wip: stub SendMessages impl

wip: attempt to split out common method

wip: use random anchor and work around path renaming

wip: improve logging

wip: changes during pairing

https://www.youtube.com/watch?v=RYonSOkZ5ZE

clean up logs

skip height bug workaround

drop debug panic

* update penumbra chain processor connection and channel message processing

* cleanup logging statements for review

Responding to review comments, honoring the style guide for logging, and
removing some unused reference code that was commented out while
debugging.

---------

Co-authored-by: Andrew Gouin <andrew@gouin.io>
Co-authored-by: Ava Howell <ava@penumbra.zone>
Co-authored-by: jtieri <justin@thetieris.com>
Co-authored-by: Ava Howell <ava@avahowell.me>
Co-authored-by: Conor Schaefer <conor@penumbralabs.xyz>
Co-authored-by: Justin Tieri <37750742+jtieri@users.noreply.github.com>

* --time-threshold example use cases (cosmos#1155)

* Make ICA waits more explicit (cosmos#1157)

* Make ICA waits more explicit

* poll for timeout

* poll for channel close confirm

* Comment out sqlite db file for scenarios tests

* Bump github.com/docker/docker in /interchaintest (cosmos#1160)

Bumps [github.com/docker/docker](https://github.com/docker/docker) from 20.10.19+incompatible to 20.10.24+incompatible.
- [Release notes](https://github.com/docker/docker/releases)
- [Commits](moby/moby@v20.10.19...v20.10.24)

---
updated-dependencies:
- dependency-name: github.com/docker/docker
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* bump version in docs (cosmos#1158)

* Bump github.com/opencontainers/runc in /interchaintest (cosmos#1153)

Bumps [github.com/opencontainers/runc](https://github.com/opencontainers/runc) from 1.1.3 to 1.1.5.
- [Release notes](https://github.com/opencontainers/runc/releases)
- [Changelog](https://github.com/opencontainers/runc/blob/v1.1.5/CHANGELOG.md)
- [Commits](opencontainers/runc@v1.1.3...v1.1.5)

---
updated-dependencies:
- dependency-name: github.com/opencontainers/runc
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Andrew Gouin <andrew@gouin.io>

* Fix QueryConnectionsUsingClient for cosmos (cosmos#1162)

* Make min query loop duration configurable (cosmos#1164)

* penumbra provider: update generated protos (cosmos#1168)

* Add sr25519 support (cosmos#1120)

* Migrate to cometbft

* Additional replaces

* Register comet proto types

* Add sr25519 support

* bump ictest

* Add keys test

* Update supported algos comment

* penumbra: update protos (cosmos#1170)

Updating the generated protos for Penumbra support. After lightly
editing the `scripts/protocgen.sh` script, I ran `make proto-gen`
and then committed the changes `relayer/chains/penumbra/`. Other
automatically updated protos I intentionally excluded from this PR, to
avoid side-effects.

Co-authored-by: Conor Schaefer <conor@penumbralabs.xyz>

* chore: add path name to logs in message processor (cosmos#1171)

* Fix multiple conn open init (cosmos#1173)

* allow register with extra_codecs (cosmos#1175)

* Harry/fee middleware (cosmos#1174)

* Register Counterparty relayer cmd and fee middleware test

* debugging the command

* debugging and finalizing the fee_middleware_test.

* debugging and finalizing the fee_middleware_test.

* merged with updated repo

* clear out some commanded code

* nits and suggestions post review

* more nits

* added one val no fullnode as chainspec

---------

Co-authored-by: Harry <harrycosmos@Harrys-MacBook-Pro.local>

* fix: nil receiver initiate for path (cosmos#1177)

* fix nil receiver initiate for path

ensure path get written to config

* add change doc

* feat: add max-gas-amount parameter in chain configs (cosmos#1178)

* add max fee

* add test

* add change doc

* Update cregistry/chain_info.go

* dep: bump sdk from v0.47.0 to v0.47.2 (cosmos#1180)

* bump sdk from v0.47.0 to v0.47.2

fix btcutil dep

* fix dep of hdkeychain

* Harry/rly tx channel (cosmos#1183)

* made a new method "logChannelOpenMessage" to log the newly opened channel into info level

* added fields

* some changes

---------

Co-authored-by: Harry <harrycosmos@Harrys-MacBook-Pro.local>

* Harry/rly tx transfer (cosmos#1184)

* made a new method "logChannelOpenMessage" to log the newly opened channel into info level

* added fields

* some changes

* recreated issue 1151, added logs

---------

Co-authored-by: Harry <harrycosmos@Harrys-MacBook-Pro.local>
Co-authored-by: Andrew Gouin <andrew@gouin.io>

* Better Error Messaging when failing to query the Block Height (cosmos#1189)

* better block data errors

* remove redundant field

* penumbra: update protos (cosmos#1181)

Matches the latest protos shipped with the Penumbra Testnet 52.

Co-authored-by: Conor Schaefer <conor@penumbralabs.xyz>
Co-authored-by: Justin Tieri <37750742+jtieri@users.noreply.github.com>

* Neutron launch fixes and optimizations (cosmos#1185)

* pipe max msgs through path processor

* only apply max msgs to packet msgs

* multiple msgs simultaneously on ordered chans

* flush should be more frequent if it fails or does not complete

* fix legacy

* handle feedback

* Problem: fixes in ibc-go v7.0.1 are not included (cosmos#1205)

* Problem: fixes in ibc-go v7.0.1 are not included

* add change doc

* Harry/rly address (cosmos#1204)

* added addresCmd to root and keys.go

* nicks

* nick

* made a common method "showAddressByChainAndKey" to be used by both addressCmd and keysShowCmd

---------

Co-authored-by: Harry <harrycosmos@Harrys-MacBook-Pro.local>
Co-authored-by: Andrew Gouin <andrew@gouin.io>

* deps: update to ibc-go v7.1.0-rc0 (cosmos#1207)

* Export wallet address for Prometheus metrics (cosmos#1206)

* export relayer address for pro

* address in updateFeesSpent

* make error messages consistent

* log error rather than return

* handle 0 balance

* chore: fix issue

* fix: connection key include

* chore: comment out flush

* fix: remove SET method from providerConfig

* chore: replace static naming with constant

* chore: packet timeout after relayer update

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Andrew Gouin <andrew@gouin.io>
Co-authored-by: Justin Tieri <37750742+jtieri@users.noreply.github.com>
Co-authored-by: Henry de Valence <hdevalence@hdevalence.ca>
Co-authored-by: Ava Howell <ava@penumbra.zone>
Co-authored-by: jtieri <justin@thetieris.com>
Co-authored-by: Ava Howell <ava@avahowell.me>
Co-authored-by: Conor Schaefer <conor@penumbralabs.xyz>
Co-authored-by: Dan Kanefsky <56059752+boojamya@users.noreply.github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Conor Schaefer <conorsch@users.noreply.github.com>
Co-authored-by: mmsqe <tqd0800210105@gmail.com>
Co-authored-by: Cosmos-Harry <110472914+Cosmos-Harry@users.noreply.github.com>
Co-authored-by: Harry <harrycosmos@Harrys-MacBook-Pro.local>
Co-authored-by: mmsqe <mavis@crypto.com>
Co-authored-by: Keefer Taylor | Tessellated <keefer@tessellated.io>
Co-authored-by: izyak <test@test.com>
Co-authored-by: viveksharmapoudel <viveksharmapoudel@gmail.com>
@agouin agouin restored the andrew/max_msgs branch July 21, 2023 21:24
izyak added a commit to izyak/ibc-relay-test that referenced this pull request Sep 7, 2023
* Flush query should include begin and end block events (cosmos#1125)

* Include begin and end block events

* disable flushing when termination condition is set

* Still flush for FlushLifecycle

* Add sort for flush logging to avoid confusion

* pre_init messages (cosmos#1131)

* Wire initial messages into path processor caches so that retry logic will occur

* Fix counterparty keys

* Remove debug log

* fix default coin type (cosmos#1134)

* fix slip44 default

* Add test case

* build: bump to Go 1.20 + bump deps (cosmos#1132)

* build: bump to Go 1.20 + bump deps

This bumps the Go version to 1.20 and also bumps the SDK version to 0.47.0 and ibc-go to v7.0.0

* chore: update GH workflows to use Go 1.20 + update interchaintest deps

* chore: update missing deps in go.sum + use 1.20 in dockerfiles

* chore: bump to `setup-go/v4` and remove caching step

* chore: bump to `checkout/v3` and remove caching step

* chore: bump 1.20.2

* Fix flushing acks (cosmos#1139)

* Fix ordered channel closure (cosmos#1142)

* Fix ordered channel closure

* Increase timeout for scenarios test

* Fix tracking of init messages when IDs aren't the same

* bump interchaintest to include explicit port bindings

* Fix flush termination condition (cosmos#1141)

* bump to main sha (cosmos#1143)

* Pre-filter flush channels (cosmos#1146)

* Add channel close correlation (cosmos#1145)

* Add channel close correlation

* Switch to pre-close key

* make tx channel-close cli command work, add test coverage

* more sweet code removals

* update comment

* Fix flush on ordered channels (cosmos#1150)

* Fix flush on ordered channels

* Queue all packets at nextseqrecv or above

* Now that we have periodic flushing, skip blocks if they can't be queried (cosmos#1154)

* Lock config file for all write operations (cosmos#1156)

* Lock config file for all write operations

* Fix linter errs

* tidy

* more tidy

* Penumbra support v2 (cosmos#1144)

* Penumbra buf go gen

* Use go prefix override

* wip: penumbra relayer provider

remove copied-over cosmos provider tests, rename processor cosmos -> penumbra

rename ccp -> pcp

reformat into new relayer dir structure

update penumbra types to point to buf.build

building again

* fix penumbra

* fix: implement MsgSubmitMisbehaviour

* fix: remove unnecessary proto file + fix msg type cast

* chore: add removal of penumbra client protos in protocgen

* working penumbra relayer functionality

wip: unbase64?

wip: multiple messages per penumbra tx

wip: stub SendMessages impl

wip: attempt to split out common method

wip: use random anchor and work around path renaming

wip: improve logging

wip: changes during pairing

https://www.youtube.com/watch?v=RYonSOkZ5ZE

clean up logs

skip height bug workaround

drop debug panic

* update penumbra chain processor connection and channel message processing

* cleanup logging statements for review

Responding to review comments, honoring the style guide for logging, and
removing some unused reference code that was commented out while
debugging.

---------

Co-authored-by: Andrew Gouin <andrew@gouin.io>
Co-authored-by: Ava Howell <ava@penumbra.zone>
Co-authored-by: jtieri <justin@thetieris.com>
Co-authored-by: Ava Howell <ava@avahowell.me>
Co-authored-by: Conor Schaefer <conor@penumbralabs.xyz>
Co-authored-by: Justin Tieri <37750742+jtieri@users.noreply.github.com>

* --time-threshold example use cases (cosmos#1155)

* Make ICA waits more explicit (cosmos#1157)

* Make ICA waits more explicit

* poll for timeout

* poll for channel close confirm

* Comment out sqlite db file for scenarios tests

* Bump github.com/docker/docker in /interchaintest (cosmos#1160)

Bumps [github.com/docker/docker](https://github.com/docker/docker) from 20.10.19+incompatible to 20.10.24+incompatible.
- [Release notes](https://github.com/docker/docker/releases)
- [Commits](moby/moby@v20.10.19...v20.10.24)

---
updated-dependencies:
- dependency-name: github.com/docker/docker
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* bump version in docs (cosmos#1158)

* Bump github.com/opencontainers/runc in /interchaintest (cosmos#1153)

Bumps [github.com/opencontainers/runc](https://github.com/opencontainers/runc) from 1.1.3 to 1.1.5.
- [Release notes](https://github.com/opencontainers/runc/releases)
- [Changelog](https://github.com/opencontainers/runc/blob/v1.1.5/CHANGELOG.md)
- [Commits](opencontainers/runc@v1.1.3...v1.1.5)

---
updated-dependencies:
- dependency-name: github.com/opencontainers/runc
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Andrew Gouin <andrew@gouin.io>

* Fix QueryConnectionsUsingClient for cosmos (cosmos#1162)

* Make min query loop duration configurable (cosmos#1164)

* penumbra provider: update generated protos (cosmos#1168)

* Add sr25519 support (cosmos#1120)

* Migrate to cometbft

* Additional replaces

* Register comet proto types

* Add sr25519 support

* bump ictest

* Add keys test

* Update supported algos comment

* penumbra: update protos (cosmos#1170)

Updating the generated protos for Penumbra support. After lightly
editing the `scripts/protocgen.sh` script, I ran `make proto-gen`
and then committed the changes `relayer/chains/penumbra/`. Other
automatically updated protos I intentionally excluded from this PR, to
avoid side-effects.

Co-authored-by: Conor Schaefer <conor@penumbralabs.xyz>

* chore: add path name to logs in message processor (cosmos#1171)

* Fix multiple conn open init (cosmos#1173)

* allow register with extra_codecs (cosmos#1175)

* Harry/fee middleware (cosmos#1174)

* Register Counterparty relayer cmd and fee middleware test

* debugging the command

* debugging and finalizing the fee_middleware_test.

* debugging and finalizing the fee_middleware_test.

* merged with updated repo

* clear out some commanded code

* nits and suggestions post review

* more nits

* added one val no fullnode as chainspec

---------

Co-authored-by: Harry <harrycosmos@Harrys-MacBook-Pro.local>

* fix: nil receiver initiate for path (cosmos#1177)

* fix nil receiver initiate for path

ensure path get written to config

* add change doc

* feat: add max-gas-amount parameter in chain configs (cosmos#1178)

* add max fee

* add test

* add change doc

* Update cregistry/chain_info.go

* dep: bump sdk from v0.47.0 to v0.47.2 (cosmos#1180)

* bump sdk from v0.47.0 to v0.47.2

fix btcutil dep

* fix dep of hdkeychain

* Harry/rly tx channel (cosmos#1183)

* made a new method "logChannelOpenMessage" to log the newly opened channel into info level

* added fields

* some changes

---------

Co-authored-by: Harry <harrycosmos@Harrys-MacBook-Pro.local>

* Harry/rly tx transfer (cosmos#1184)

* made a new method "logChannelOpenMessage" to log the newly opened channel into info level

* added fields

* some changes

* recreated issue 1151, added logs

---------

Co-authored-by: Harry <harrycosmos@Harrys-MacBook-Pro.local>
Co-authored-by: Andrew Gouin <andrew@gouin.io>

* Better Error Messaging when failing to query the Block Height (cosmos#1189)

* better block data errors

* remove redundant field

* penumbra: update protos (cosmos#1181)

Matches the latest protos shipped with the Penumbra Testnet 52.

Co-authored-by: Conor Schaefer <conor@penumbralabs.xyz>
Co-authored-by: Justin Tieri <37750742+jtieri@users.noreply.github.com>

* Neutron launch fixes and optimizations (cosmos#1185)

* pipe max msgs through path processor

* only apply max msgs to packet msgs

* multiple msgs simultaneously on ordered chans

* flush should be more frequent if it fails or does not complete

* fix legacy

* handle feedback

* Problem: fixes in ibc-go v7.0.1 are not included (cosmos#1205)

* Problem: fixes in ibc-go v7.0.1 are not included

* add change doc

* Harry/rly address (cosmos#1204)

* added addresCmd to root and keys.go

* nicks

* nick

* made a common method "showAddressByChainAndKey" to be used by both addressCmd and keysShowCmd

---------

Co-authored-by: Harry <harrycosmos@Harrys-MacBook-Pro.local>
Co-authored-by: Andrew Gouin <andrew@gouin.io>

* deps: update to ibc-go v7.1.0-rc0 (cosmos#1207)

* Export wallet address for Prometheus metrics (cosmos#1206)

* export relayer address for pro

* address in updateFeesSpent

* make error messages consistent

* log error rather than return

* handle 0 balance

* chore: fix issue

* fix: connection key include

* chore: comment out flush

* fix: remove SET method from providerConfig

* chore: replace static naming with constant

* chore: packet timeout after relayer update

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Andrew Gouin <andrew@gouin.io>
Co-authored-by: Justin Tieri <37750742+jtieri@users.noreply.github.com>
Co-authored-by: Henry de Valence <hdevalence@hdevalence.ca>
Co-authored-by: Ava Howell <ava@penumbra.zone>
Co-authored-by: jtieri <justin@thetieris.com>
Co-authored-by: Ava Howell <ava@avahowell.me>
Co-authored-by: Conor Schaefer <conor@penumbralabs.xyz>
Co-authored-by: Dan Kanefsky <56059752+boojamya@users.noreply.github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Conor Schaefer <conorsch@users.noreply.github.com>
Co-authored-by: mmsqe <tqd0800210105@gmail.com>
Co-authored-by: Cosmos-Harry <110472914+Cosmos-Harry@users.noreply.github.com>
Co-authored-by: Harry <harrycosmos@Harrys-MacBook-Pro.local>
Co-authored-by: mmsqe <mavis@crypto.com>
Co-authored-by: Keefer Taylor | Tessellated <keefer@tessellated.io>
Co-authored-by: izyak <test@test.com>
Co-authored-by: viveksharmapoudel <viveksharmapoudel@gmail.com>
izyak added a commit to icon-project/ibc-relay that referenced this pull request Sep 8, 2023
* Flush query should include begin and end block events (cosmos#1125)

* Include begin and end block events

* disable flushing when termination condition is set

* Still flush for FlushLifecycle

* Add sort for flush logging to avoid confusion

* pre_init messages (cosmos#1131)

* Wire initial messages into path processor caches so that retry logic will occur

* Fix counterparty keys

* Remove debug log

* fix default coin type (cosmos#1134)

* fix slip44 default

* Add test case

* build: bump to Go 1.20 + bump deps (cosmos#1132)

* build: bump to Go 1.20 + bump deps

This bumps the Go version to 1.20 and also bumps the SDK version to 0.47.0 and ibc-go to v7.0.0

* chore: update GH workflows to use Go 1.20 + update interchaintest deps

* chore: update missing deps in go.sum + use 1.20 in dockerfiles

* chore: bump to `setup-go/v4` and remove caching step

* chore: bump to `checkout/v3` and remove caching step

* chore: bump 1.20.2

* Fix flushing acks (cosmos#1139)

* Fix ordered channel closure (cosmos#1142)

* Fix ordered channel closure

* Increase timeout for scenarios test

* Fix tracking of init messages when IDs aren't the same

* bump interchaintest to include explicit port bindings

* Fix flush termination condition (cosmos#1141)

* bump to main sha (cosmos#1143)

* Pre-filter flush channels (cosmos#1146)

* Add channel close correlation (cosmos#1145)

* Add channel close correlation

* Switch to pre-close key

* make tx channel-close cli command work, add test coverage

* more sweet code removals

* update comment

* Fix flush on ordered channels (cosmos#1150)

* Fix flush on ordered channels

* Queue all packets at nextseqrecv or above

* Now that we have periodic flushing, skip blocks if they can't be queried (cosmos#1154)

* Lock config file for all write operations (cosmos#1156)

* Lock config file for all write operations

* Fix linter errs

* tidy

* more tidy

* Penumbra support v2 (cosmos#1144)

* Penumbra buf go gen

* Use go prefix override

* wip: penumbra relayer provider

remove copied-over cosmos provider tests, rename processor cosmos -> penumbra

rename ccp -> pcp

reformat into new relayer dir structure

update penumbra types to point to buf.build

building again

* fix penumbra

* fix: implement MsgSubmitMisbehaviour

* fix: remove unnecessary proto file + fix msg type cast

* chore: add removal of penumbra client protos in protocgen

* working penumbra relayer functionality

wip: unbase64?

wip: multiple messages per penumbra tx

wip: stub SendMessages impl

wip: attempt to split out common method

wip: use random anchor and work around path renaming

wip: improve logging

wip: changes during pairing

https://www.youtube.com/watch?v=RYonSOkZ5ZE

clean up logs

skip height bug workaround

drop debug panic

* update penumbra chain processor connection and channel message processing

* cleanup logging statements for review

Responding to review comments, honoring the style guide for logging, and
removing some unused reference code that was commented out while
debugging.

---------

Co-authored-by: Andrew Gouin <andrew@gouin.io>
Co-authored-by: Ava Howell <ava@penumbra.zone>
Co-authored-by: jtieri <justin@thetieris.com>
Co-authored-by: Ava Howell <ava@avahowell.me>
Co-authored-by: Conor Schaefer <conor@penumbralabs.xyz>
Co-authored-by: Justin Tieri <37750742+jtieri@users.noreply.github.com>

* --time-threshold example use cases (cosmos#1155)

* Make ICA waits more explicit (cosmos#1157)

* Make ICA waits more explicit

* poll for timeout

* poll for channel close confirm

* Comment out sqlite db file for scenarios tests

* Bump github.com/docker/docker in /interchaintest (cosmos#1160)

Bumps [github.com/docker/docker](https://github.com/docker/docker) from 20.10.19+incompatible to 20.10.24+incompatible.
- [Release notes](https://github.com/docker/docker/releases)
- [Commits](moby/moby@v20.10.19...v20.10.24)

---
updated-dependencies:
- dependency-name: github.com/docker/docker
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* bump version in docs (cosmos#1158)

* Bump github.com/opencontainers/runc in /interchaintest (cosmos#1153)

Bumps [github.com/opencontainers/runc](https://github.com/opencontainers/runc) from 1.1.3 to 1.1.5.
- [Release notes](https://github.com/opencontainers/runc/releases)
- [Changelog](https://github.com/opencontainers/runc/blob/v1.1.5/CHANGELOG.md)
- [Commits](opencontainers/runc@v1.1.3...v1.1.5)

---
updated-dependencies:
- dependency-name: github.com/opencontainers/runc
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Andrew Gouin <andrew@gouin.io>

* Fix QueryConnectionsUsingClient for cosmos (cosmos#1162)

* Make min query loop duration configurable (cosmos#1164)

* penumbra provider: update generated protos (cosmos#1168)

* Add sr25519 support (cosmos#1120)

* Migrate to cometbft

* Additional replaces

* Register comet proto types

* Add sr25519 support

* bump ictest

* Add keys test

* Update supported algos comment

* penumbra: update protos (cosmos#1170)

Updating the generated protos for Penumbra support. After lightly
editing the `scripts/protocgen.sh` script, I ran `make proto-gen`
and then committed the changes `relayer/chains/penumbra/`. Other
automatically updated protos I intentionally excluded from this PR, to
avoid side-effects.

Co-authored-by: Conor Schaefer <conor@penumbralabs.xyz>

* chore: add path name to logs in message processor (cosmos#1171)

* Fix multiple conn open init (cosmos#1173)

* allow register with extra_codecs (cosmos#1175)

* Harry/fee middleware (cosmos#1174)

* Register Counterparty relayer cmd and fee middleware test

* debugging the command

* debugging and finalizing the fee_middleware_test.

* debugging and finalizing the fee_middleware_test.

* merged with updated repo

* clear out some commanded code

* nits and suggestions post review

* more nits

* added one val no fullnode as chainspec

---------

Co-authored-by: Harry <harrycosmos@Harrys-MacBook-Pro.local>

* fix: nil receiver initiate for path (cosmos#1177)

* fix nil receiver initiate for path

ensure path get written to config

* add change doc

* feat: add max-gas-amount parameter in chain configs (cosmos#1178)

* add max fee

* add test

* add change doc

* Update cregistry/chain_info.go

* dep: bump sdk from v0.47.0 to v0.47.2 (cosmos#1180)

* bump sdk from v0.47.0 to v0.47.2

fix btcutil dep

* fix dep of hdkeychain

* Harry/rly tx channel (cosmos#1183)

* made a new method "logChannelOpenMessage" to log the newly opened channel into info level

* added fields

* some changes

---------

Co-authored-by: Harry <harrycosmos@Harrys-MacBook-Pro.local>

* Harry/rly tx transfer (cosmos#1184)

* made a new method "logChannelOpenMessage" to log the newly opened channel into info level

* added fields

* some changes

* recreated issue 1151, added logs

---------

Co-authored-by: Harry <harrycosmos@Harrys-MacBook-Pro.local>
Co-authored-by: Andrew Gouin <andrew@gouin.io>

* Better Error Messaging when failing to query the Block Height (cosmos#1189)

* better block data errors

* remove redundant field

* penumbra: update protos (cosmos#1181)

Matches the latest protos shipped with the Penumbra Testnet 52.

Co-authored-by: Conor Schaefer <conor@penumbralabs.xyz>
Co-authored-by: Justin Tieri <37750742+jtieri@users.noreply.github.com>

* Neutron launch fixes and optimizations (cosmos#1185)

* pipe max msgs through path processor

* only apply max msgs to packet msgs

* multiple msgs simultaneously on ordered chans

* flush should be more frequent if it fails or does not complete

* fix legacy

* handle feedback

* Problem: fixes in ibc-go v7.0.1 are not included (cosmos#1205)

* Problem: fixes in ibc-go v7.0.1 are not included

* add change doc

* Harry/rly address (cosmos#1204)

* added addresCmd to root and keys.go

* nicks

* nick

* made a common method "showAddressByChainAndKey" to be used by both addressCmd and keysShowCmd

---------

Co-authored-by: Harry <harrycosmos@Harrys-MacBook-Pro.local>
Co-authored-by: Andrew Gouin <andrew@gouin.io>

* deps: update to ibc-go v7.1.0-rc0 (cosmos#1207)

* Export wallet address for Prometheus metrics (cosmos#1206)

* export relayer address for pro

* address in updateFeesSpent

* make error messages consistent

* log error rather than return

* handle 0 balance

* chore: fix issue

* fix: connection key include

* chore: comment out flush

* fix: remove SET method from providerConfig

* chore: replace static naming with constant

* chore: packet timeout after relayer update

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Andrew Gouin <andrew@gouin.io>
Co-authored-by: Justin Tieri <37750742+jtieri@users.noreply.github.com>
Co-authored-by: Henry de Valence <hdevalence@hdevalence.ca>
Co-authored-by: Ava Howell <ava@penumbra.zone>
Co-authored-by: jtieri <justin@thetieris.com>
Co-authored-by: Ava Howell <ava@avahowell.me>
Co-authored-by: Conor Schaefer <conor@penumbralabs.xyz>
Co-authored-by: Dan Kanefsky <56059752+boojamya@users.noreply.github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Conor Schaefer <conorsch@users.noreply.github.com>
Co-authored-by: mmsqe <tqd0800210105@gmail.com>
Co-authored-by: Cosmos-Harry <110472914+Cosmos-Harry@users.noreply.github.com>
Co-authored-by: Harry <harrycosmos@Harrys-MacBook-Pro.local>
Co-authored-by: mmsqe <mavis@crypto.com>
Co-authored-by: Keefer Taylor | Tessellated <keefer@tessellated.io>
Co-authored-by: izyak <test@test.com>
Co-authored-by: viveksharmapoudel <viveksharmapoudel@gmail.com>
ashishchandr70 added a commit to sagaxyz/relayer that referenced this pull request Oct 4, 2023
* Better Error Messaging when failing to query the Block Height (cosmos#1189)

* better block data errors

* remove redundant field

* penumbra: update protos (cosmos#1181)

Matches the latest protos shipped with the Penumbra Testnet 52.

Co-authored-by: Conor Schaefer <conor@penumbralabs.xyz>
Co-authored-by: Justin Tieri <37750742+jtieri@users.noreply.github.com>

* Neutron launch fixes and optimizations (cosmos#1185)

* pipe max msgs through path processor

* only apply max msgs to packet msgs

* multiple msgs simultaneously on ordered chans

* flush should be more frequent if it fails or does not complete

* fix legacy

* handle feedback

* Problem: fixes in ibc-go v7.0.1 are not included (cosmos#1205)

* Problem: fixes in ibc-go v7.0.1 are not included

* add change doc

* Harry/rly address (cosmos#1204)

* added addresCmd to root and keys.go

* nicks

* nick

* made a common method "showAddressByChainAndKey" to be used by both addressCmd and keysShowCmd

---------

Co-authored-by: Harry <harrycosmos@Harrys-MacBook-Pro.local>
Co-authored-by: Andrew Gouin <andrew@gouin.io>

* deps: update to ibc-go v7.1.0-rc0 (cosmos#1207)

* Export wallet address for Prometheus metrics (cosmos#1206)

* export relayer address for pro

* address in updateFeesSpent

* make error messages consistent

* log error rather than return

* handle 0 balance

* chore: replace gogo/protobuf to cosmos/gogoproto (cosmos#1208)

* rm dup pb

* add missing cp for messages.proto

* make proto-gen

* migrate gogo/protobuf to cosmos/gogoproto

* add change log

* mod tidy

* feat: add extension_options parameter in chain configs (cosmos#1179)

* support extension options for build tx

* add test

* add change doc

* rm dup pb

* update mod

* point to v0.47.3

* feat: support localhost ibc (cosmos#1191)

* WIP: testing localhost ibc client

* WIP: localhost ibc support

* WIP: debugging channel handshake correlation bug

* WIP: debugging localhost IBC

* debugging failed ibc transfers

* chore: remove debug output

* fix: get acks working + cleanup localhost data handling

* test: add additional assertions, debug failing timeouts

* fix: remove redundant chankey assignment

* fix: update to latest interchaintest commit + fix test

* fix: hack to get ordered channels working on localhost

* test: implement ica test case for localhost

* fix: update reverted Go version

* test: fix flaky scenario e2e test

* fix: address suggestions from code review

* dep: bump cometbft and ibc-go (cosmos#1221)

* bump cometbft to v0.37.2

* bump ibc-go to v7.2.0

* add change doc

* add missing stop relayer to avoid log after test complete (cosmos#1229)

Co-authored-by: Justin Tieri <37750742+jtieri@users.noreply.github.com>

* fix: avoid invalid Bech32 prefix in scenario test (cosmos#1226)

* avoid invalid Bech32 prefix

due to singleton GetConfig

* add change doc

* separate process in ci

* separate fee middleware test for juno

* wait more blks for ack (cosmos#1222)

* penumbra provider: update proof spec (cosmos#1232)

* fix: flag accessed but not defined: flush-interval (cosmos#1238)

* penumbra provider: update protos (cosmos#1245)

* fix: Suppressing scary SDK error on redundant packets (cosmos#1214)

Co-authored-by: Andrew Gouin <andrew@gouin.io>

* catch error if type is missing (cosmos#1234)

Co-authored-by: Andrew Gouin <andrew@gouin.io>

* Export client expiration metric to prometheus (cosmos#1235)

* export client expiration metric

* finalize

* add path name

* snake case

* change label to `chain`

* trusting period as string

* Export configured gas prices to prometheus wallet balance metric (cosmos#1236)

* export gas price to prom

* update label

* update fees spent metric

* snake case

* Exports block query errors to prometheus metrics (counter) (cosmos#1239)

* separate by type

* add help info

* remove new line in help and fix readme

* feedback

* Export TX failures to prometheus metrics (counter) (cosmos#1240)

* export tx failures to prometheus

* change label to `cause`

* use the name given by the user to generate the fetch URL (cosmos#1233)

* use the name given by the user to generate the fetch URL

* add example

---------

Co-authored-by: Andrew Gouin <andrew@gouin.io>
Co-authored-by: Dan Kanefsky <56059752+boojamya@users.noreply.github.com>

* feegrant PR (cosmos#1140)

* Feegrant support

* Test case for address caching bugfix

* Bugfix for SDK account prefix. Feegrant test passing.

* Mutex for signer expanded to include feegrantees

* Cleaned up feegrant test case

* Cleaned up feegrant test case

* Cleaned up feegrant test case

* check round robin feegrant behavior by counting number of TXs each grantee signer

* module updates from merge

* v0.47.0 with bech32 address cache fix

* Move SetAddrCacheEnabled to NewRelayer func for full coverage

* Do not hardcode chain id in feegrant test case

* Wait more blocks for ibc transfers

* disable cosmos SDK bech32 address cache for rly start command

* Fix sloppy comments/remove unnecessary code

* Faster acc caching unit test

* Penumbra provider feegrant support

* Merge upstream

* Fixed merge issue where feegrant config wasn't being written to file

* feegrant patch for cosmos-sdk v0.47.1

* merge from main

* Update to cosmos-sdk v0.47.2

* Increase test case blocks to wait

* Fixed data race by moving test parallelization after relayer wallet build

* Increased TestScenarioICAChannelClose timeout height

* Cleanup feegrant test case

* Fixed race condition in sequence guard w/ mutex

* Automatic retry for TX lookup in feegrant test case

---------

Co-authored-by: Andrew Gouin <andrew@gouin.io>

* Export Client Trusting Period to Prometheus metrics (cosmos#1246)

* export client trusting period

* update docs

* separate feegrant test to avoid no space left on device (cosmos#1250)

from scenarios test in ci

* Add extra client info when querying client expiration (cosmos#1247)

* extra client info

* cleanup print

* remove extra comments

* add alias

* allow only one client

* spelling

* remainingTime

---------

Co-authored-by: Justin Tieri <37750742+jtieri@users.noreply.github.com>

* next seq ack handling (cosmos#1244)

* next seq ack handling and chan order

* use max msgs for ack flush

* improve logs

* fix check

* don't override unless not chantypes.NONE

* fix: Suppressing scary SDK error on redundant packets (cosmos#1214)

Co-authored-by: Andrew Gouin <andrew@gouin.io>

* tidy logic

* improve logic and order detection

* shorten flushFailureRetry

* check empty string

* tidy logs. better account sequence regex. don't split up ordered channel batches

---------

Co-authored-by: Joe Abbey <joe.abbey@gmail.com>
Co-authored-by: jtieri <justin@thetieris.com>

* chore: update penumbra protos to v0.57.0 (cosmos#1249)

Version 0.57.0 of Penumbra was released on 2023-07-26 [0].
This commit pulls in the latest proto defs from BSR.

[0] https://github.com/penumbra-zone/penumbra/releases/tag/v0.57.0

Co-authored-by: Conor Schaefer <conor@penumbralabs.xyz>
Co-authored-by: Justin Tieri <37750742+jtieri@users.noreply.github.com>

* fix: reduce get bech32 prefix when get signer (cosmos#1231)

* add missing getFeePayer for clienttypes

* add missing getFeePayer for feetypes

* apply in penumbra

* add change doc

---------

Co-authored-by: Justin Tieri <37750742+jtieri@users.noreply.github.com>

* update setup-go action (cosmos#1251)

* fix for feegrants (cosmos#1256)

* Feegrant support

* Test case for address caching bugfix

* Bugfix for SDK account prefix. Feegrant test passing.

* Mutex for signer expanded to include feegrantees

* Cleaned up feegrant test case

* Cleaned up feegrant test case

* Cleaned up feegrant test case

* check round robin feegrant behavior by counting number of TXs each grantee signer

* module updates from merge

* v0.47.0 with bech32 address cache fix

* Move SetAddrCacheEnabled to NewRelayer func for full coverage

* Do not hardcode chain id in feegrant test case

* Wait more blocks for ibc transfers

* disable cosmos SDK bech32 address cache for rly start command

* Fix sloppy comments/remove unnecessary code

* Faster acc caching unit test

* Penumbra provider feegrant support

* Merge upstream

* Fixed merge issue where feegrant config wasn't being written to file

* feegrant patch for cosmos-sdk v0.47.1

* merge from main

* Update to cosmos-sdk v0.47.2

* Increase test case blocks to wait

* Fixed data race by moving test parallelization after relayer wallet build

* Increased TestScenarioICAChannelClose timeout height

* Cleanup feegrant test case

* Fixed race condition in sequence guard w/ mutex

* Automatic retry for TX lookup in feegrant test case

* Disable cosmos SDK address cache on app initialization via main package init()

* Added docs for feegrant in the advanced usage guide

* Removed commented out code

* Removed commented out code

* Added detail to feegrant docs, fixed minor issue with test case

* Added detail to feegrant docs, fixed minor issue with test case

---------

Co-authored-by: Andrew Gouin <andrew@gouin.io>
Co-authored-by: Kyle <kyle@strange.love>

* chore: update penumbra protos (cosmos#1260)

Updating to the latest Penumbra upstream protos.
We'll likely submit another round of changes ahead of the next public
testnet release, as a heads up.

Co-authored-by: Conor Schaefer <conor@penumbralabs.xyz>

* Change 2.3.0 to 2.4.0 (cosmos#1253)

Co-authored-by: Justin Tieri <37750742+jtieri@users.noreply.github.com>

* rename path to path_name for consistency (cosmos#1262)

* Use unique names for relayer images & cleanup when purpose served (cosmos#1269)

* Use unique names for relayer images & cleanup when purpose served

* move random tag generation and teardown image to within BuildRelayerImage

* fix return line

* use ibc-go capability module (cosmos#1277)

* use ibc-go capability module

* tidy interchaintest

* fix: use resp.Events to parse events instead of logs (cosmos#1271)

* fix: use resp.Events to parse events instead of logs

* revert: use legacy behaviour as fallback mechanism

* refactor: use legacy approach first, fallback onto new parsing approach

---------

Co-authored-by: Justin Tieri <37750742+jtieri@users.noreply.github.com>

* working correctly with 7.3 and 0.47.5 (cosmos#1280)

Co-authored-by: Justin Tieri <37750742+jtieri@users.noreply.github.com>

* Add use command to "rly keys" (cosmos#1282)

* Update README.md with an update to leverage 'rly key use' in step 5 (cosmos#1289)

* Update README.md with an update to leverage 'rly key use' in step 5

* change alias from 'a' to 'u' for use subcommand

* Ability to fetch specific chain paths only (cosmos#1291)

* feat: allow a relayer to fetch a specific chain only

* minor: check specific path pair logic earlier

* faddat/upgrade go (cosmos#1279)

* upgrade ci to go 1.21

* upgrade to go1.21

---------

Co-authored-by: Justin Tieri <37750742+jtieri@users.noreply.github.com>

* Split scenarios test (cosmos#1294)

* Split scenarios test

* use matrix

* updates

* need to cd into dir first

* handle deprication

* Remove rouge entry into the matrix

* Ensure other parallel tests run to completion even if one of them fail

* Add explanation

* Remove rougue whitespace

* Add verbosity and timeout for scenarios tests (cosmos#1295)

* Add output flag for query sub commands  results printed to console.  (cosmos#1281)

* output json for query cmd's  balance & clients-expirations
* print proper json instead of bytes for headers command
* Add output flag . Use legacy,json options
* Update according to reviews

* Add ability to fetch testnet chains and paths + force-add ability (cosmos#1285)

* add testnet and force-add

* update cmd examples

* improve usage description

* update interchaintest workflow (cosmos#1298)

* Query param prop directly (cosmos#1264)

* Query param prop directly

* Flip order of queries for QueryUnbondingPeriod

* Add fallback for chains using cosmos-sdk 47+

* Trusting period logic remains same

* Add Fallback

* Consolidate functions into a single queryParamsSubspaceTime

* Saga IBC transfers between SPC and chainlet

* Set keystore password

* Poll chainlet until it's up and running

* Remove unused mnemonic file

* Rename example .json files to .json.example

* Env vars renaming

* Rename SPC_EXTERNAL_ADDRESS -> SPC_RPC_EXTERNAL_ADDRESS

* Feature/ccv (#2)

* Updates to support CCV

* Re-added out/err redirect to log files

* Add multiple tries for linking. Upd chain waiting

* Minor fix to Github action. Update rly start

---------

Co-authored-by: Ashish Chandra <ashish@saga.xyz>

* Committing go.work.sum

---------

Co-authored-by: Keefer Taylor | Tessellated <keefer@tessellated.io>
Co-authored-by: Conor Schaefer <conorsch@users.noreply.github.com>
Co-authored-by: Conor Schaefer <conor@penumbralabs.xyz>
Co-authored-by: Justin Tieri <37750742+jtieri@users.noreply.github.com>
Co-authored-by: Andrew Gouin <andrew@gouin.io>
Co-authored-by: mmsqe <mavis@crypto.com>
Co-authored-by: Cosmos-Harry <110472914+Cosmos-Harry@users.noreply.github.com>
Co-authored-by: Harry <harrycosmos@Harrys-MacBook-Pro.local>
Co-authored-by: Dan Kanefsky <56059752+boojamya@users.noreply.github.com>
Co-authored-by: Ava Howell <ava@avahowell.me>
Co-authored-by: mindcarver <32150062+mindcarver@users.noreply.github.com>
Co-authored-by: Joe Abbey <joe.abbey@gmail.com>
Co-authored-by: murataniloener <anil@b9lab.com>
Co-authored-by: KyleMoser <KyleMoser@users.noreply.github.com>
Co-authored-by: jtieri <justin@thetieris.com>
Co-authored-by: Kyle <kyle@strange.love>
Co-authored-by: Sr20de <106104431+Sr20dem@users.noreply.github.com>
Co-authored-by: danb <dan@bryanlabs.net>
Co-authored-by: vimystic <122659254+vimystic@users.noreply.github.com>
Co-authored-by: Jacob Gadikian <jacobgadikian@gmail.com>
Co-authored-by: colin axnér <25233464+colin-axner@users.noreply.github.com>
Co-authored-by: Reece Williams <31943163+Reecepbcups@users.noreply.github.com>
Co-authored-by: Konstantin Munichev <toobwn@gmail.com>
Co-authored-by: Ashish Chandra <ashish@saga.xyz>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants