v1.10.0

June 24th, 2024

This release enhances filter configurations and includes the following updates:

1. excluded_sequences supports sequence ranges in addition to exact values, e.g. [1, 2, "5-10", 13] is now valid and would exclude packet with sequences 1, 2, 5, 6, 7, 8, 9, 10, 13.
2. packet_filter now ignores unintended whitespace.
3. A new allow_ccq per-chain configuration has been added to skip the relaying of ICS31 Cross Chain Queries.

Additionally, various improvements to testing and bug fixes have been implemented.

• General

  • Fix a bug where in some cases, Hermes would drop all events in a
    batch that came after an event rejected by the filtering policy
    (#4034)

• Relayer Library

  • Discard CrossChain queries intended for unconfigured chains.
    (#4021)

• Integration Test Framework

  • Add tests to ensure that Hermes correctly relays transfer messages
    from a grantee address with granted authorisation using authz module.
    (#4046)

• Relayer Library

  • Add a new per-chain configuration allow_ccq to enable or disable
    relaying of ICS31 Cross Chain Query packets.
    (#4040)

• Integration Test Framework

  • Update the version of Gaia running the integration tests in the CI from v15.2.0
    to v17.2.0 (#4023)
  • Update the version of Osmosis running the integration tests in the CI from v24.0.1
    to v25.0.0 (#4024)
  • Update the version of Juno running the integration tests in the CI from v21.0.0
    to v22.0.0 (#4025)
  • Update the version of Neutron running the integration tests in the CI from v3.0.2
    to v3.0.5 (#4026)
  • Update the version of Celestia app running the integration tests in the CI from v1.4.0
    to v1.11.0 (#4027)
  • Update the version of wasmd running the integration tests in the CI from v0.50.0
    to v0.51.0 (#4029)
  • Reduce run time for ICS29 tests by immediately verifying if either
    the legacy fees, recv_fee + ack_fee + timeout_fee or current
    fees, max(recv_fee + ack_fee, timeout_fee) have been escrowed.
    (#4053)
  • Refactored the test-framework bootstrapping for n-ary chain tests
    to utilize the specified topology.
  • Currently, only linear, cyclic and fully connected topologies are supported.
    (#4038)

• Relayer Library

  • Use custom User-Agent for Hermes queries
    (#3979)
  • Updated the channel and port filter parsing to ignore whitespaces.
    This will prevent unintended channel scanning due to accidental
    whitespaces when exact matches are specified in the packet_filter
    configuration.
    (#4045)
  • Improve the excluded_sequences configuration so that it now accepts
    ranges of sequence values in addition to exact values.
    Accepted format:

  • Exact sequence, e.g. [1, 2, 3]

  • "-" separator, e.g. ["1-3"]

    These can be combined making the following configurations equivalent:

  • excluded_sequences = { 'channel-0' = [1, "3-5", 7, "9-12"] }

  • excluded_sequences = { 'channel-0' = [1, 3, 4, 5, 7, 9, 10, 11, 12] }

    (#4047)

v1.9.0

May 30th, 2024

This v1.9.0 release introduces new features and improvements to Hermes.

Major Features:

1. Channel Upgrades: Hermes now handles channel upgrade events introduced in ibc-go v8, helping chains opting-in to new functionality on existing channels.
2. Dynamic Gas Fees Compatibility: Hermes is now compatible with Skip's x/feemarket module for dynamic gas fees, in addition to Osmosis' implementation, providing more flexibility in gas fee management.

Additionally, this release includes various bug fixes enhancing the stability and performance of Hermes. These fixes address issues with channel and connection creation on older ibc-go versions, event extraction, health-check messages, and more.

BREAKING CHANGES

• Telemetry & Metrics
  • Remove the telemetry and rest-server feature flags, ensuring Hermes is always built with telemetry and REST support. (#3878)
    Both servers can still be disabled in the configuration file, by setting telemetry.enabled = false and rest.enabled = false, respectively.

FEATURES

• Relayer Library
  • Add support for upgrading channels, as per the ICS 004 specification (#3228)
    This feature allows chains to upgrade an existing channel to take advantage of new features without having to create a new channel, thus preserving all existing packet state processed on the channel. For example, a channel could now be upgraded to enable the ICS 029 fee middleware, allowing relayer operators on that channel to receive fees each time they relay an incentivized packet.
  • Improve reliability of event source in pull mode by proceeding to next block even if Hermes cannot parse the current block (#3894)
    Add new configuration option to event_source setting: max_retries defines how many times Hermes should attempt to pull a block over RPC.

    event_source = { mode = 'pull', interval = '1s', max_retries = 4 }

IMPROVEMENTS

• Integration Test Framework
  • Update the version of Juno running the integration tests in the CI from v17.1.1 to v21.0.0 (#3959)
  • Update the version of Migaloo Chain running the integration tests in the CI from v3.0.2 to v4.1.3 (#3960)
  • Update the version of wasmd running the integration tests in the CI from v0.30.0 to v0.50.0 (#3961)
  • Update the version of ibc-go simapp running the integration tests in the CI from v8.2.0 to v8.3.1 (#4009)
• Relayer Library
  • Update to tendermint-rs v0.36.0 (#3966)
  • Use packet_ack_hex event attribute instead of deprecated packet_ack attribute to decode WriteAck event (#3921)
  • Add support for dynamic gas fee for chains using Skip's x/feemarket module, while keeping compatibility with Osmosis' bespoke implementation (#4000)
• Relayer CLI
  • Use RPC (pull) event source instead of WebSocket (push) when generating configuration with hermes config auto (#3913)

BUG FIXES

• Relayer Library
  • Fix creation of channels and connection on chains using an unsupported version of ibc-go, eg. Sei (#3817)
  • Fix a bug where Hermes would only ever extract the first emitted ICS 031 CrossChain Query event, which would cause it to miss the other CCQ events. (#3954)
• Relayer CLI
  • Fixed minimum-gas-prices health-check messages and make it more verbose and legible (#3893)
  • Set compat_mode for pull mode in hermes listen command (#3910)
  • Fixed the trusted height consensus state query when submitting the double vote evidence (#3999)

v1.8.3

May 28th, 2024

This patch release fixes a bug that may happen prevent the relayer from submitting the evidence for a duplicate vote in some cases.

Bug Fixes

• Relayer CLI
  • Fix the trusted height consensus state query when submitting the double vote evidence (#3999)

v1.8.2

March 12th, 2024

This release fixes the two following bugs and improves the connection and channel handshake retry mechanism:

• Fix erroneous warnings for incompatible version of IBC-Go during health checks, ensuring accurate compatibility reporting
• Fix a bug in the clear packets command which caused packet clearing to fail with an "counterparty channel not found" error

BUG FIXES

• Correctly use the counterparty channel and port IDs when clearing the packets from the destination chain to the source chain in the packet clear command (#3889)
• Fix parsing of IBC-Go version in health check and improve health check reporting (#3880)

FEATURES

• Add Injective chain to the chains running the integration tests in the CI (#3887)

IMPROVEMENTS

• Change connection and channel handshake retry strategy to retry at most 10 times (5 times per block max) (#3864)

v1.8.1

March 7th, 2024

This v1.8.1 release brings better reliability when relaying, more enhanced configuration and improved monitoring.

Reliability has been improved:

• It is now possible to relay ICS-04 packets with non-UTF-8 payloads
• Packet sequences are now verified for ordered channels before trying to relay

Additional per-chain configurations have been added:

• excluded_sequences used to skip problematic packets when clearing
• memo_overwrite allowing users to overwrite the relayer memo when chains have a strict limit for the size of the memo.

Monitoring issues improvements:

• A new metric simulate_errors which counts the number of failed simulated transactions
• Out of gas error diagnostic gives more information and a dedicated entry to the guide has been added
• Failed gas simulation will not be considered as unrecoverable for legacy chains.
• The compatibility check during the health-check has been improved will assess more correctly the versions for Ibc-Go and Cosmos SDK

Special thanks to our contributors for their valuable additions to this release:

• Sergey (@freak12techno) for adding the simulate_errors metric (#3845)
• Martin Dyring-Andersen (@mdyring) for adding recovery from failed gas simulation for legacy chains (#3792)

BUG FIXES

• Allow relaying ICS-04 packets with non-UTF-8 payloads (#3770)
  Hermes does not assume anymore that an ICS-04 packet data is valid UTF-8,
  by using the packet_data_hex attribute when assembling a packet from events, instead of the deprecated packet_data attribute.
  Relying on the packet_data attribute enforces a UTF-8 encoded payload (eg. JSON), disallowing eg. Protobuf-encoded payloads.
  The packet_data attribute has been deprecated in favor of packet_data_hex since IBC-Go v1.0.0.
• Improve reliability of compatibility check and fix parsing of expected modules versions (#3831)

FEATURES

• Add a per-chain configuration excluded_sequences allowing users to specify a list of packet sequences which will not be cleared.
  This configuration has no impact on standard packet relaying. (#3754)
• Add a per-chain configuration memo_overwrite allowing users to overwrite the relayer memo used for each transaction (#3811)
• Added a new Prometheus metric simulate_errors for tracking when a transaction simulation fails, with the following labels: (#3845)

  • recoverable (can the execution continue if this happened?)

  • account (account from which the tx was sent)

  • error_description (description of the error)

    # HELP simulate_errors_total Number of errors observed by Hermes when simulating a Tx
    # TYPE simulate_errors_total counter
    simulate_errors_total{account="osmo17ndx5qfku28ymxgmq6zq4a6d02dvpfjjul0hyh",error_description="Unknown error",recoverable="false",service_name="unknown_service",otel_scope_name="hermes",otel_scope_version=""} 4
    

IMPROVEMENTS

• Use the consensus state at client latest height in status CLI (#3814)
• Add syncing check for gRPC node (#3814)
• Improve the log diagnostic when an out of gas error is thrown. A new entry related to gas error has been added to the Hermes guide. (#3530)
• Improve resilience when relaying on ordered channels.
  When relaying packets on an ordered channel, Hermes will now attempt to detect whether the next message to send has the sequence number expected on that channel. If there is a mismatch, then Hermes will trigger a packet clear on the channel to unblock it before resuming operations on that channel. (#3540)
• Recover from gas simulation failures on legacy chains. (#3792)

v1.8.0

January 23rd, 2024

This v1.8.0 release introduces new features and improvements to Hermes.

One key feature is that Hermes is now compatible with both the legacy UpgradeProposal and the newer MsgIbcSoftwareUpgrade message when upgrading a chain.
This allows Hermes to be compatible with ibc-go v8.0.0. The compatibility check that Hermes performs on startup has been updated to reflect this.

Additional configuration settings have been added:

• The new global settings ics20_max_memo_size and ics20_max_receiver_size allow users to specify a limit for the size of the memo and receiver fields for ICS20 packets. Any packet with either field having a size exceeding the configured values will not be relayed.
• The new per-chain setting query_packets_chunk_size allows users to specify how many packets are queried at once from the chain when clearing pending packets. This is useful to tweak when there are many large pending packets and the RPC endpoints times out or refuses to answer the pending packets query.
• The new per-chain setting client_refresh_rate can be use to specify how often the clients referencing this chain should be refreshed. The rate is expressed as a fraction of the trusting period.
• The new per-chain setting dynamic_gas_price can be enabled to have the relayer query for and use a dynamic gas price instead of using the static gas_price specified in the config. This should only be used for chains which have a EIP-1559-like fee market enabled and support the osmosis.txfees.v1beta1.Query/GetEipBaseFee gRPC query.

Telemetry now features new metrics:

• Monitoring the ICS20 packets filtered due to the memo and/or receiver field size exceeding the configured limits.
• Monitoring the distribution of dynamic gas fees queried from the chain, if enabled.

BREAKING CHANGES

• Bump MSRV to 1.71 (#3688)

FEATURES

• Relayer

  • Use legacy UpgradeProposal or newer MsgIbcSoftwareUpgrade message when upgrading
    a chain depending on whether the chain is running IBC-Go v8 or older.
    (#3696)

  • Add a new per-chain configuration table dynamic_gas_price which enables
    querying the current gas price from the chain instead of the static gas_price,
    when the chain has EIP-1559-like dynamic gas price.
    The new configuration setting can be configured per-chain as follows:

    dynamic_gas_price = { enabled = true, multiplier = 1.1, max = 0.6 }

    At the moment, only chains which support the osmosis.txfees.v1beta1.Query/GetEipBaseFee
    query can be used with dynamic gas price enabled.
    (#3738)

  • Add two new packet configurations:

    • ics20_max_memo_size which filters ICS20 packets with memo field bigger than the configured value
    • ics20_max_receiver_size which filters ICS20 packets with receiver field bigger than the configured value
      (#3766)

  • Add a client_refresh_rate setting to specify the rate at which to refresh clients referencing this chain, relative to its trusting period.
    (#3402)

  • Add a --packet-sequences flag to the clear packets, tx packet-recv, and tx packet-ack commands.
    When this flag is specified, these commands will only clear the packets with the specified sequence numbers
    on the given chain. If not provided, all pending packets will be cleared on both chains, as before.
    (#3672)

    This flag takes either a single sequence number or a range of sequences numbers.
    Each element of the comma-separated list must be either a single sequence number or
    a range of sequence numbers.

    Examples:

    • 10 will clear a single packet with sequence number 10
    • 1,2,3 will clear packets with sequence numbers 1, 2, 3
    • 1..5 will clear packets with sequence numbers 1, 2, 3, 4, 5
    • ..5 will clear packets with sequence numbers 1, 2, 3, 4, 5
    • 5.. will clear packets with sequence numbers greater than or equal to 5
    • ..5,10..20,25,30.. will clear packets with sequence numbers 1, 2, 3, 4, 5, 10, 11, ..., 20, 25, 30, 31, ...
    • ..5,10..20,25,30.. will clear packets with sequence numbers 1, 2, 3, 4, 5, 10, 11, ..., 20, 25, 30, 31, ...

  • Add a --gov-account option to hermes tx upgrade-chain to specify the authority account used to sign upgrade proposal for chains running IBC-Go v8+.
    (#3696)

  • Add a query_packets_chunk_size config option and a --query-packets-chunk-size flag to the clear packets CLI to configure how
    many packets to query at once from the chain when clearing pending packets. Lower this setting if one or more of packets you are
    trying to clear are huge and make the packet query time out or fail.
    (#3743)

• Telemetry & Metrics
  • Add three metrics related to EIP gas price:
  • dynamic_gas_queried_fees contains data on the queried values before applying any filter
  • dynamic_gas_queried_success_fees contains data on the queried values if the query was successful and before applying any filter
  • dynamic_gas_paid_fees contains data on the queried values after applying the max filter
    (#3738)
  • Add a new metric filtered_packets which counts the number of packets filtered due to having a memo or receiver field too big
    (#3794)
• Integration Test Framework
  • Add a test for asynchronous Interchain Query relaying
    (#3455)
  • Add an ICA test to assert a channel correctly closes after a packet time-outs
    (#3778)

IMPROVEMENTS

• Relayer CLI
  • Update compatibility check to allow IBC-Go 4.1.1 to 8.x and SDK 0.45.x to 0.50.x.
    (#3745)

v1.7.4

December 15th, 2023

Special thanks to Yun Yeo (@beer-1) for their contributions (#3697 and #3703).

This release improves the monitoring of Hermes instances by fixing the broadcast_errors metric so that it correctly batches the same errors together. It also improves the metrics backlog_* by updating them whenever Hermes queries pending packets.

This release also improves the reliability of the idle worker clean-up and fixes a bug within the evidence command which would sometimes prevent the misbehaviour evidence from being reported.

BUG FIXES

• Relayer CLI
  • Fix a bug in the evidence command which would sometimes prevent the detected misbehaviour evidence from being submitted, instead erroring out with a validator set hash mismatch. (#3697)
• Relayer Library
  • Avoid retrieving a worker which is being removed by the idle worker clean-up process. (#3703)
• Telemetry & Metrics
  • Fix the issue where broadcast_errors metric would not correctly batch the same errors together. (#3720)
  • Update the values of backlog metrics when clearing packets.
    Change the backlog_oldest_timestamp to backlog_latest_update_timestamp which shows the last time the backlog metrics have been updated. (#3723)

Full changes: v1.7.3...v1.7.4

v1.7.3

November 29th, 2023

This release improves the reliability of the evidence command and fixes a bug that was preventing evidence to be reported, as seen on the Gaia RS testnet.

BUG FIXES

• Relayer CLI
  • Improve reliability of evidence command and fix a bug that was preventing evidence to be reported, as seen on the Gaia RS testnet (#3702)

Full changes: v1.7.2...v1.7.3

v1.7.2

November 28th, 2023

v1.7.2

November 28th, 2023

This patch release of Hermes adds a metric to improve monitoring errors and one
to measure the efficiency of the client update skip feature released in patch v1.7.1.

• broadcast_errors records the number of times a specific error is observed by Hermes when broadcasting transactions.
• client_updates_skipped records the number of client updates skipped due to the consensus states already existing.

FEATURES

• Telemetry & Metrics
  • Added metric client_updates_skipped to track the number of client
    update messages skipped due to the conscensus state existing already.
    (#3707)
  • Add a new metric broadcast_errors which
    records the number of times a specific error is observed by Hermes when broadcasting transactions
    (#3708)

Full changes: v1.7.1...v1.7.2

v1.7.1

November 13th, 2023

This patch release of Hermes now allows operators to set the clearing interval at a different value for each chain, using the new per-chain clear_interval setting. The global clear_interval setting is used as a default value if the per-chain setting is not defined.

Additionnaly, operators can now override the CometBFT compatibility mode to be used for a chain by using the new compat_mode per-chain setting. The main use case for this is to override the automatically detected compatibility mode in case Hermes gets it wrong or encounters a non-standard version number and falls back on the wrong CometBFT version.

On top of that, Hermes will now attempt to save on fees by not building a client update for a given height if the consensus state for that height is already present on chain.

FEATURES

• Add an optional per-chain setting compat_mode, which can be used to specify which CometBFT compatibility mode is used for interacting with the node over RPC. (#3623)
• Add a configuration which allows to override the clear_interval for specific chains (#3691)

IMPROVEMENTS

• Hermes now saves on fees by not including a client update if the consensus state at desired height is already present on chain. (#3521)

Full changes: v1.7.0...v1.7.1