Sealevel: Kathy messages are sent to/from sealevel chains

[tkporter]

Partially blocked by #2502 - I think we can start this ticket before that's finished though

• We decided in https://discord.com/channels/935678348330434570/935678739663192184/1126547994884460634 to keep this TS native
• We need to decide if we want to do some SDK-level refactoring to accommodate this, or if we should just keep the SDK mostly as-is and shove the sealevel stuff in the Kathy script
• Note key-wise, I don't 100% recall if kathy uses a hexKey or AWS KMS key, but sealevel requires hexKey. Take a look at what was done for the relayer to accommodate using AWS KMS keys for Ethereum & hexKey for sealevel
• Expectation is that Kathy can be ran in testnet3 (or mainnet2 too, if we have Sealevel deployments by the time work is being done here), and messages will be sent to & from sealevel chains

Tasks

Beta Give feedback

1. Minimal SDK support for sealevel #2587
   solana +1
   
2. https://github.com/hyperlane-xyz/issues/issues/558

[jmrossy]

Sealevel support proposal

Overview

We've so far taken a somewhat top-down approach to adding Sealevel support; the warp UI, hyp-deploy have been updated before the SDK and Utils packages. We can continue that trend with HelloWorld/Kathy or we can make a bigger upfront investment and redesign the SDK+Utils now.

Option 1: Shortest path

Kathy is built around HelloWorld, which is just a thin wrapper around the App/Deployer/Checker abstractions in the SDK. So updating just HelloWorld to be sealevel compatible wouldn't make sense. Sealevel support for Kathy would require a new, parallel HelloWorld version that live alongside the EVM one.

This option is still less work than Option 2 but feels short-sighted.

Option 2: Multi-env SDK

Updating the common utilities (e.g. address encoder, msg formatter) and SDK primitives (e.g. MultiProvider, App, Checker, Deployer) feels like a daunting task but also seems like the natural next step for Sealevel support.

I won't lay out a complete plan just yet until we have agreement on this direction but structural changes would include:

• Create multi-env compatible address/msg utilities (some of this is done in warp UI already)
• Redesign MultiProvider to handle different provider/connection libs
• Decouple all SDK primitives from Ethers v5 contracts/factories/providers
  • Separate implementation + interface for current classes
  • Build Sealevel equivalent impls
• Update/extend tests to cover Sealevel too (currently hardhat-based)

The new architecture would ideally be a plugin-like architecture with sub-packages. Something that allows SDK users to bundle only the libs for the environments they care about. Otherwise all EVM-only users would be unintentionally including in large libs for Sealevel (and soon Cosmos).

IMHO, this option is tricky but feasible.

[jmrossy]

Continuing on the last post above ^, we decided on Discord to try option 1.5. We will not modify existing classes in SDK but we will add new multi-protocol versions (e.g. MultiProtocolProvider) as needed to get a HelloWorld (and then Kathy) working.

So far I've got a Draft PR open with a new MultiProtocolProvider based partly on the one I made for Warp UI. This introduces a new set of Provider-related types and builders, which allow us to support different libraries (Viem, Ethers6, SolanaWeb3, etc.)

Next I took a stab at a MultiProtocolApp abstraction today but sadly the concept falls apart almost immediately 😭 . Our HyperlaneApp is a fancy map, from chain -> contracts. Sealevel's JS lib (solana/web3.js) doesn't have anything comparable to an Ethers/Viem Contract. Data must be manually (de)serialized with serializers custom defined for each contract. So scrap MultiProtocolApp I think.

Children Apps like RouterApp add some functionality like fetching ISMs or quoting gas. We'll def still want those features somewhere. Another step down, TokenApp adds a transfer method, i.e. a method to translate a user intent into a tx. Like @yorhodes pointed out last week, this is where the TokenApp overlaps with my ITokenAdapter in the Warp UI:

TokenApp = TokenAdapter + ChainMap<Addresses & Ethers Contracts>

I guess it will need to be something like: MultiProtocolTokenApp = TokenAdapter + ChainMap<Addresses & Desired provider types>, where the adapter part needs to do more heavy lifting than it does now for non-EVM chains.

[tkporter]

I deployed a helloworld program onto solanadevnet - here's the current state of things:

1. At the moment only fuji -> solanadevnet messages via the rc helloworld router contract are deliverable (i.e. from this one https://github.com/hyperlane-xyz/hyperlane-monorepo/blob/main/typescript/infra/config/environments/testnet3/helloworld/rc/addresses.json#L6). This is just because only fuji validators have been enrolled in the ISM that lives on solanadevnet for now. This is the address of the helloworld contract on solanadevnet https://explorer.solana.com/address/DdTMkk9nuqH5LnD56HLkPiKMV3yB3BNEYSQfgmJHa5i7?cluster=devnet (DdTMkk9nuqH5LnD56HLkPiKMV3yB3BNEYSQfgmJHa5i7 in base58, 0xbba2f483e642449d0a39efe5f9603f7c559423acebd3c854d07560ccd0439228 in hex).
2. You can find an example of a message from fuji I sent here https://testnet.snowtrace.io/tx/0x3ce33292790db41a4604b4744862acbf5df39bad995ad5ac3711442386c51833 with ID 0x5983bd101129f7c3d717ad2c7bdb81979cf076e8ace97c2096736a1778e33b0e that was delivered by this transaction https://explorer.solana.com/tx/vBTHYuhM2BrWQ7ivFFa5AChpsQQyst7CQNAFt1Ntc6ut77RCCVXusWibYFGUXBvScX2MDAoDMC7ooDseU5dss1j?cluster=devnet (https://explorer.solana.com/address/F2Qpm4DrYVoUg6HKT1foQvjcJSxoZqwUdbutcHSwicrQ?cluster=devnet is the relayer address that delivered the message)
3. Sending from other evm chains to solanadevnet atm will revert because the IGPs on those EVM chains haven't been configured with a gas oracle for solanadevnet.
4. Sending from solanadevnet to fuji works on the origin side (i.e. the Solana tx won't revert), but it won't be delivered atm because the solanadevnet validators haven't been enrolled on the ISM that fuji uses

Wanted to share this stuff ASAP though just to unblock you as much as possible

Here's info on how to get the # of sent or received messages and/or the remote routers:

1. They're stored in the "program storage PDA" of the helloworld program, which is found using these seeds

   hyperlane-monorepo/rust/sealevel/programs/helloworld/src/processor.rs

   Lines 44 to 59 in dd7ff72

   	macro_rules! program_storage_pda_seeds {
   	() => {{
   	&[b"hello_world", b"-", b"handle", b"-", b"storage"]
   	}};
   	($bump_seed:expr) => {{
   	&[
   	b"hello_world",
   	b"-",
   	b"handle",
   	b"-",
   	b"storage",
   	&[$bump_seed],
   	]
   	}};
   	}

   and the helloworld program ID
2. The account data layout can be found here

   hyperlane-monorepo/rust/sealevel/programs/helloworld/src/accounts.rs

   Lines 17 to 44 in dd7ff72

   	pub type HelloWorldStorageAccount = AccountData<HelloWorldStorage>;
   	/// The storage account's data.
   	#[derive(BorshSerialize, BorshDeserialize, Debug, Default)]
   	pub struct HelloWorldStorage {
   	/// The local domain.
   	pub local_domain: u32,
   	/// The mailbox.
   	pub mailbox: Pubkey,
   	/// The ISM.
   	pub ism: Option<Pubkey>,
   	/// The IGP.
   	pub igp: Option<(Pubkey, InterchainGasPaymasterType)>,
   	/// The owner.
   	pub owner: Option<Pubkey>,
   	/// A counter of how many messages have been sent from this contract.
   	pub sent: u64,
   	/// A counter of how many messages have been received by this contract.
   	pub received: u64,
   	/// Keyed by domain, a counter of how many messages that have been sent
   	/// from this contract to the domain.
   	pub sent_to: HashMap<u32, u64>,
   	/// Keyed by domain, a counter of how many messages that have been received
   	/// by this contract from the domain.
   	pub received_from: HashMap<u32, u64>,
   	/// Keyed by domain, the router for the remote domain.
   	pub routers: HashMap<u32, H256>,
   	}

   -- note this also has the first byte as the "initialized" flag like the warp routes do (i.e. pub type HelloWorldStorageAccount = AccountData<HelloWorldStorage>; is what's actually serialized and stored, which inserts that first byte prefix)

And to send a hello world message from solanadevnet:

This is the expected instruction data passed in:

hyperlane-monorepo/rust/sealevel/programs/helloworld/src/instruction.rs

Line 44 in dd7ff72

SendHelloWorld(HelloWorldMessage),

(note there is no 8-byte prefix here or anything, unlike the warp routes)

The expected accounts passed in can be found here

hyperlane-monorepo/rust/sealevel/programs/helloworld/src/processor.rs

Lines 157 to 174 in dd7ff72

	/// Accounts:
	/// 0. [writeable] Program storage.
	/// 1. [executable] The Mailbox program.
	/// 2. [writeable] Outbox PDA.
	/// 3. [] This program's dispatch authority.
	/// 4. [executable] System program.
	/// 5. [executable] SPL Noop program.
	/// 6. [signer] Payer.
	/// 7. [signer] Unique message account.
	/// 8. [writeable] Dispatched message PDA. An empty message PDA relating to the seeds
	/// `mailbox_dispatched_message_pda_seeds` where the message contents will be stored.
	/// ---- if an IGP is configured ----
	/// 9. [executable] The IGP program.
	/// 10. [writeable] The IGP program data.
	/// 11. [writeable] The gas payment PDA.
	/// 12. [] OPTIONAL - The Overhead IGP program, if the configured IGP is an Overhead IGP.
	/// 13. [writeable] The IGP account.
	/// ---- end if an IGP is configured ----

, for clarity I'll explain each one:

/// 0. [writeable] Program storage.
/// 1. [executable] The Mailbox program.
/// 2. [writeable] Outbox PDA.
/// 3. [] This program's dispatch authority.
/// 4. [executable] System program.
/// 5. [executable] SPL Noop program.
/// 6. [signer] Payer.
/// 7. [signer] Unique message account.
/// 8. [writeable] Dispatched message PDA. An empty message PDA relating to the seeds
/// mailbox_dispatched_message_pda_seeds where the message contents will be stored.
/// ---- if an IGP is configured ----
/// 9. [executable] The IGP program.
/// 10. [writeable] The IGP program data.
/// 11. [writeable] The gas payment PDA.
/// 12. [] OPTIONAL - The Overhead IGP program, if the configured IGP is an Overhead IGP.
/// 13. [writeable] The IGP account.
/// ---- end if an IGP is configured ----

0. [writeable] Program storage. - Same as described above for getting stats or routers - the PDA relating to these seeds

   hyperlane-monorepo/rust/sealevel/programs/helloworld/src/processor.rs

   Lines 44 to 59 in dd7ff72

   	macro_rules! program_storage_pda_seeds {
   	() => {{
   	&[b"hello_world", b"-", b"handle", b"-", b"storage"]
   	}};
   	($bump_seed:expr) => {{
   	&[
   	b"hello_world",
   	b"-",
   	b"handle",
   	b"-",
   	b"storage",
   	&[$bump_seed],
   	]
   	}};
   	}

   and the helloworld program ID
1. [executable] The Mailbox program. - The mailbox program ID
2. [writeable] Outbox PDA. - The mailbox's outbox PDA
3. [] This program's dispatch authority. - The helloworld program's dispatch authority. I.e. the PDA relating to these seeds https://github.com/hyperlane-xyz/hyperlane-monorepo/blob/trevor/helloworld-program/rust/sealevel/programs/mailbox/src/pda_seeds.rs#L55-L68 and the helloworld program ID.
4. [executable] System program. - the system program id
5. [executable] SPL Noop program. - the noop program id
6. [signer] Payer. - the account that is paying for the tx
7. [signer] Unique message account. - the unique message account - i.e. the random keypair that needs to be created, just like in warp routes
8. [writeable] Dispatched message PDA - the PDA relating to the dispatched message, i.e. from these seeds https://github.com/hyperlane-xyz/hyperlane-monorepo/blob/trevor/helloworld-program/rust/sealevel/programs/mailbox/src/pda_seeds.rs#L30-L51, the "unique message account"'s public key passed in there, and the mailbox program ID. Just like in warp routes
9. This thru account 13 relate to paying for gas and aren't required if the program isn't paying for gas. For now, the deployed helloworld contract doesn't pay for gas (i.e. no IGP is configured on it). I'll be re-deploying things along with an IGP (no IGP has been deployed on solanadevnet yet) soon, at which point I'll move helloworld over to paying for gas. For now I think it's best to just not worry about these accounts and then when I have an IGP deployment up and ready I'll write up some info on the required changes to the warp route UI & this. For helloworld, the only change will be adding some accounts, & for warp routes there will be some new accounts and a new data layout to the HyperlaneToken type

[jmrossy]

Kathy updates for sealevel are now live!
We'll monitor Kathy's health over the next few days to confirm everything is okay