refactor(experimental): add cluster level API for transports

[buffalojoec]

This change introduces cluster-level transports as well as cluster-level JSON RPCs
that infer the cluster from the provided transport.

[buffalojoec]

Current dependencies on/for this PR:

• master
  • PR refactor(experimental): add ClusterUrl types #2084
    • PR refactor(experimental): move transport types out of rpc-transport #2085
    • PR refactor(experimental): add cluster level API for transports #2053 👈
      • PR refactor(experimental): add cluster level API for rpc-core #2054
        • PR refactor(experimental): add cluster level API for library #2055
          • PR refactor(experimental): move web socket transport types out of rpc-transport #2086
          • PR refactor(experimental): add cluster level subscriptions API for transports #2056
            • PR refactor(experimental): add cluster level subscriptions API for library #2058
            • PR refactor(experimental): add cluster level subscriptions API for rpc-core #2057

This stack of pull requests is managed by Graphite.

[lorisleiva]

I like this stack a lot. I love that only with types, we can cherry-pick the RPC methods available in different clusters.

The only API change I want to put on the table is, instead of requesting a second TCluster type parameter which makes it tricky when you don't need to offer the first one, why don't we infer that type from the config object provided. IIRC @steveluscher mentioned something like this a while ago. For instance, we could attach that piece of information on the URL directly.

type MainnetUrl = string & { '~cluster': 'mainnet' }
type DevnetUrl = string & { '~cluster': 'devnet' }
type TestnetUrl = string & { '~cluster': 'testnet' }
type ClusterUrl = string | MainnetUrl | DevnetUrl | TestnetUrl

function mainnet(putativeString: string): MainnetUrl { return putativeString as MainnetUrl }
function devnet(putativeString: string): DevnetUrl { return putativeString as DevnetUrl }
function testnet(putativeString: string): TestnetUrl { return putativeString as TestnetUrl }

// Devnet example.
const devnetTransport = createDefaultRpcTransport({ url: devnet('https://api.devnet.solana.com') });
createSolanaRpc({ transport: devnetTransport });
//  ^ RpcDevnet<DevnetSolanaRpcMethods> 

// Generic example.
const genericTransport = createDefaultRpcTransport({ url: 'http://127.0.0.1:8899' });
createSolanaRpc({ transport: genericTransport });
//  ^ Rpc<SolanaRpcMethods> 

Wdyt?

[buffalojoec]

The only API change I want to put on the table is, instead of requesting a second TCluster type parameter which makes it tricky when you don't need to offer the first one, why don't we infer that type from the config object provided.

Interesting. Just thinking about your suggestion out loud here:

For just @solana/rpc-transport, we have three methods:

• createHttpTransport(..): This one takes a url argument and can probably be refactored to use your approach. I actually left this one alone intentionally.
• createJsonRpcApi<TMethods>(..): This one requires the API generic, and we can optionally pass TCluster after it. The problem is that without TCluster, this function doesn't care what the cluster is. There's no url parameter here.
• createJsonRpc(.): This one does not take a generic, and we infer the cluster from it's config arg.

In either case, now that I look at it some more, when it comes to the main library in @solana/web3.js, I definitely can refactor the createDefaultRpcTransport(..) to apply the cluster argument there and eliminate the need for it - in either form - on createSolanaRpc(..).

There is one problem here. If we put the cluster configuration on the URL as provided to transport, there's no way to also get it inside of createJsonRpcApi(..), since it only takes the transformers as arguments. We'd still need to provide it there as TCluster, and then we'd have a diverging API.

Wdyt @lorisleiva ?

P.S. I edited/re-wrote this after checking out the code some more. These commits are a tad old for me, had to refresh.

[lorisleiva]

What if we don't tag the API methods with a cluster though? What if we just have various API methods types available like SolanaRpcMethodsMainnet, SolanaRpcMethodsDevnet that includes requestAirdrop, etc.

You can see them as various libraries of RPC methods available in different clusters.

Then we select the one we need based on the given transport URL when we create the RPC (transport + api). Is this making any sense? 😅

[buffalojoec]

What if we don't tag the API methods with a cluster though? What if we just have various API methods types available like SolanaRpcMethodsMainnet, SolanaRpcMethodsDevnet that includes requestAirdrop, etc.

You can see them as various libraries of RPC methods available in different clusters.

Then we select the one we need based on the given transport URL when we create the RPC (transport + api). Is this making any sense? 😅

I had them isolated into per-cluster sets when I originally created this stack (or one before it I forget). Is this what you're thinking?

createJsonRpc({
    api: createJsonRpcApi<SolanaRpcMethodsDevnet>(),
    transport: createHttpTransport({ url: devnet('https://api.devnet.solana.com') }),
}); // RpcDevnet<SolanaRpcMethodsDevnet>

createJsonRpc({
    api: createJsonRpcApi<SolanaRpcMethodsTestnet>(),
    transport: createHttpTransport({ url: testnet('https://api.testnet.solana.com') }),
}); // RpcTestnet<SolanaRpcMethodsTestnet>

createJsonRpc({
    api: createJsonRpcApi<SolanaRpcMethodsMainnet>(),
    transport: createHttpTransport({ url: mainnet('https://api.mainnet-beta.solana.com') }),
}); // RpcMainnet<SolanaRpcMethodsMainnet>

[lorisleiva]

Yes, exactly that. Then it's the responsibility of createSolanaRpc to provide the right default api based on the provided transport. Was there any issues with this approach?

[buffalojoec]

Yes, exactly that. Then it's the responsibility of createSolanaRpc to provide the right default api based on the provided transport. Was there any issues with this approach?

No I think that will work fine. Anyone who creates their own RPC client from our libraries would just have to make sure they align the proper API based on cluster as well, similar to what I just did above.

For our Solana-implemented helpers, we can wrap that.

[buffalojoec]

I went with overloads here because we've designed this API for users to provide their RPC methods. It can be removed, but this function would then take 2 generic arguments, and the moment you provide your RPC methods it will complain you didn't provide TConfig as well.

[lorisleiva]

Love it!

[buffalojoec]

Merge activity

• Feb 2, 11:35 AM EST: @buffalojoec started a stack merge that includes this pull request via Graphite.
• Feb 2, 11:37 AM EST: Graphite rebased this pull request as part of a merge.
• Feb 2, 11:38 AM EST: @buffalojoec merged this pull request with Graphite.

[github-actions]

🎉 This PR is included in version 1.90.0 🎉

The release is available on:

• npm package (@latest dist-tag)
• GitHub release

Your semantic-release bot 📦🚀

[github-actions]

Because there has been no activity on this PR for 14 days since it was merged, it has been automatically locked. Please open a new issue if it requires a follow up.