Brainstorming ideas to make the Rust and Web Client super easy and intuitive to use

[partylikeits1983]

I am starting this discussion to brainstorm ideas on how to make the Miden client as easy to use as possible.

The Miden client is one of the most critical pieces of the Miden protocol. This is because it is most likely the first Miden repository any new application / smart contract developer coming to Miden will interact with. This means making it as simple and intuitive to use is critical to make developers have a good first experience building applications on Miden.

Many experienced smart contract developers who are experts in Solidity or TypeScript may be less comfortable building with Rust. While we should support experienced Rust developers, we shouldn't exclude smart contract developers from other ecosystems who aren't as familiar with Rust-based smart contract development. This means that the Client should have opinionated sane defaults, but keep the ability for flexibility.

Last week we had an internal hackathon to build a basic application on Miden which led to finding a lot of potential areas of improvement. This is by no means a criticism of the client, this is just to start a brainstorm discussion on how to make the Client super easy to use.

Concretely, these are some things I think would make the Miden Rust client even easier to use. These are in no specific order:

---

1) Make Client initialization even easier

PR #741 made initialization easier, however, there are still improvements to be made.

Currently this is the easiest way to initialize the client:

let mut client = ClientBuilder::new()
    .with_rpc(rpc_api)
    .with_filesystem_keystore("./keystore")
    .in_debug_mode(true)
    .build()
    .await?;

This is unintuitive, because developers want to initialize the client via Client::new(). I also think that we should have sane defaults for .with_rpc() and .with_filesystem_keystore().

The .with_rpc() method should be able to just take a string like this:
with_rpc("https://rpc.testnet.miden.io")

Also, the .with_filesystem_keystore() method should default to using "./keystore" as the default path, which just creates a /keystore directory at the root of the project.

I think the end goal of client initialization should look something like this:

let mut client = Client::new().with_testnet_rpc().with_debug_mode();

or:

let mut client = Client::new().with_mainnet_rpc();

or:

let mut client = Client::new().with_endpoint("https://rpc.testnet.miden.io");

Using Client::new() is much more intuitive as opposed to ClientBuilder::new().some_chain_of_methods().

---

2) Use less Rust generics!

I think this might be a bit of a contentious issue / hot take, but I think the Miden Rust client should be more opinionated. This is because Miden is already difficult to use, and by forcing developers to use "sane defaults" will help guide them. This doesn't mean to remove flexibility of the Miden client, but instead to help guide new developers. I think Miden Assembly is more or less easy to use, but it's the setup required to run MASM code which is difficult.

---

3) Remove the need for feature flags for default use of miden client

Developers should be able to run cargo add miden-client and then in their src/main.rs file have it look like this:

use miden_client::{Client, ClientError};

#[tokio::main]
async fn main() -> Result<(), ClientError> {
    let mut client = Client::new()
        .with_testnet_rpc()
        .with_debug_mode();

    let sync_summary = client.sync_state().await?;
    println!("Latest block: {}", sync_summary.block_num);

    Ok(())
}

The cargo.toml file would look like this with no feature flags turned on for the Client.

[package]
name = "testest"
version = "0.1.0"
edition = "2024"

[dependencies]
miden-client = "0.8.1"
tokio = "1.44.2"

Currently this is how the client import looks like in a basic Miden project:

miden-client = { version = "0.8.1", features = ["testing", "concurrent", "tonic", "sqlite"] }

---

4) Better error message handling from the node and client

When submitting a transaction via the Client to the node which the node reverts, this error message is not returned to the Client.

The method submit_transaction() should return a type that has the status / error message associated with why the transaction reverted on the node:

This code does not fail, and returns a successful result even if the node rejects the tx:

let _ = client.submit_transaction(tx_result).await;

Here is a concrete example of a transaction that is successful, but the node rejects:
https://github.com/partylikeits1983/miden-clob-designs/blob/84d8e52d88a4659daaaa63e1db8d51100711ef6f/tests/account_sig_check_update.rs#L195

Conclusion and final thoughts

The Miden client is an amazing technical feat and is an extremely high quality Rust repository. However, there room for improvement to lower the bar to entry for developers getting started with Miden.

The Miden rust Client will most likely become an integral part of the test harness that the compiler team @bitwalker and @greenhat are working on. Additionally the Miden node could / should be integrated into the test harness to be able to spin up a lightweight test blockchain where users can run integration tests.

Please feel free to add any thoughts / ideas of ways to make the client easier and more intuitive to use.

[PhilippGackstatter]

One quick comment I have is with regards to in_debug_mode(true);. I think this should just be false by default in the client and when calling with_debug_mode() it sets it to true, but the boolean argument is not exposed at the API level. I find bool arguments bad for readability in general, and would usually replace them with enums, but here I think it is not necessary in the first place to have that.
Another quick way to set it would be to have Client::new be a non-debug client and Client::new_debug be a debug-mode client, pretty much analogously to miden_processor::Process::{new, new_debug}.

[igamigo]

I agree that bool parameters are bad for readability, though I'd say in these cases where it's fairly easy to parse in the context of the function call, it's not as bad. I think the builder right now has a function to set it, and Client::new() takes it as a parameter. This should be consistent with all other client fields that are also passed as parameters and the builder has a function to set it separately, with false as default.

In code where the setting has to flow to the client constructor, I think using a variable can be a bit more ergonomic. Otherwise, you need to check whether the variable is true and change the client constructor or client builder accordingly:

let mut client_builder = ClientBuilder::new();
let debug_mode: bool = env::var("MIDEN_DEBUG").parse()?;

if debug_mode {
    client_builder.with_debug_mode();
} 
let client = client_builder.build().await?;

as opposed to

let debug_mode: bool = env::var("MIDEN_DEBUG").parse()?;
let client = ClientBuilder::new().in_debug_mode(debug_mode).build().await?;

(something like this occurs in the CLI code)

For Client::new() and Client::new_debug(), something similar would apply.
But this is also quite a minor detail and easy to get around so I wouldn't mind using either approach.

If we were to keep the current way though, we could definitely use an enum.

[PhilippGackstatter]

Makes sense. In this case, it's a bit tricky to design for both human and programmable consumption of APIs. I think a custom enum might be ideal from my POV.

[igamigo]

I agree there is room for making the client much more accessible in various ways. I also think focusing on user feedback will provide the most value here, so thanks for opening the discussion and providing some ideas!

I'll try to add some comments about specific points and then provide some other ideas (probably in a different comment).

1) Make Client initialization even easier

PR #741 made initialization easier, however, there are still improvements to be made.

Currently, this is the easiest way to initialize the client:

let mut client = ClientBuilder::new()
    .with_rpc(rpc_api)
    .with_filesystem_keystore("./keystore")
    .in_debug_mode(true)
    .build()
    .await?;

This is unintuitive, because developers want to initialize the client via Client::new(). I also think that we should have sane defaults for .with_rpc() and .with_filesystem_keystore().

At this point (as of #782), actually the simplest way to build the client in an std context should be:

let mut client = ClientBuilder::new()
   .build()     
   .await?;

The .with_rpc() method should be able to just take a string like this: `with_rpc("https://rpc.testnet.miden.io")

I think we also have a specific with_tonic_rpc(url, timeout) that should address this.

Also, the .with_filesystem_keystore() method should default to using "./keystore" as the default path, which just creates a /keystore directory at the root of the project.

I think I'd prefer it as it is right now (passing the directory), but not a strong opinion at all so I wouldn't mind changing the default.

Using Client::new() is much more intuitive as opposed to ClientBuilder::new().some_chain_of_methods().

I agree with the sentiment here, there's room to polish the structures and usability.

The defaults you are proposing are based on the std-compatible structures (basically sqlite and the std tonic client), but we also want to keep it general enough to make it no-std compatible. I think there are some things we could do to abstract away some of the context out of these structs, but originally the idea of the client was to make it not very opinionated (as opposed to other projects such as the CLI), so that more specific tools could be built over it. Also, the more functionality we isolate behind feature flags, the more complicated it is to provide good defaults behind a single piece of code, since they have to depend on what's available at compile time.

2) Use less Rust generics!

I think this might be a bit of a contentious issue / hot take, but I think the Miden Rust client should be more opinionated. This is because Miden is already difficult to use, and by forcing developers to use "sane defaults" will help guide them. This doesn't mean to remove flexibility of the Miden client, but instead to help guide new developers. I think Miden Assembly is more or less easy to use, but it's the setup required to run MASM code which is difficult.

We use trait objects for the main client components rather than generics, which is more a consequence of the client not being too opinionated as you imply and as mentioned above. Maybe there is room to make something on top of the client that provides better defaults and be a better option for higher-level uses? Or maybe we could make it more opinionated too. There is also the work for making more components that can be taken and used in a stand-alone manner (though while related, it's also independent of this argument).

3) Remove the need for feature flags for default use of miden client

This also feels very related to the above points.
I also believe we could improve features overall, but with the idea of being able to separate some functionality and components (especially for different environments), it's also nice to be able to have the option to not include and compile a whole bunch of code of which you know you won't make use.
One simple improvement that I've kept in mind for some time is that we could include the std-related dependencies with the set of default features to make it a decent amount more accessible at the expense of making it just a bit more opinionated (curious to know what @bobbinth thinks for changes like these).

4) Better error message handling from the node and client

When submitting a transaction via the Client to the node which the node reverts, this error message is not returned to the Client.

The method submit_transaction() should return a type that has the status / error message associated with why the transaction reverted on the node.

I think this is more a consequence of the lifecycle of a transaction in the node/protocol. Unless basic validations fail, the transaction is accepted at the RPC level (so as far as the client goes, the submission went well) and sent to the mempool so that the block producer can eventually attempt to include it in a block. Up until that point, some things could happen that could revert the transaction (expired transaction or batch, another transaction using the same input notes just before, etc.). I believe the RPC endpoint should return an error if something is wrong when submitting the proof (if not, we should take a look). Though this only involves verifying the proof itself, as far as I remember. With more recent and not so recent changes, we are adding support for discarded transactions with a way of identifying why transactions were reverted, but these updates can only be received as part of the SyncSummary by progressively syncing with the network instead of at the moment of submitting the transaction. For instance, the client can deduce that a transaction was reverted due to double spend if an input note was consumed by a different client or if the chain is past the transaction expiry block.

Please feel free to add any thoughts / ideas of ways to make the client easier and more intuitive to use.

At the core of mostof these arguments is how we should balance some attributes of the client while trying to make it much easier and intuitive to use. In many environments, there is this pattern of providing functionality which splits SDKs or APIs into two different packages or interfaces: a more "native" one and a higher level one which is more opinionated, maybe less configurable and less powerful that wraps around the native package and makes it easier to access a system for entry-level users. The CLI sort of works as the more opinionated wrapper right now, but from a library perspective maybe we could do something more toward that model. It wouldn't have to be a different crate necessarily but maybe a better defined set of functionality which could be used as the default most of the time.

[bobbinth]

I think one of the reasons for the complexity is that we use the same client for both WASM and std targets. The current thinking is that there is enough of the shared logic that having Client used for both of these use cases is beneficial - but if we decide that it is causing more issues than it is solving, I think we can split them up and say that Client can be used only in std context and something like WasmClient would be used for WASM context. Maybe once we "break" the current client into a set of "lego-like" components, this would not be too difficult to do.

Make Client initialization even easier

As things stand now, I think having a ClientBuilder is still probably the way to simplify client construction. We could even have two version of it - one for std context (which I believe we already have) and another one for WASM context. The builders can be pretty opinionated and have a good set of defaults. This could be a good way to have a flexible Client but at the same time have a simple way of instantiating it. I'd create a separate issue about how the current ClientBuilder can be improved.

Use less Rust generics!

As @igamigo mentioned, the current client does not require any generic types - but it does use trait objects. I think the way to abstract these away from the user is through the ClientBuilder. We could even have multiple builders that assume different set of defaults - if this simplifies things.

Remove the need for feature flags for default use of miden client

I think here we can probably re-visit how we define the default features. For example, std feature could probably imply tonic, sqlite, and concurrent feature. If the user doesn't need these, they can start with no-std state and add the required features manually.

I wouldn't however, make testing a default feature, and actually not sure if we'd want the users of the crate to enable it at all (what are some reasons why external users of the crate may want to use the testing feature?).

Better error message handling from the node and client

Totally agree! Though, as @igamigo mentioned above, there are some limitations as to what can be done as all transactions are "pending" until they are included in a block. One of the ways to provide a better UX here is to set a default expiration time for transactions to something like 4 blocks (currently ~20 seconds). This way, if a transaction did not appear on chain within 4 blocks, the client can mark it as failed and update the state appropriately. Though, in such cases, it still won't be easy to tell exactly why the transaction didn't make it in.

[tomyrd]

Given what was said here, a possible way to deal with "Make Client initialization even easier" and "Remove the need for feature flags for default use of miden client" would be to separate the web components (right now they would be the WebStore and TonicRpcClient with the web-tonic feature) to a separate crate.

We could ship miden_client with just SqliteStore and the standard TonicRpcClient (while keeping the Client no-std) so that these function as standard/default implementations and a separate miden_wasm_client crate with other types that implement the traits for wasm.

This way we can remove features altogether, and the builder can just use the default components. The new miden_wasm_client crate can use the bare bone new method and implement its own builder if we want.

[bobbinth]

Please feel free to add any thoughts / ideas of ways to make the client easier and more intuitive to use.

One potential conceptual simplification we can make is to have Client manage a single account. Then, "higher-level" components (e.g., CLI, wallet) would use multiple clients, if needed, for situations when a single user needs to work with multiple accounts. My intuition is that this would make the "base" client much simpler and would also make it less ambiguous as to how to use the client when the users want to use it for building apps.

[tomyrd]

Right now the CLI works both as a tool that simplifies the interaction with the Miden network and also as an example use case for the client. I think it works well for the former but it is not as useful for the latter as it is more of a wrapper for the client's functionality.

We could have more standalone examples of use cases for the client. Small, well documented programs that explore the client's functionality. Similar to the integration tests but better documented, and maybe with some real world use cases and not just for testing purposes.
These could showcase the different client's features and serve as a complement to the documentation. New developers can use these examples to learn by example and use them as a base for their own, more elaborate, ideas/projects.

One of the cons with this is that the client's interface is not stable right now so these would maybe need to change a lot in each PR (similar to what happens with the integration tests). We could make it so that these examples are only updated for each stable version.

[partylikeits1983]

We could have more standalone examples of use cases for the client.

I've been writing tutorials for how to use the client, in the tutorials repository there are lots of examples of how to use the rust client.
https://github.com/0xMiden/miden-tutorials/tree/main/rust-client/src/bin

We could add some of these programs as integration tests to the client if there is value in that. Or, we could refactor the tutorial repository to not have any code at all, and the Rust & Web client tutorial code would be as examples in the client repository, but this is just an idea.

[igamigo]

Posting here some takeaways from the recent discussion mixed with my two cents:

• It's hard to keep parity between the web SDK and the Rust client. One of the reasons for this is that a lot of the client relies on the interaction between many structs, some here and some from miden-base. Each of those structs requires a Javascript binding and this comes with its own set of limitations. Additionally, most of the app development will go through the web SDK. For dapp and web developers, it makes sense to hide as much abstraction as possible so that the main primitives are boiled down to build,execute and not too much more than that.
  I've mentioned a couple times before (here and here) that I've thought of the Rust client as the more "low-level" way of interacting with the protocol, and that while ideally it would be nice to have the flexibility of the Rust client in a web library, there is a lot of room for higher level libraries (in both Rust and JS) as well. So in general I agree with this approach. In this regard also, I agree with the fact that one important aspect is being able to express and interpret TransactionRequest (or something that translates to/from that) that the client's executor can take and execute.
• Originally, the Rust client was conceived as a non-opinionated library geared toward building wallet-like applications whereas the CLI would be more opinionated and a proof of concept for an actual wallet. We should look into making it more development/app-oriented. Considerations and conclusions from the midenup discussions should come into play here.
• Some common examples or tests (counter contract/FPI calls) seem complex at first sight.
  Here, I'd probably disagree in that there is too much complexity involved. However, I definitely agree in that there is a lot of boilerplate involved though (one of the tests involves setting up a nested FPI scenario, which is already not trivial to express, but then you also need to create the custom accounts, deploy them, etc.). In reality, most of the complexity of actually doing the FPI transaction is hidden behind actually writing the transaction scrip, compiling it and then creating a TransactionRequest to execute it with the account's state involved. I'd also say that looking at tests is probably not a great idea overall in order to get a feel for how a library should be used, and should not drive discoverability at all. They are not designed to not be verbose nor describe real-world scenarios necessarily, they include some testing-only helpers, etc., so the signal/noise ratio is very low already. Instead, it should be more driven by real use-cases, tutorials, docs, etc. In this sense, I also agree with the idea that lines of code is a decent metric to look to minimize while keeping a good level of expressivity for these real life scenarios.

cc @bobbinth @Dominik1999 @evanmarshall @dagarcia7 @krwang

[maksimryndin]

If we decide to use OPFS, we can re-use the logic of the Rust client with SqliteStore #1008.

Also probably a radical idea but may worth it - making the rust client sync instead of async (in a similar vein to https://github.com/algesten/ureq):

• simplification of the code and alignment with the fundamental thing that sqlite access is sync
• it would simplify handling issues like Implement SQLite-based MastForestStore #977 (comment)
• easier FFI interoperability - vision - rust client as a common layer for bindings in other languages (web sdk now, Python etc in the future)
• it would cut many async dependencies
• also some operations are CPU bound
• the tutorials highlight the usage pattern - it is sequential
• Miden CLI doesn't need async actually

So in the base Rust client we make minimal assumptions about future runtime - the language runtime can then make it async via threads, pooling etc (e.g. web would run the code in a dedicated worker with message passing with the main thread)

@igamigo @tomyrd guys, what do you think also about storing keys in the sqlite for uniformity?

[tomyrd]

what do you think also about storing keys in the sqlite for uniformity?

We actually did, but it was removed in #718. The idea was that we wanted the key storage to be independent of the Client so that users could have the ability of implementing their own ways of storing/accessing keys without the need of changing the Store.