Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The runtime fails to provide users sufficient feedback to investigate issues on their own #7223

Closed
silvestrpredko opened this issue Jul 20, 2022 · 14 comments
Labels
T-contract-runtime Team: issues relevant to the contract runtime team

Comments

@silvestrpredko
Copy link

Describe the bug
Added library to a smart contract that is compatible with wasm32-unknown-unknown.
Deploy it to the blockchain and call a single method. Virtual machine initialization failed with a FunctionCallError(CompilationError(PrepareError(Instantiate)))

To Reproduce

// lib.rs
use near_sdk::{
    borsh::{self, BorshDeserialize, BorshSerialize},
};

use vodozemac::megolm::{GroupSession, InboundGroupSession, SessionKey};
#[near_bindgen]
#[derive(BorshDeserialize, BorshSerialize, Default)]
pub struct Key {}

#[near_bindgen]
impl Key {
    pub fn key(&self) -> String {
        "Name".to_owned()
    }
}
# Cargo.toml

[package]
name = "key"
version = "0.1.0"
edition = "2021"

[lib]
crate-type = ["cdylib", "rlib"]

[dependencies]
near-sdk = "4.0.0"
near-contract-standards = "4.0.0"
near-units = "0.2.0"
serde = { version = "1", features = ["derive"] }
vodozemac = { version = "0.2.0", features = ["js"] }

Deploy a smart contract and call a key function.

near view <accountId> key

Output: wasm execution failed with error: FunctionCallError(CompilationError(PrepareError(Instantiate)))

Expected behavior
At least tell where exactly the error happens, which part of the used libraries is not compatible

Version (please complete the following information):

  • rust version 1.62.1
  • testnet
@akhi3030
Copy link
Collaborator

CC: @austinabell , @nagisa . Maybe one of you has some hints here?

@nagisa
Copy link
Collaborator

nagisa commented Jul 20, 2022

Last time I had to investigate something similar, I ended up looking at the wasm itself, in some instances minimizing it using wasm-shrink. There are currently two reasons why PrepareError::Instantiate error could be raised:

  1. Module imports names that are not in the allow-list of defined host functions (env namespace is the only permitted);
  2. Module is malformed and some of the imports refer to non-existent type indices.

If you were to post the failing .wasm contract, I’d be happy to take a look at it.

The error messages are indeed terrible, and I would love to see them improved. Unfortunately, doing so is going to be somewhat non-trivial – the current errors are set up in a way that enables development of the underlying code without accidental reproducibility breaks. This is primarily achieved by making errors not necessarily super straightforward to interpret.

Having some better developer tooling that could aid in investigating these sorts of problems is definitely on the roadmap, but the priority on that work is tad a bit lower than some other tasks at the moment, so it probably isn’t going to materialize very soon, unfortunately.

@austinabell
Copy link
Contributor

CC: @austinabell , @nagisa . Maybe one of you has some hints here?

My assumption is that rand 0.7 with wasm-bindgen feature (what is enabled by vodozemac) uses some host function exposed when using in a JS context but not available to our runtime. My assumption would be that it's how it's pulling in its entropy for the randomness you are using, and it seems like it's pulling in getrandom which I believe is incompatible because of what they call into.

Seems as if pulling the js feature of vodozemac might be, but not limited to, the issue.

@akhi3030
Copy link
Collaborator

@nagisa: I wonder if it would be possible to offer some sort of stand-alone wasm validator. Something that a contract developer could use to validate that their wasm would indeed pass the checks if uploaded to the network. Does something like this make sense?

@nagisa
Copy link
Collaborator

nagisa commented Jul 20, 2022

Yes, it would make sense and be possible to implement, though at that point it is half-way there to re-introducing the standalone wasm runner binary which we just removed. I know I definitely would love a tool like this personally, as obtaining post-preparation WASM, and working around that area of code base in general, is quite painful today.

I’d see that sort of work ideally as a sub-task in implementing a larger debugging and introspection tool that I’ve been briefly mentioning at some points in the past. But we might end up having no choice if the pain points end up being too unbearable…

@Ekleog
Copy link
Contributor

Ekleog commented Jul 20, 2022

For this specific instance, for "offering a standalone wasm validator" I think the same solution as the one I suggested in #7220 (comment) might work? As just having additional logs would probably be enough to know why the validation is failing.

@nagisa
Copy link
Collaborator

nagisa commented Jul 20, 2022

More observability is usually never a bad thing, but devex wise it is not really a tenable long term solution I feel.

@akhi3030
Copy link
Collaborator

@silvestrpredko : Based on the discussions above, I wonder in this specific case, if you can reproduce the problem on localnet. If you are able to, you can add some printfs around where we throw the specific error and then you can figure out what is going wrong.

@akhi3030 akhi3030 added the T-contract-runtime Team: issues relevant to the contract runtime team label Jul 21, 2022
@silvestrpredko
Copy link
Author

Last time I had to investigate something similar, I ended up looking at the wasm itself, in some instances minimizing it using wasm-shrink. There are currently two reasons why PrepareError::Instantiate error could be raised:

  1. Module imports names that are not in the allow-list of defined host functions (env namespace is the only permitted);
  2. Module is malformed and some of the imports refer to non-existent type indices.

If you were to post the failing .wasm contract, I’d be happy to take a look at it.

The error messages are indeed terrible, and I would love to see them improved. Unfortunately, doing so is going to be somewhat non-trivial – the current errors are set up in a way that enables development of the underlying code without accidental reproducibility breaks. This is primarily achieved by making errors not necessarily super straightforward to interpret.

Having some better developer tooling that could aid in investigating these sorts of problems is definitely on the roadmap, but the priority on that work is tad a bit lower than some other tasks at the moment, so it probably isn’t going to materialize very soon, unfortunately.

Hi, thank you for your support. Here is a smart contract that is failing key.wasm.zip Also do you have a specification of NEAR runtime and how does it differs from a JS runtime, what is allowed and what is not. I know that networking isn't allowed, but I was surprised with rand stuff.

@silvestrpredko
Copy link
Author

@silvestrpredko : Based on the discussions above, I wonder in this specific case, if you can reproduce the problem on localnet. If you are able to, you can add some printfs around where we throw the specific error and then you can figure out what is going wrong.

Currently, workspaces not working for me(( M1 problem. Or do you mean something else under localnet?

@nagisa
Copy link
Collaborator

nagisa commented Jul 21, 2022

Also do you have a specification of NEAR runtime and how does it differs from a JS runtime, what is allowed and what is not. I know that networking isn't allowed, but I was surprised with rand stuff.

The various APIs exposed to the NEAR contract are specified in this section of the specification. In JavaScript world the wasm code is allowed to set up and expose whatever APIs they wish. AFAIK usually developers will run their code through tools like wasm-bindgen and wasm-pack to set up these APIs for them automatically.


Let us run your contract through the wasm2wat tool first, to get a readable representation of the code. Looking at the imports we can see these:

imports
  (import "__wbindgen_placeholder__" "__wbindgen_is_undefined" (func $_ZN12wasm_bindgen23__wbindgen_is_undefined17hd5096780d189d788E (type 1)))
  (import "__wbindgen_placeholder__" "__wbg_self_86b4b13392c7af56" (func $_ZN9getrandom3imp6Global8get_self27__wbg_self_86b4b13392c7af5617h3ab47533c76ae43aE (type 13)))
  (import "__wbindgen_placeholder__" "__wbg_crypto_b8c92eaac23d0d80" (func $_ZN9getrandom3imp5Self_6crypto29__wbg_crypto_b8c92eaac23d0d8017hff605b69df2c41a9E (type 1)))
  (import "__wbindgen_placeholder__" "__wbg_msCrypto_9ad6677321a08dd8" (func $_ZN9getrandom3imp5Self_9ms_crypto31__wbg_msCrypto_9ad6677321a08dd817hf3f8f5dc84af062aE (type 1)))
  (import "__wbindgen_placeholder__" "__wbg_require_f5521a5b85ad2542" (func $_ZN9getrandom3imp10NodeModule7require30__wbg_require_f5521a5b85ad254217hea14bb6263c0c6daE (type 11)))
  (import "__wbindgen_placeholder__" "__wbg_getRandomValues_dd27e6b0652b3236" (func $_ZN9getrandom3imp13BrowserCrypto20get_random_values_fn38__wbg_getRandomValues_dd27e6b0652b323617hfe4b774914f6f3b7E (type 1)))
  (import "__wbindgen_placeholder__" "__wbg_randomFillSync_d2ba53160aec6aba" (func $_ZN9getrandom3imp10NodeCrypto16random_fill_sync37__wbg_randomFillSync_d2ba53160aec6aba17h9abba5b30cc1bb43E (type 3)))
  (import "__wbindgen_placeholder__" "__wbg_getRandomValues_e57c9b75ddead065" (func $_ZN9getrandom3imp13BrowserCrypto17get_random_values38__wbg_getRandomValues_e57c9b75ddead06517h286726fd8a6c66b9E (type 10)))
  (import "__wbindgen_placeholder__" "__wbg_static_accessor_MODULE_452b4680e8614c81" (func $_ZN9getrandom3imp6MODULE4init45__wbg_static_accessor_MODULE_452b4680e8614c8117h15b30e327ca0f032E (type 13)))
  (import "__wbindgen_placeholder__" "__wbindgen_object_drop_ref" (func $_ZN12wasm_bindgen26__wbindgen_object_drop_ref17h2d0c764e483bffa9E (type 0)))
  (import "__wbindgen_placeholder__" "__wbindgen_describe" (func $_ZN12wasm_bindgen19__wbindgen_describe17h1d5dffb576c62f5bE (type 0)))
  (import "__wbindgen_placeholder__" "__wbg_buffer_de1150f91b23aa89" (func $_ZN6js_sys11WebAssembly6Memory6buffer29__wbg_buffer_de1150f91b23aa8917h3aa70f3a6bcb1421E (type 1)))
  (import "__wbindgen_placeholder__" "__wbg_new_97cf52648830a70d" (func $_ZN6js_sys10Uint8Array3new26__wbg_new_97cf52648830a70d17h4241b996df6c6d6eE (type 1)))
  (import "__wbindgen_placeholder__" "__wbg_newwithlength_e833b89f9db02732" (func $_ZN6js_sys10Uint8Array15new_with_length36__wbg_newwithlength_e833b89f9db0273217h937eebccdafba65cE (type 1)))
  (import "__wbindgen_placeholder__" "__wbg_subarray_9482ae5cd5cd99d3" (func $_ZN6js_sys10Uint8Array8subarray31__wbg_subarray_9482ae5cd5cd99d317hfbc4e69e7490de6bE (type 11)))
  (import "__wbindgen_placeholder__" "__wbg_length_e09c0b925ab8de5d" (func $_ZN6js_sys10Uint8Array6length29__wbg_length_e09c0b925ab8de5d17hd53c873ca0d4adf5E (type 1)))
  (import "__wbindgen_placeholder__" "__wbg_set_a0172b213e2469e9" (func $_ZN6js_sys10Uint8Array3set26__wbg_set_a0172b213e2469e917h03d5ec4dbd548c78E (type 3)))
  (import "__wbindgen_placeholder__" "__wbindgen_throw" (func $_ZN12wasm_bindgen16__wbindgen_throw17hf0416b09e9451751E (type 10)))
  (import "__wbindgen_placeholder__" "__wbindgen_memory" (func $_ZN12wasm_bindgen17__wbindgen_memory17hbf7175b901445af8E (type 13)))
  (import "__wbindgen_externref_xform__" "__wbindgen_externref_table_grow" (func $_ZN12wasm_bindgen9externref31__wbindgen_externref_table_grow17hab300b53f6cf602aE (type 1)))
  (import "__wbindgen_externref_xform__" "__wbindgen_externref_table_set_null" (func $_ZN12wasm_bindgen9externref35__wbindgen_externref_table_set_null17h8cc4f9feb9c99eb0E (type 0)))
  (import "env" "read_register" (func $read_register (type 14)))
  (import "env" "register_len" (func $register_len (type 15)))
  (import "env" "value_return" (func $value_return (type 14)))
  (import "env" "panic_utf8" (func $panic_utf8 (type 14)))
  (import "env" "storage_read" (func $storage_read (type 16)))

Notice the two different styles of imports. The "env" "..." ones are importing the NEAR functions, whereas the __wbindgen_* ones are going to be the wasm-bindgen stuff. Unfortunately you won’t be able to use any crates that utilize wasm-bindgen for their webassembly support in NEAR – wasm-bindgen generates code incompatible with the near runtime bindings specification.

@silvestrpredko
Copy link
Author

silvestrpredko commented Jul 21, 2022

Also do you have a specification of NEAR runtime and how does it differs from a JS runtime, what is allowed and what is not. I know that networking isn't allowed, but I was surprised with rand stuff.

The various APIs exposed to the NEAR contract are specified in this section of the specification. In JavaScript world the wasm code is allowed to set up and expose whatever APIs they wish. AFAIK usually developers will run their code through tools like wasm-bindgen and wasm-pack to set up these APIs for them automatically.

Let us run your contract through the wasm2wat tool first, to get a readable representation of the code. Looking at the imports we can see these:

Notice the two different styles of imports. The "env" "..." ones are importing the NEAR functions, whereas the __wbindgen_* ones are going to be the wasm-bindgen stuff. Unfortunately you won’t be able to use any crates that utilize wasm-bindgen for their webassembly support in NEAR – wasm-bindgen generates code incompatible with the near runtime bindings specification.

Understood, thanks for the support.

@nagisa nagisa changed the title During function call VM failed with a FunctionCallError(CompilationError(PrepareError(Instantiate))) The runtime fails to provide users sufficient feedback to investigate issues on their own Jul 21, 2022
@matklad
Copy link
Contributor

matklad commented Jul 28, 2022

Closing as the particular mystery has been solved. The error handling is tracked at #5315.

@matklad matklad closed this as completed Jul 28, 2022
@frol
Copy link
Collaborator

frol commented Jun 5, 2023

Another mystery appeared with Rust 1.70.0 release: near/create-near-app#2009

This time it is severe as it seems to affect all contracts compiled with Rust 1.70.0 while 1.69.0 works just fine.

~@nagisa Could you help debug it? 🙏 ~

UPD: It has been investigated in #9140 (comment)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
T-contract-runtime Team: issues relevant to the contract runtime team
Projects
None yet
Development

No branches or pull requests

7 participants