Skip to content
This repository has been archived by the owner on Aug 14, 2023. It is now read-only.

SDK Restricted Wasm (prerequisite for the Fixed-Gas) #228

Merged
merged 30 commits into from
Apr 28, 2021

Conversation

YaronWittenstein
Copy link
Contributor

@YaronWittenstein YaronWittenstein commented Apr 8, 2021

Motivation

We want the Wasm-compiled code of SVM Apps to have restrictions that will enable us to give fixed-gas estimations for transactions.

From the end-user point of view, he/she will know prior to dispatching a transaction to the network the exact quantity of gas units required to execute the transaction without hitting the out of gas error.

Having said that, a transaction can still fail - for example by panic-ing.

A restricted Wasm program should meet the following:

  • No usage of the loop opcode.
  • No recursive calls /cycles detected when doing static analysis of the code.
  • No usage of the call_indirect opcode (no function pointers, i.e no Polymorphism).

In order to achieve the above, the PR adds a crate named svm-sdk-std that will serve as a tiny replacement for the default Rust std. By introducing this crate we have full control of the emitted Wasm and we can cherry-pick only features that are relevant for us. Many places in the code base have been adapted to use this crate.

A special note should be given to heap-allocation. Since running transactions are short-lived programs, we can shun away from deallocations of heap memory. Also, we can have a fixed amount of Wasm memory allocated upon instance initialization.

Consequently, this PR adds a feature flag named static-alloc that is enabled for SVM Apps. The global allocator used when running such programs is delegating the allocation to the Host (it merely increments the total size of allocated memory).

There are pieces of common code (mainly encoding related) that are shared by the svm-codec crate and the SVM Apps.
When used in the context of the codec a feature flag named dynamic-alloc is being used.

By turning on this feature flag, the code of the svm_codec.wasm artifact is self-contained. We need it to be so since it's a component that should be used from within smapp (or similar). So we end up having the allocation code is embedded within as opposed to SVM Apps that rely on the Host to do the allocation).

A feature PR will add static-code analysis to check for Wasm programs inputs validity (as part of the deploy-template flow)
and it will also expose the gas costs calculation for transactions. (spawn-app and exec-app transactions).

other: This PR also rewrites the GitHub Actions workflow to be more robust.

@YaronWittenstein YaronWittenstein self-assigned this Apr 8, 2021
@YaronWittenstein YaronWittenstein added big Big-sized task must-for-mainnet Must be done before Mainnet labels Apr 8, 2021
@YaronWittenstein YaronWittenstein changed the title Fixed gas Fixed-Gas Apr 8, 2021
Implemented the `svm_allocate` host function.

adding some debugging printing to the new `svm_allocate` host function.

WIP: computing the `returns` byte-size for a function (sdk-macros)

* `calldata` is allocated on instance only when non-empty.

The `svm-sdk` relies on the host function for Wasm targeted code and on the default global allocator otherwise.

adding `svm_sdk_std::panic` (aborts for Wasm and unwinds otherwise)

WIP: adding an explicit feature flag for the host static allocation.

svm-codec is using the default global-alloactor (not the "static allocation")

codec - js tests pass (commenting the `verify_data` for now)

tests pass

CI - running only on Windows + running the tests 10 times (there seems to be a sporadic failures)

CI - skipping the build stage

svm-sdk-std: exposing Rust `Vec`

svm-sdk-std: using `panic!`

running only the `svm-runtime` crate tests.

Removing `wabt` (using only `wat`) - it takes much less time to build.

using cache action v2

Adding GitHub Action for LLVM

Adding caching to the llvm

LLVM 10

Returning back the original LLVM action for Windows

adding feature-flags `default-cranelift` and `default-llvm`

trying to run CI against `llvm`

running tests with a single thread

runtime - removing two ignored tests related to gas-metering

Commenting a test

runtime: ignoring all tests except one

running tests in dev-mode

Building on Linux and macOS

svm-sdk-std: returning back Rust `Vec`

Commenting part of the problematic test

Commenting another part of the problematic test

uncomment part of the problematic test

cbindgen version bump

problemtic test - wasm input file "runtime_calldata.wasm" isn't using the "static-alloc"

problemtic test: input wasm is being compiled with less code and no `.cargo/config` file

Uncommenting all the problematic test

Trying to locate the bug

Trying again to pinpoint the bug

Adding asserts against setting explicitly empty `returndata`

WIP: debugging

WIP: debugging...

WIP: debugging...

skipping the last CI stages for now...

WIP: trying to locate the cause of the bug...

WIP

commenting the "static-alloc" related code from `svm-sdk-alloc`

WIP

Trying again to make the test fail

Trying to figure out whether the #[endpoint] return-type has anything to do with the bug...

Commenting "Storage" from the wasm input of the failing test...

More debugging the root cause of the problem

debugging...

adding "Cache workspace" step

Splitting cargo caching into isloated steps.

adding "Dump GitHub context" step.

CI: changing to `crates/runtime` before running the tests

Try again to run CI...

Try again

CI: disabling LLVM installation for now (Windows)

Trying again to reproduce a broken CI on Windows...

Checking whether the bug has something to do with `svm_sdk::Address`

Trying to find a minimal failing input using `Amount`

WIP: Debugging...

WIP: removing the parameter from the problematic endpoint

This should pass (not using `endpoint`)

This should fail again...

`ExtHost#get_calldata` - returning a static data

Re-compile the wasm input

Empty `endpoint` prologue.

debugging

debugging...

Trying to narrow the bug

WIP

Narrowing the epilogue part

should fail...

Uncommenting part of the epilogue

WIP
@YaronWittenstein YaronWittenstein marked this pull request as ready for review April 18, 2021 19:12
@YaronWittenstein YaronWittenstein merged commit c9b3fb1 into master Apr 28, 2021
@YaronWittenstein YaronWittenstein deleted the fixed-gas branch April 28, 2021 17:55
@YaronWittenstein YaronWittenstein changed the title Fixed-Gas SDK Restricted Wasm (prerequisite for the Fixed-Gas) Jun 3, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
big Big-sized task must-for-mainnet Must be done before Mainnet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants