add new target like wasm32-unknown-unknown in rust

[hunjixin]

Simple Summary

Add new wasm32-unkonw-unkonw/unknow_wasm target in TinyGo. Use this target to generate clean wasm which have no dependecies on JS or WASI.

Abstract

In the current implementation, WASM target depends heavily on the JS execution environment, and the WASI target requires the corresponding execution platform to suppoprt the POSIX protocol or something like POSIX.

When demands come to generate an algorithm to run on a specific WASM VM, there is usually no JS execution environment and no posix support in these vm environment. For example, if one would like to run the WASM code generated by TinyGo, we have to introduce some JS/WASI compatibility layer code. However, when the execution environment of the VM is developed by a third party and not within control, the developer then cannot run the compatibility layer code. At this time, it is forced to use the WASI target which translates to mannual changes of many modules in its project causing much inconvenience and infeasibilities in some cases.

A functional target similar to Rust's wasm32-unknown-unknown is needed to generate clean WASM without changing the existing library code.

Change Motivation

When the WASM file is to be run in a foreign VM environment and this target relies heavily on the JS running environment, it becomes impossible to use the WASM target. When the target is WASI, the generated code will have dependency on the WASI protocol resulting in having some import that depends on the WASI protocol. Compared with the WASM target, there is a possibility of realizing the implementation which developer can make mannual changes to the code (user module, standard package) to remove the imports on WASI. However, this approach implies large amount of engineering efforts.

For the general user library code（include dependecy), there are various parts that use POSIX functions such as (file/os/net/time) etc, which can be patched by cherry picking the corresponding module, modifying the module, removing the code that have WASI dependency and then replacing them with edited modules throughout your project. Through this method, we can achieve the goal of running targets, but it will be very cumbersome. In practice, due to the depth of dependencies, it will be difficult to find the module with problems. In addition, replacement will also bring side effects, which will require the replacement of other libraries that references the edited library.

For the standard library, it is even more cumbersome. In practice, fmt standard library has been one of the most serious problems. fmt use the stdout and stdin objects, which means that the fmt standard library depends on the os standard library that comes with an init function...(shown below)

// Args hold the command-line arguments, starting with the program name.
var Args []string

func init() {
	Args = runtime_args()
}

Not only does args_size invokes dependency on WASI, but runtime_args function further includes...

  (import "wasi_snapshot_preview1" "args_sizes_get" (func $runtime.args_sizes_get (;0;) (type 3)))
  (import "wasi_snapshot_preview1" "args_get" (func $runtime.args_get (;1;) (type 3)))
  (import "wasi_snapshot_preview1" "clock_time_get" (func $runtime.clock_time_get (;2;) (type 6)))

Unfortunately the fmt library is widely used (Eg: fmt.Errorf), which brings great troubles,

• Standard library cannot be replaced
• There are too many user modules which use fmt package, making not realistic to replace all of them.

Therefore, we hope to refer to Rust's wasm32-unkonow-unkonwn, and use plateform specific code to handle these problems (io/file/net) when specifying the directive.

Implementation

1. Similar to WASI, add a new directive wasm_unknow to isolate the specific implementation to make sure previous WASM/WASI behaviour stays the same.
2. Add new target type wasm_unknow to generate cleaner WASM files through this kind of utility target.

The main idea is to refer to the implementation of rust wasm32-unknown-unknown https://github.com/rust-lang/rust/tree/master/library/std/src/sys/unsupported.

packages to be changed

1. src/os
2. src/syscall
3. src/runtime add new target support

behavior change

1. os.args/os.env always return empty slice
2. file/net not support ,
3. stdout. discard
4. stdin return io.EOF
5. time flow rust panic behavire (many people use time as rand seed, autoincrease nano time cause predicatable randomness
6. syscall use xx_emulated.go, and no Signal support
7. Signal remove signal (Signal always used in main file, less used in library, so remote it)

Maintainer

I am also very happy to maintain this target, and I also have a team member who can invest in the development of this target.

@dankegel @deadprogram @aykevl

[hunjixin]

os: add wasm_freestanding directive in dir_others, file_others, state_others, remove_other

systcall：

• add wasm_freestanding directive in file_emulated, syscall_nonhosted, proc.emulated

runtime:

• add !wasm_freestanding in other plateform file include(runtime_tinywasm， runtime_wasi)
• add new file runtiem_wasm_freestanding
  1. os_runtime_args return empty slice
  2. syscall_runtime_envs return empty slice
  3. putchar do nothing
  4. syscall_Exit return 0
  5. time related function return 0 or panic?

[codefromthecrypt]

@hunjixin ps some of the smart contract stuff bled into discussions on wazero and one thing in general was decent fakes. With some trickery the host can actually write wasm to do the same as a freestanding target, but I like the idea of inlining in the compiler. Basically, this is saying this program has no need and will never have a need for certain things.

Some things we did in wazero for consistency by default and maybe effect choices here:

• seed the random source with 42 always (because it is the "Ultimate Answer")
• set walltime to 2022-01-01 UTC and increment it by 1ms on each reading
• set nanotime to 0 and increment it by 1ms on each reading
• nanosleep returns immediately
• no args or env
• no filesystem
• stdin reads io.EOF
• stdout/stderr to io.Discard

We've not recently had any new requests for this sort of thing so maybe the above are working.

Hope this helps!

[codefromthecrypt]

another motivation to do something like this is even if a host stubs similarly, there's a very real cost to function calls from wasm->host and back. This can cause inconsistency in contract performance, and raises the amount of potential errors, too.

[hunjixin]

@codefromthecrypt thanks！ most of the current behaviors are implemented flowing the behavior of rust. however, usage habits of rust and go are inconsistent and may need some change. especially in time, it's not a good idea to panic directly

[ltp456]

I have the same problem, hope for a quick update

[Pl8tinium]

I have the same problem, hope for a quick update

+1

:)

[deadprogram]

I have been working on a version of this, please see #4125

[deadprogram]

#4125 has been merged so now tagging this issue to be closed on next release.

[deadprogram]

Completed with v0.31.0 so now closing. Thank you!