[Enhancement] KCL needs a lighter backend end for easy installation and use.

[Peefy]

Problems

1. LLVM dependency is heavy, users need to install additional LLVM and cc when using kclvm through Rust library. (e.g. the inkwell crate: https://github.com/TheDan64/inkwell). Besides, this causes the KCL compiler to compile itself into WASM and embed it in the browser with heavy burden. Related Issue: Provide Statically Linked Binaries #1100
2. There are many problems when compiling KCL to a native target on the linker including macOS, linux, ubuntu with high maintenance costs. (e.g. rustc targets: https://github.com/rust-lang/rust/tree/master/compiler/rustc_target/src/spec and linker: https://github.com/rust-lang/rust/blob/master/compiler/rustc_codegen_ssa/src/back/link.rs). Besides, compiling to native results in poor concurrency at runtime, requiring CPython GIL locks or JVM partial locks.
3. The original kclvm python version can be directly installed and run through pip python3 -m pip install kclvm. The kclvm trust also needs to be compiled cargo install kclvm, installed and run through cargo install (convenient for integration by other two party and three party applications).

Design

Design 1

Migrate KCLVM python's stack virtual machine to Rust.

Pros.

• The implementation and migration of the stock are relatively reliable. KCLVM Python has verified the problems of runtime concurrency and memory, which is relatively stable.
• The implementation and migration of the stock is relatively reliable. KCLVM Python has verified the runtime concurrency and memory problems, which are relatively stable and have good performance. (even though not much optimization has been done, because the performance bottleneck is memory copy).

Cons.

• KCL code cannot compile WASM target by itself.

Design 2

Just like rego (https://github.com/open-policy-agent/opa), cadence (https://github.com/onflow/cadence), and wa (https://github.com/wa-lang/wa), write a WASM codegen adaptation layer, and the executive layer uses the industry's mature WASM virtual machine, such as wasmtime (https://github.com/bytecodealliance/wasmtime).

Pros

• The WASM target is universal and mainstream in the industry.

Cons

• High development cost

Transition Plan

Just like libtaichi.dylib in the taichi repo (https://github.com/taichi-dev/taichi), package the KCLVM rust core library into the pip package python3 -m pip install kclvm or go package for installation go get kclvm-go.

[NeverRaR]

We can combine the first and second solutions.
Step 1: Migrate the stack virtual machine of python kclvm to rust.
Step 2: After ensuring availability, then we can implement a transpiler from python bytecode to wasm.

[Peefy]

We need to study these pre-dependencies:

1. What problems can't be solved by LLVM, or the cost is very high @Peefy
2. Give priority to the lightweight scheme from LLVM to WASM @chai2010
3. WASM can be used as a long-term solution to continue to sort out the technical details: the possible pitfalls, difficulties, workload, and wasm runtime model selection when KCL is directly compiled into WASM, and whether the performance and stability meet the demands of various internal and external scenarios. @Peefy @chai2010

[reckless-huang]

Or we can take a different approach and allow KCL to run as a standalone server,client communicate via HTTP or gRPC.

[Peefy]

Thank you! @reckless-huang ❤️

This is a great suggestion. Actually, we have already built a KCL server for interacting with external systems, and you can see here https://kcl-lang.io/docs/reference/xlang-api/rest-api and view a list of all HTTP/gRPC functions, including the ListOptions API. But the kcl server will include both the kcl compiler and runtime, especially LLVM, which results in a larger binary size. The main purpose of this issue is to track lighter clients without LLVM, and it is expected to reduce the size by at least half.