Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for Zig #144

Closed
Angelmmiguel opened this issue May 30, 2023 · 2 comments · Fixed by #196
Closed

Add support for Zig #144

Angelmmiguel opened this issue May 30, 2023 · 2 comments · Fixed by #196
Assignees
Labels
🚀 enhancement New feature or request

Comments

@Angelmmiguel
Copy link
Contributor

Angelmmiguel commented May 30, 2023

Zig is a simple and general-purpose programming language. Recently, it's gaining more traction thanks to its benefits. Zig supports Wasm + WASI compilation natively and they use it extensively.

Since it can be compiled to WebAssembly, so there's no requirement of a language runtime like Python, Ruby or JavaScript. The goal for this task is to create the wrapper code to interact with Wasm Workers Server. The same approach we took for the Rust and Go support.

Technical requirements

These are the steps to add Zig support:

  1. Read the JSON metadata that comes via STDIN. Here you have the example from the Rust kit:

    pub struct Input {
    url: String,
    method: String,
    headers: HashMap<String, String>,
    body: String,
    kv: HashMap<String, String>,
    #[serde(default)]
    params: HashMap<String, String>,
    }

  2. Define the Request, Response and Cache entities. If it's available, it's better to use primitives from the Golang standard library or common usage libraries.

  3. Initialize a Request object from the JSON metadata.

  4. Initialize the Cache store.

  5. Call the worker function with the Request instance.

  6. Detect the content in the response. If it's a valid UTF-8 string, we can return. If it uses a different encoding, you need to encode it in base64.

  7. Transform the response and the Cache status into the expected JSON output. Note that the base64 flag is only enabled when the content is encoded.

    pub struct Output {
    data: String,
    headers: HashMap<String, String>,
    status: u16,
    kv: HashMap<String, String>,
    base64: bool,
    }

  8. Print the JSON output via STDOUT

Additional notes

  • The library should be defined in the kits/zig folder.
  • The documentation is available in the site: https://workers.wasmlabs.dev/docs/languages/introduction.
  • Currently, we're not publishing the libraries to language registries. Instead, we document how to get it from the GitHub repository.
@Angelmmiguel Angelmmiguel added the 🚀 enhancement New feature or request label May 30, 2023
voigt added a commit to voigt/wasm-workers-server that referenced this issue Aug 14, 2023
@voigt
Copy link
Contributor

voigt commented Aug 14, 2023

Hey @Angelmmiguel,

I started working on this issue (see PR). I followed the Golang implementation as closely as possible. Unfortunately, I was not able to use Zig standard library for the Response entity (see issue). Request seems to be working.

The implementation is still quite hacky as I wanted to get the "break through" first; tests are missing as well.

This is what executing wws and curl http://127.0.0.1:8080/basic looks like at the moment:

$ zig build -Dtarget="wasm32-wasi" && WASMTIME_BACKTRACE_DETAILS=1 wws ./zig-out/bin/
⚙️  Preparing the project from: ./zig-out/bin/
⚙️  Loading routes from: ./zig-out/bin/
⏳ Loading workers from 2 routes...
✅ Workers loaded in 332.203667ms.
    - http://127.0.0.1:8080/basic
      => ./zig-out/bin/basic.wasm
    - http://127.0.0.1:8080/main
      => ./zig-out/bin/main.wasm
🚀 Start serving requests at http://127.0.0.1:8080

raw input json: {"url":"/basic","method":"GET","headers":{"sec-ch-ua":"\"Chromium\";v=\"115\", \"Not/A)Brand\";v=\"99\"","accept":"text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7","sec-fetch-site":"none","host":"127.0.0.1:8080","sec-ch-ua-platform":"\"macOS\"","sec-fetch-user":"?1","sec-fetch-mode":"navigate","dnt":"1","accept-encoding":"gzip, deflate, br","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/115.0.0.0 Safari/537.36","sec-ch-ua-mobile":"?0","sec-fetch-dest":"document","upgrade-insecure-requests":"1","accept-language":"en-US,en;q=0.9","connection":"keep-alive"},"body":"","kv":{},"params":{}}
Hello from function
output json: {"data":"Zig World!","status":200,"base64":false,"headers":{"content-type":"text/plain","x-generated-by":"wasm-workers-server"},"kv":{"hello":"world"}}
Done.
error while executing at wasm backtrace:
    0: 0x30ba - _start
                    at /Users/c.voigt/.asdf/installs/zig/0.11.0/lib/std/start.zig:209:42
[2023-08-14T11:52:06Z INFO  actix_web::middleware::logger] 127.0.0.1 "GET /basic HTTP/1.1" 503 47 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/115.0.0.0 Safari/537.36" 0.013806

Input & Output entities look good to me (please correct me if I'm wrong!). Output JSON is written to stdin as well as stderr (therefore, it is displayed in the log).

However, the wasm runtime doesn't exit gracefully, and I'm puzzled why. I assume this is the reason, why the request fails. Maybe you (@Angelmmiguel) have an idea?

Edit: After doing some research I understood that there are two ways a WASI module is executed: command- and reactor-mode. Seems like WWS requires reactor! It is working now, as I changed the compile configuration - I will now proceed with making things nice & polished.

voigt added a commit to voigt/wasm-workers-server that referenced this issue Aug 17, 2023
@voigt voigt mentioned this issue Aug 17, 2023
3 tasks
@Angelmmiguel
Copy link
Contributor Author

That's amazing @voigt ! Thank you very much for your contributions here! I'm looking to demo Zig support 🤩

Angelmmiguel pushed a commit that referenced this issue Sep 5, 2023
* feat: add support for zig (#144)

* add wasi_exec_model reactor

* use custom request/response entities

* add params and env

* add example zig-basic

* add example zig-kv

* add example zig-params

* add example zig-envs

* add zig docs to example README

* add .gitignore to ignore zig build and cache directories

* add utf8 validity check

* add corrections to docs

Co-authored-by: Rafael Fernández López <ereslibre@gmail.com>

* rename writeHeader to setStatus

* rename s/data/body to be consistent with other languages

* rename s/bash/shell-session

Co-authored-by: Rafael Fernández López <ereslibre@gmail.com>

* add info for RequestAndOutput wrapper struct

* fix read input until EOF

---------

Co-authored-by: Rafael Fernández López <ereslibre@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🚀 enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants