generate: output WIT package to the generated code directory#179
generate: output WIT package to the generated code directory#179Mossaka wants to merge 6 commits intobytecodealliance:mainfrom
Conversation
Mossaka
force-pushed
the
output-wit
branch
3 times, most recently
from
f9812aa to
4e656a3
Compare
Signed-off-by: Jiaxiao Zhou (Mossaka) <duibao55328@gmail.com>
adds function `writeWITPackage` to output the given WIT package to the wit directory placed inside of the generated Go code
instead of outputting the WIT package encoded as a Wasm component
Signed-off-by: Jiaxiao Zhou (Mossaka) <duibao55328@gmail.com>
generate WIT files using wit.Resolve.WIT() instead of calling `wasm-tools` Signed-off-by: Jiaxiao Zhou (Mossaka) <duibao55328@gmail.com>
|
rebased this PR and ready for review @ydnar |
| if err := os.MkdirAll(witDir, cfg.outPerm); err != nil { | ||
| return err | ||
| } | ||
| witFilePath := filepath.Join(witDir, "webassembly.wit") |
There was a problem hiding this comment.
Could the name should be based on the top-level WIT package, if possible, and nested in that package's directory, e.g. $out/wasi/cli/webassembly.wit?
There was a problem hiding this comment.
I think a common practice for placing WIT files is to place it at a wit directory. In addition, a WIT package can have multiple worlds, and wit-bindgen-go generate generates all of them. Take wasi:cli as an example, it generates
.
├── go.mod
├── go.sum
├── main.go
├── main.wasm
├── wasi
│ ├── cli
│ │ ├── stderr
│ │ │ ├── empty.s
│ │ │ └── stderr.wit.go
│ │ ├── stdin
│ │ │ ├── empty.s
│ │ │ └── stdin.wit.go
│ │ └── stdout
│ │ ├── empty.s
│ │ └── stdout.wit.go
│ ├── clocks
│ │ ├── monotonic-clock
│ │ │ ├── empty.s
│ │ │ └── monotonic-clock.wit.go
│ │ └── wall-clock
│ │ ├── empty.s
│ │ └── wall-clock.wit.go
│ ├── http
│ │ ├── incoming-handler
│ │ │ ├── empty.s
│ │ │ ├── incoming-handler.exports.go
│ │ │ └── incoming-handler.wit.go
│ │ ├── outgoing-handler
│ │ │ ├── abi.go
│ │ │ ├── empty.s
│ │ │ └── outgoing-handler.wit.go
│ │ ├── proxy
│ │ │ └── proxy.wit.go
│ │ └── types
│ │ ├── abi.go
│ │ ├── empty.s
│ │ └── types.wit.go
│ ├── io
│ │ ├── error
│ │ │ ├── empty.s
│ │ │ └── error.wit.go
│ │ ├── poll
│ │ │ ├── empty.s
│ │ │ └── poll.wit.go
│ │ └── streams
│ │ ├── abi.go
│ │ ├── empty.s
│ │ └── streams.wit.go
│ └── random
│ └── random
│ ├── empty.s
│ └── random.wit.go
└── wit
└── webassembly.wit
I think place the WIT file in the wasi/cli directory is not the best option here.
There was a problem hiding this comment.
Multiple WIT files might be generated in the same out directory. So we need some way to avoid overwriting WIT files in the same directory
There was a problem hiding this comment.
Perhaps, we should limit wit-bindgen-go generate to require a World name, and the WIT file can be named <out>/wit/<world-name>.wit?
There was a problem hiding this comment.
wkg pattern is single file <namespace>_<package>@<version>.wit
zz ❯ wkg get wasi:http
No version specified; fetching version list...
Getting wasi:http@0.2.2...
Wrote './wasi_http@0.2.2.wit'
zz ❯ wkg get wasi:cli
No version specified; fetching version list...
Getting wasi:cli@0.2.2...
Wrote './wasi_cli@0.2.2.wit'
zz ❯ ls
wasi_cli@0.2.2.wit wasi_http@0.2.2.wit
If possible I'd like this PR to wait until next BA meeting so we can sync on overall strategy.
I think we all agree we don't want to shell out to wasm-tools either here and in Tinygo, and some ideas floated before ( ex: go embed ) have downstream side effects to the Go ecosystem.
There is a path to combine the component compilation story & wit handling like the Rust export! macro, removing the need for interaction with the actual wit files altogether.
https://docs.rs/wit-bindgen/0.33.0/wit_bindgen/macro.generate.html#exports-the-export-macro
The 🔑 apis we're missing:
- wit grammar parser: goes from files/oci artifacts to structs
- wit world resolver: apply aliases, includes, and other resolution rules
- wasm patcher: stitch wasm files & embed wit metadata
I think if we split up the work we can get there real quick.
If we do continue down the path of using wit files & shelling out, then we should align the story to other Bytecode Alliance tools as much as possible as they cover other concerns as well. In this case, following wkg file pattern means users can upload/parse/code-gen-other-languages using wasm-tools, wit-bindgen, and wkg as they are all converging on wkg's implementation.
|
Added #203 which can generate tree-shaken WIT for a single world, as a precursor for this PR to emit a named WIT file inside the equivalent Go package path for the world. |
|
Close since #206 implements this feature. |
3 participants
This PR allows the generator to output WIT package to th generated Go directory. builds on top of #146
TODO:
needs a rebase once generate: pull WIT files from OCI registry #146 merges to main.