Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add minimal support for WebAssembly #277

Closed
wants to merge 1 commit into from

Conversation

albrow
Copy link

@albrow albrow commented Apr 17, 2019

This PR adds minimal support for compiling LevelDB to WebAssembly. WebAssembly support was added to the Go compiler starting with version 1.11 and you can read the Official Wiki Page for more information about how it works.

Most of the codebase already works in WebAssembly without any changes 😄One thing that doesn't work out of the box is the leveldb/storage package. The Go compiler already makes use of the fs package in Node.js when compiling to WebAssembly. It is easy to open, read, and write to files in a Node.js environment using the Go standard library. The only thing I needed to add in this PR was support for file locks, since the existing implementations (e.g., file_storage_unix.go) rely on OS-specific syscalls.

The implementation of file locks that I wrote here is not perfect. Unfortunately Node.js does not support true file locks, so my implementation relies on in-memory maps protected by a mutex. This means it does not prevent other processes from accessing the files. However, I believe it is good enough for most use cases.

You can compile Go tests to WebAssembly and run them in a Node.js environment by running GOOS=js GOARCH=wasm go test ./.... Currently, large parts of the test suite fail to build because Ginkgo does not support WebAssembly. It is possible that there are test cases that would fail too if the build were to succeed. However, if you limit the tests to only the leveldb/storage package, they pass! You can try it yourself with GOOS=js GOARCH=wasm go test ./leveldb/storage -v.

Additionally, with the changes in this PR, I was able to compile this simple program to WebAssembly and run it with GOOS=js GOARCH=wasm go run main.go:

package main

import (
	"fmt"

	"github.com/syndtr/goleveldb/leveldb"
)

func main() {
	db, err := leveldb.OpenFile("leveldb_test", nil)
	defer db.Close()
	if err != nil {
		panic(err)
	}

	fmt.Println("Setting value foo = bar")
	if err := db.Put([]byte("foo"), []byte("bar"), nil); err != nil {
		panic(err)
	}
	value, err := db.Get([]byte("foo"), nil)
	if err != nil {
		panic(err)
	}
	fmt.Printf("Got value: %q\n", value)
}

Working in Node.js is one thing, but browser support is arguably more interesting. I have done some initial testing and it looks like it is possible to get LevelDB working in the browser by using BrowserFS to emulate the Node.js fs package. One problem with that right now is that the Go compiler/runtime expects the fs package to support the special file descriptors 0, 1, and 2 for stdin, stdout, and stderr respectively. I opened this issue to add support for these special file descriptors to BrowserFS.

In any case, if you are interested, I can work more on supporting LevelDB in the browser in the future. There are a number of strategies we could use here and not all of them depend on making changes to BrowserFS.

@albrow
Copy link
Author

albrow commented Aug 2, 2019

Superseded by #288 which includes support for browsers.

@albrow albrow closed this Aug 2, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant