You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Being able to compile the rust bindings into wasm32-unknown-unknown and/or wasm32-wasi would be interesting. This could allow some interesting interactions between a browser and spark. A wasm32-wasi target would allow spark programs to run on any runtime.
Early Experiments
A feature flag under the core bindings for wasm already exists and does compile successfully to those targets mentioned above. The issue arises when trying to send a HTTP1.1 request with grpc-web to the Spark Connect server, which only accepts normal HTTP2 grpc requests. There are methods of standing up a proxy server with envoy to forward the gRPC browser request to the backend server. But this feels like a lot of effort for the client to do.
The branch feat/wasm contains the early experiment and trying to run wasm with wasmtime. Issue arises with using async code in wasm. There is probably a way to code it correctly, but I don't have time to finish the experiment
The text was updated successfully, but these errors were encountered:
Was able to create a wasm program to run a simple spark structure streaming job. The brach feat/wasm was updated with envoy to handle the grpc-web requests. Might consider having an 'example' section for how to setup envoy and use the wasm feature flag.
Description
Being able to compile the rust bindings into
wasm32-unknown-unknown
and/orwasm32-wasi
would be interesting. This could allow some interesting interactions between a browser and spark. Awasm32-wasi
target would allow spark programs to run on any runtime.Early Experiments
A feature flag under the
core
bindings forwasm
already exists and does compile successfully to those targets mentioned above. The issue arises when trying to send a HTTP1.1 request withgrpc-web
to the Spark Connect server, which only accepts normal HTTP2 grpc requests. There are methods of standing up a proxy server with envoy to forward the gRPC browser request to the backend server. But this feels like a lot of effort for the client to do.The branch feat/wasm contains the early experiment and trying to run wasm with wasmtime. Issue arises with using async code in wasm. There is probably a way to code it correctly, but I don't have time to finish the experiment
The text was updated successfully, but these errors were encountered: