Cloudflare workers

[dantodor]

Has anyone tried to use this within a CF worker ? I think it's a great use case, put your data in some parquet files somewhere ( web server, maybe later even integrate with CF R2 ), run your query engine at the edge.

[ankoh]

We'd love to! But the bundle size of Cloudflare workers is unfortunately capped at 1 MB right now, which gets problematic for Wasm very quickly.
Our gzipped wasm module is around 1.8 MB so we cannot bundle the module alongside the worker script today.
(We could probably just fetch the module but we'd have to find out what that means for the startup latency of the worker)
We'd like to explore this a bit more in the following months since CF unbounded workers could be a perfect fit for certain Serverless SQL workloads.

[ankoh]

Exciting! Can you please join our discord server here and write me a DM? I'd love to chat about this.

[ilyabo]

@ankoh You mentioned that there's a gzipped wasm module which is around 1.8 MB. I am using the one from jsdelivr and chrome dev tools say that it's 2.8 MB gzipped. Is it a different one?

[ankoh]

@ilyabo brotli compression should bring you down to 1.8. maybe 2.0 in more recent builds.

[ankoh]

Right, the jsdelivr ones seem to be served with gzip and are a bit larger.

[ilyabo]

Okay I see, thanks André

[ankoh]

Update here:
The biggest limitation with cloudflare workers is the fact that cloudflare only implements the Service Worker API.
This hinders us to, for example, use synchronous XHRs that are a strong prerequisite for our web filesystem at the moment.
That might change in the future, but remains a blocker for the time being.

[ignoramous]

@ankoh the size limit for Workers now stands at 5MB (enough to bundle in duckdb-wasm which is around ~3.5MB gzipd). Going from the conversation here, synch xhr is the only hold up now? Generally speaking, ignoring the effort involved, would async IO speed up DuckDB? If so, there's at least hope that DuckDB will eventually support it.

[elithrar]

(I work at Cloudflare on Workers + Databases)

• Bundle size is an easy one. Increases to that very soon.
• Synchronous XHR much harder, but…
• Wasm stack switching (https://v8.dev/blog/jspi) is something we’re evaluating.

The good thing is that DuckDB works with R2 now. The less good thing is that we’d obviously love to allow folks to run DuckDB on Workers — but once we get unblocked I’ll reach out to Hannes & let y’all know.

[carlopi]

Stack switching might be a way around sync XHR, or another potential path is work by @samansmink on resumable pipelines, a big step towards async I/O.

Looking forward for any news you might have, and might come back later with some questions.

[clementgalopin]

hi @elithrar, any news regarding the support of duckdb?

[ShravanSunder]

As of 2024-04

A Worker can be up to 10 MB in size after compression on the Workers Paid plan, and up to 1 MB on the Workers Free plan.
The bundle size limits have been increased.
Snchronous xhr is still not supported,
i think workers have first class support for wasm.