wasm range request fails on Firefox

[Fil]

What happens?

If you type this query into shell.duckdb.org:
SELECT bbox FROM read_json('https://earthquake.usgs.gov/earthquakes/feed/v1.0/summary/all_day.geojson');

on Firefox, you get a "failed range request" error.

See observablehq/framework#1886

To Reproduce

Open Firefox to shell.duckdb.org, and type this query:
SELECT bbox FROM read_json('https://earthquake.usgs.gov/earthquakes/feed/v1.0/summary/all_day.geojson');

Browser/Environment:

Firefox 133.0.3 (aarch64)

Device:

Macbook Air

DuckDB-Wasm Version:

@duckdb/duckdb-wasm@1.29.1-dev18.0

DuckDB-Wasm Deployment:

https://shell.duckdb.org/

Full Name:

Fil Rivière

Affiliation:

Observable

[ROBBYCHERPES2]

Go

[fkauker-fsq]

I'd suggest trying

SELECT bbox FROM read_json_auto('https://earthquake.usgs.gov/earthquakes/feed/v1.0/summary/all_day.geojson');

source: https://motherduck.com/blog/getting-started-gis-duckdb/

[Fil]

@fkauker-fsq exact same error

[fkauker-fsq]

Ok reproduced it. There is also another message if you run it twice!

[carlopi]

Thanks for raising. I do reproduce.

Problem here is a non-trivial problem where the request being sent out of the browser has this headers:

Range	bytes=0-16383
Accept-Encoding	gzip, deflate, br, zstd, identity

and comes back a resource with those headers:

content-length	16384
content-encoding	gzip

Problem is that the resource here should have length 16384 AFTER decompression, while here it's 16384 pre-decompression and 125KB afterwards.

Given there is a browser in the middle, I would think here it's the server that it's not compliant, I would need to check how to detect this and fall back to a simpler strategy (ideally warning of current problem).

[greenmtnboy]

Any workarounds for this at the moment? Can you disable range requests?

[mootari]

@greenmtnboy Afaik the config options allowFullHTTPReads and reliableHeadRequests are meant to address that, but they may come with their own issues (see e.g. #1984).

[greenmtnboy]

Thanks! I played around with various combos of both and still got the same errors, but that might have been user error - I'll take another look. For now randomizing a URL suffix seems like the most reliable workaround.

[tobilg]

Is this eventually fixed by #2060?