Runtime TypeError with apache-arrow

[cjwallace]

Hey, this is an exciting project and I'm having fun diving in!

I ran into a problem that surprised me when using duckdb-wasm with apache-arrow. If I import anything from @duckdb/duckdb-wasm, and anything (except types) from apache-arrow, I get a runtime type error:

TypeError: Cannot read properties of undefined (reading 'getVisitFnByTypeId')
    at vector.ts:365:36
    at vector.ts:381:5

I'm using SvelteKit (thus, Vite), and curiously, this error occurs only in dev (vite dev) but not prod (vite build && vite preview). However, once the error has been hit in dev, removing either of the imports does not fix it. I guess this is vite's hot module reload failing.

Here's a repo that minimally reproduces the problem: sveltekit-duckdb-arrow.
It is a deliberately boiler-plate SvelteKit app, created with npm init svelte@next ... and with only duckdb-wasm and apache-arrow installed on top.

Here is a complete svelte component that results in the error.

<script>
  import { AsyncDuckDB } from '@duckdb/duckdb-wasm';
  import { Table } from 'apache-arrow';
</script>

<h1>:)</h1>

I am not 100% convinced this is a duckdb-wasm problem. It could be a Vite problem, or something at the intersection, but I thought this was a reasonable place to ask, and I'm grateful for any help. Thanks!

[ankoh]

I'm not entirely sure yet what's happening here but can you please try to import the browser bundle directly?
(e.g. @duckdb/duckdb-wasm/dist/duckdb-browser)
This fixed the arrow error for me in your repro repository (but brought up an issue about a missing worker polyfill)
Have you seen this? https://github.com/duckdb-wasm-examples/sveltekit-typescript

I haven't really used svelte myself in the past but this looks like a bundling issue.

[cjwallace]

Yep, that fixes the TypeError in my actual project too (not just the repro) and raises the Worker problem (Worker missing in node - makes sense since the code runs on the server for SSR). I think you’re right that it’s a bundling issue.

I saw the SvelteKit example, which didn’t help with the Arrow problem, but it looks like it does include a solution to polyfill the Worker, so I should manage to get something working from here.

Thanks for the help!