You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Part of epic #40. Depends on #3 (ArchiveContents shared state).
Goal
Let SQL templates reference bundled files directly: read_csv('embed://data/cities.csv'). This bypasses IFileProvider
and goes through DuckDB's VirtualFileSystem, so we need a custom duckdb::FileSystem subclass registered with the DuckDB instance
flapi already owns.
Scope
New src/duckdb_embed_fs.{hpp,cpp} subclassing duckdb::FileSystem.
Modified: src/main.cpp (register on DB instance when bundle exists)
Notes
DuckDBVFSProvider already touches DuckDB's internal FileSystem
API (src/vfs_adapter.cpp:218-397) and has TODOs about that being
unstable. The embed FS uses the same surface — we accept the same
trade-off, and the round-trip integration test (Flapi exe is not working for non docker local testing #6) catches drift
on every CI run.
Part of epic #40. Depends on #3 (
ArchiveContentsshared state).Goal
Let SQL templates reference bundled files directly:
read_csv('embed://data/cities.csv'). This bypassesIFileProviderand goes through DuckDB's
VirtualFileSystem, so we need a customduckdb::FileSystemsubclass registered with the DuckDB instanceflapi already owns.
Scope
src/duckdb_embed_fs.{hpp,cpp}subclassingduckdb::FileSystem.std::shared_ptr<ArchiveContents>asthe embedded file provider (Ability to pass default values to query parameters #3) — one decompressed map, two readers.
OpenFile— return a customFileHandleover an in-memory blob.Read(both overloads),Seek,GetFileSize.Glob—read_csv()expands paths before opening; basethrows "not implemented" (spike caught this at runtime).
SeekPosition— same reason.CanHandleFilechecks forembed://scheme prefix.(
src/main.cpp, after the database manager exists) — only when abundle is present.
Red tests (
test/cpp/duckdb_embed_fs_test.cpp)EmbeddedFileSystemover a fixture archive containingdata/cities.csv.SELECT * FROM read_csv('embed://data/cities.csv').Glob('embed://data/*.csv')returns the expected entries(catches
read_csvglob expansion).embed://against a regularparquet path on disk works (no interference with default
filesystem).
Green criteria
make integration-teststill green.Files
src/duckdb_embed_fs.{hpp,cpp},test/cpp/duckdb_embed_fs_test.cppsrc/main.cpp(register on DB instance when bundle exists)Notes
DuckDBVFSProvideralready touches DuckDB's internal FileSystemAPI (
src/vfs_adapter.cpp:218-397) and has TODOs about that beingunstable. The embed FS uses the same surface — we accept the same
trade-off, and the round-trip integration test (Flapi exe is not working for non docker local testing #6) catches drift
on every CI run.