Skip to content

Release 2.0.0

Choose a tag to compare

@robinskil robinskil released this 29 Sep 10:28
· 184 commits to main since this release
3feeb74

What's Changed

  • N-dimensional execution spine: flow zarr/netcdf through beacon.nd-encoded batches by @robinskil in #328
  • Fix view types (utf8view -> utf8) by @robinskil in #326
  • nd projection pushdown: evaluate element-wise projections before the broadcast by @robinskil in #329
  • nd filter pushdown: run WHERE predicates before the broadcast by @robinskil in #330
  • Features/redb object store by @robinskil in #332
  • Move the wire contract out of the runtime, gate the metadata schemas, and rebuild the query editor by @robinskil in #333
  • Reposition Beacon as one product: a query engine and server for scientific data by @robinskil in #334
  • Rewrite the README by @robinskil in #335
  • fix(nd): decode every row of a coalesced nd batch by @robinskil in #336
  • feat(netcdf): add the oxcdf pure-Rust reader behind a flag by @robinskil in #337
  • fix(netcdf): report the chunk shape of the file by @robinskil in #341
  • fix(netcdf): null the _FillValue cells of a CF time variable by @robinskil in #343
  • fix(nd): fill the chunk shape from the last axis by @robinskil in #344
  • Expose Arrow decoder and streaming capabilities in TypeScript SDK by @robinskil in #346
  • Persistent per-file statistics: a store, a background collector, and scan-time file pruning by @robinskil in #347
  • Optional pure-Rust HDF5 reader via oxcdf-hdf5 by @robinskil in #358
  • Read Icechunk repositories through the Zarr reader by @robinskil in #356
  • Read Apache Iceberg tables: external tables and read_iceberg by @robinskil in #357
  • Prune HDF5 and netCDF files with their recorded statistics by @robinskil in #360
  • File statistics: Zarr supplies column ranges by @robinskil in #362
  • One provider for every file scan, pruning before it reads by @robinskil in #364
  • See what file statistics are doing, and collect them at boot by @robinskil in #370
  • Divide an nd scan across every partition with a work queue by @robinskil in #371
  • Assert the nd worker wait directly, not by a worker count by @robinskil in #376
  • Admin web UI respects the server URL prefix by @robinskil in #369
  • Keep each file's schema, so a query does not read the file twice by @robinskil in #375
  • Use datafusion-spatial for 123 spatial functions with PostGIS names by @robinskil in #379
  • Features/deps cleanup by @robinskil in #380
  • Render admin UI results from Arrow columns by @robinskil in #381
  • Add a pull request template by @robinskil in #384
  • Fix the GeoParquet scan, and add bounding box pushdown by @robinskil in #383
  • Close an OIDC claim bypass and update dependencies for security by @robinskil in #385
  • Merge every schema through the ArrowTypeWidening trait (#377) by @robinskil in #395
  • Enable file statistics by default, and correct the reader defaults by @robinskil in #394
  • Read netCDF and HDF5 with the pure-Rust reader by default (#392) by @robinskil in #400
  • Add an integration test for each file format (#399) by @robinskil in #404
  • Adapt each CSV and Arrow IPC file to the merged schema by @robinskil in #405
  • Admin-gated path alias for Admin UI by @robinskil in #408
  • fix: restore BEACON_S3_DATA_LAKE and document the log files by @robinskil in #415
  • docs: add a complete example for file statistics inspection by @robinskil in #418
  • feat: read HDF5 files with unnamed dimensions by @robinskil in #423
  • fix: use NC as the netCDF storage type in the admin UI by @robinskil in #426
  • docs: separate CREATE TABLE and CREATE EXTERNAL TABLE by @robinskil in #425
  • docs: list the OPTIONS of each format for CREATE EXTERNAL TABLE by @robinskil in #427
  • feat: name the source file of a failed schema merge by @robinskil in #429
  • fix: keep the case of a table name in every statement by @robinskil in #430
  • feat: support PARTITIONED BY for netCDF, HDF5 and TIFF tables by @robinskil in #431
  • feat: show the disk space of the datasets store in the admin UI (#432) by @robinskil in #436
  • deps: bump datafusion-spatial to 0.2.0 (#438) by @robinskil in #439
  • feat: use ST_Within for the GeoJSON filter of the JSON query by @robinskil in #440
  • feat: publish the beacondb wheel again, with PROJ in the wheel (#414) by @robinskil in #461
  • docs: bump the docs version to 2.0.0-rc4 by @robinskil in #462
  • fix: stop the admin query editor race, and remove the query timeout by @robinskil in #465
  • feat: add the COMPACT TABLE statement for managed Lance tables by @robinskil in #464
  • fix: rebuild geometry constants before a federated plan becomes SQL by @robinskil in #466
  • feat: a file statistics pass drains the queue and runs alone by @robinskil in #467
  • fix: compare the object metadata against the record before a prune (#412) by @robinskil in #468
  • feat: sign in to the admin UI through the gateway that holds the credentials by @robinskil in #469
  • feat: keep the first type of a column that no type holds by @robinskil in #471
  • release 2.0.0-rc.5 and rebuild the docs home page by @robinskil in #472
  • feat: serve a home page at the server root by @robinskil in #474
  • feat: run queries on their own Tokio runtime by @robinskil in #476
  • feat: merge schemas as numpy promotes types by @robinskil in #478
  • fix: refuse a CREATE on a name a table already holds by @robinskil in #479
  • Rebuild the Atlas reader on the single-file format by @robinskil in #490
  • release: bump the version to 2.0.0-rc6 by @robinskil in #499
  • Clean up the type widening module and drop the conflict mark by @robinskil in #504
  • Register the bearer scheme in the OpenAPI document by @robinskil in #506
  • Give Lance its local I/O parallelism, not the CPU count by @robinskil in #509
  • Read BBF through the nd pipeline, with a column-list rule by @robinskil in #512
  • Refactor the atlas reader into one module per stage by @robinskil in #507
  • Add a flag to skip datasets that cannot broadcast by @robinskil in #515
  • Recognize the nc3, nc4 and netcdf file extensions by @robinskil in #516
  • Stream dataset listings and walk S3 in parallel by @robinskil in #513
  • Store and show the statement that created a table by @robinskil in #517
  • Fix bugs found in release testing by @robinskil in #521
  • Release 2.0.0 and make it the latest docs by @robinskil in #524

Full Changelog: v1.8.0...v2.0.0