v0.9.2

v0.9.2 is a point release of mostly fixes and chores.

Installing or Updating

CLI

To install or update just run the following command in the directory you want
the binary:

curl https://glaredb.com/install.sh | sh

Python

pip install glaredb

Node.js

npm install @glaredb/glaredb

What's Changed

• chore: add dbt read_csv test by @talagluck in #2732
• feat: insert into sqlite tables by @tychoish in #2745
• refactor: move parser into own crate by @universalmind303 in #2747
• feat: Initial globbing ranges support by @vrongmeal in #2654
• feat: initial external table schema by @tychoish in #2333
• feat: multi-line json streaming by @tychoish in #2729
• chore: add relative path regression test by @tychoish in #2749
• chore: cleanup in python bindings tests by @tychoish in #2750
• fix: kdl support in hybrid by @tychoish in #2751
• chore: add note to python init.py files by @tychoish in #2753
• fix: error string clarifying quotes by @tychoish in #2755
• feat: add some table aliases for lance,iceberg and delta by @universalmind303 in #2752
• chore: upgrade dependencies by @tychoish in #2769
• fix: Retry writing lease when error code 429 (too many requests). by @vrongmeal in #2773
• chore: add explicit schema support for json streaming by @tychoish in #2766
• Update profiles.yml by @talagluck in #2780
• chore: change lance ref by @tychoish in #2781
• chore: update cargo lock by @tychoish in #2782
• fix: Remove incorrect temp table check during resolve by @scsmithr in #2783
• chore: deps upgrade 2024-03-15 by @tychoish in #2792
• chore: Create a new temp object when writing lease all the time. by @vrongmeal in #2776
• fix: respect bigquery limit by @tychoish in #2754
• fix: resolve aliased catalogs by @universalmind303 in #2793
• feat: access sqlite dbs from object storage by @tychoish in #2772
• fix: Allow HTTP object store for json/bson etc. by @vrongmeal in #2784
• chore: benchmark read_json functions by @tychoish in #2794
• feat: change default 'json' extension handler by @tychoish in #2797
• feat: add nix derivation by @cmoog in #2727
• chore: add better regclass and oid cast support by @universalmind303 in #2795
• chore: update readme table and style by @tychoish in #2800
• fix: properly alias all sessions by @universalmind303 in #2802
• release: v0.9.2 by @greyscaled in #2801
• chore(deps): bump black from 23.3.0 to 24.3.0 by @tychoish in #2807
• chore(npm): fix napi argument by @greyscaled in #2808

Full Changelog: v0.9.1...v0.9.2

v0.9.1

v0.9.1 is a point release of mostly fixes and chores.

Notable Improvements

• cloud_upload can now be used in hybrid and remote contexts, such as with the CLI or Python and Node.js language bindings
• You can now create tables from excel

CREATE EXTERNAL TABLE t FROM excel OPTIONS (
    location = '<location>',     -- required
    has_header = 'true',         -- optional, 'true' or 'false'
    sheet_name = '<sheet_name>'  -- optional, defaults to first sheet in the workbook
);

Installing or Updating

CLI

To install or update just run the following command in the directory you want
the binary:

curl https://glaredb.com/install.sh | sh

Python

pip install glaredb

Node.js

npm install @glaredb/glaredb

What's Changed

• chore: make bson copy to options more consistent by @tychoish in #2704
• chore: add postgres dbt model to dbt tests by @talagluck in #2703
• perf: improve bson iteration performance by @tychoish in #2707
• feat: support for create external table excel <options> by @hailelagi in #2684
• fix(rpc): set is_cloud_instance remote session var by @greyscaled in #2715
• refactor: extensible function registry by @universalmind303 in #2699
• chore: add SQLite to list of supported Databases by @judahrand in #2717
• chore: dependency upgrade by @tychoish in #2726
• fix: python dependency specification by @tychoish in #2483
• chore(deps): bump mio from 0.8.9 to 0.8.11 by @dependabot in #2733
• fix: Iceberg for format v1 by @vrongmeal in #2718
• chore: pytest path cleanup and port management by @tychoish in #2734
• fix: Generic object store access implementation by @vrongmeal in #2709
• fix: Allow a process to acquire a lease its already acquired by @scsmithr in #2735
• release: v0.9.1 by @greyscaled in #2736
• hotfix(py v0.9.1): omit version specification by @tychoish in #2741

New Contributors

• @hailelagi made their first contribution in #2684
• @judahrand made their first contribution in #2717

Full Changelog: v0.9.0...v0.9.1

v0.9.0

Highlights

For highlights and details since 0.8, check out our full announcement: https://glaredb.com/blog/announcing-v0.9.0 🎉 !

Installing or Updating

CLI

To install or update just run the following command in the directory you want the binary:

curl https://glaredb.com/install.sh | sh

Python

pip install glaredb

Node.js

npm install @glaredb/glaredb

What's Changed

• fix(virtual_listing): signatures by @greyscaled in #2648
• chore(ci): build images from 'main' by @greyscaled in #2653
• chore(args): refactor imports by @greyscaled in #2656
• fix: infer excel files by extension by @tychoish in #2655
• chore(ci): move postgres tests and simplify tunnels testing by @tychoish in #2647
• chore: dependency upgrade for 2024-02-16 by @tychoish in #2669
• Feat: Create External Table support for Excel by @hemanth94 in #2652
• feat: SQLite data source by @vrongmeal in #2568
• fix: excel files should default to having a header like csv files by @tychoish in #2672
• fix: move supervisor into metastore by @universalmind303 in #2674
• chore: parametrize dbt test, add test case by @talagluck in #2673
• feat(read_ file funcs): infer from compressed formats by @hemanth94 in #2639
• chore(infer_func_for_file): remove dead code by @greyscaled in #2677
• feat: cloud_upload(<filename>) table func by @greyscaled in #2670
• feat: const aliases and read_xlsx alias by @tychoish in #2657
• fix: Alter table renames view successfully by @vrongmeal in #2678
• fix: report time display for DDL commands by @hemanth94 in #2681
• feat: cosine_similarity function by @universalmind303 in #2680
• feat: add object store table support for bson format data by @tychoish in #2600
• chore: reduce caching overhead for build steps by @tychoish in #2686
• feat: add support for reading local parquet metadata by @universalmind303 in #2685
• chore: upgrade dependencies by @tychoish in #2698
• chore(cloud_upload): improve err msg by @greyscaled in #2705
• chore: pytest hacking by @tychoish in #2606
• release: v0.9.0 by @greyscaled in #2711

Full Changelog: v0.8.5...v0.9.0

v0.8.5

v0.8.5 is a point release of mostly fixes and chores

Installing or Updating

CLI

To install or update just run the following command in the directory you want the binary:

curl https://glaredb.com/install.sh | sh

Python

pip install glaredb

Node.js

npm install @glaredb/glaredb

What's Changed

• fix: fixedsizelist in delta by @universalmind303 in #2607
• fix: add pg_rewrite and pg_depend tables by @talagluck in #2612
• chore(publish nodejs): set working-dir by @greyscaled in #2611
• fix(deps): remove dynamic linking to 'lzma' by @universalmind303 in #2631
• feat: support inserts into mongodb by @tychoish in #2605
• feat: lance insert_into support by @tychoish in #2613
• fix(information_schema): identity_cycle, is_updatable by @greyscaled in #2616
• chore: upgrade dependencies by @tychoish in #2632
• fix: create view nested query by @universalmind303 in #2637
• chore(deps): bump libgit2-sys from 0.16.1+1.7.1 to 0.16.2+1.7.2 by @dependabot in #2638
• fix(metastore): make builtin schemas readonly by @greyscaled in #2617
• fix: Unsigned integer types being cast to integers by @hemanth94 in #2640
• fix: make insert plan schema correct and clear by @tychoish in #2642
• release: v0.8.5 by @greyscaled in #2646

Full Changelog: v0.8.4...v0.8.5

v0.8.4

v0.8.4 is a point release of mainly fixes and chores, notably with some performance improvements (see: #2584, #2601). Refer to the commits below for more detail.

Installing or Updating

CLI

To install or update just run the following command in the directory you want the binary:

curl https://glaredb.com/install.sh | sh

Python

pip install glaredb --upgrade

What's Changed

• feat: add openai_embed function by @universalmind303 in #2538
• fix: options keyword is required before options block by @tychoish in #2445
• fix: ci cleanup cache command by @tychoish in #2552
• chore: disable regular benchmark runs by @tychoish in #2553
• ci: python build names by @greyscaled in #2559
• ci: fix pypi release by @universalmind303 in #2557
• fix: parquet over http by @universalmind303 in #2565
• chore(cloudauth): remove unused fields by @greyscaled in #2567
• chore(deps): Bump rust deps by @scsmithr in #2582
• chore(deps): bump slackapi/slack-github-action from 1.24.0 to 1.25.0 by @dependabot in #2577
• fix: predicate pushdowns for supported filetypes by @universalmind303 in #2584
• fix: bson timestamp type casting by @tychoish in #2593
• chore(release): atomic build and push jobs by @tychoish in #2564
• chore: df 35 upgrade by @universalmind303 in #2599
• fix: local file path resolution for json and bson by @tychoish in #2594
• fix: optimize ddl subqueries by @universalmind303 in #2601
• chore: add dbt test and fixtures by @talagluck in #2580
• chore: increase the history length for cli by @universalmind303 in #2608
• release: v0.8.4 by @greyscaled in #2595

New Contributors

• @talagluck made their first contribution in #2580

Full Changelog: v0.8.3...v0.8.4

v0.8.3

v0.8.3 is a point release of mainly fixes and chores. Refer to the commits below for more detail.

Notable fixes

• Postgres: Fixed an issue reading 0 timestamps (0001-01-01 00:00:00)

Installing or Updating

CLI

To install or update just run the following command in the directory you want the binary:

curl https://glaredb.com/install.sh | sh

Python

pip install glaredb

Node.js

npm install @glaredb/glaredb

What's Changed

• chore(binary build): slack alert by @greyscaled in #2537
• chore: ci reorganize dag by @tychoish in #2529
• feat: streaming json document/array data by @tychoish in #2494
• fix: normalize cache keys by @tychoish in #2539
• fix: Use actual precision for clickhouse data types by @vrongmeal in #2528
• fix: Use microsecond precision for postgres datasource by @vrongmeal in #2530
• chore: improve cache utility by @tychoish in #2540
• chore: populate shareable cache on main by @tychoish in #2541
• chore: ci clean up caches after a PR merges by @tychoish in #2542
• fix: add ci cache cleanup credential by @tychoish in #2544
• fix: update release macos runners by @tychoish in #2543
• fix: pr cleanup action jq operation by @tychoish in #2546
• release: v0.8.3 by @greyscaled in #2548

Full Changelog: v0.8.2...v0.8.3

v0.8.2

v0.8.2 is a point release of fixes and chores. Refer to the commits below for more detail.

Cassandra Update

We now support Cassandra data sources with PasswordAuthenticator.

-- Add an external table
CREATE EXTERNAL TABLE my_cassandra_table FROM cassandra OPTIONS (
    host = '<host>',
    keyspace = '<keyspace>',
    table = '<table>',
    username = '[username]',
    password = '[password]'
);

-- Add an external database
CREATE EXTERNAL DATABASE my_cassandra_db FROM cassandra OPTIONS (
    host = '<hostname>:<port>',
    username = '[username]',
    password = '[passowrd]'
);

-- Table function
SELECT count(*) FROM read_cassandra('<host>', '<keyspace>', '<table>', '[username]', '[password]');

Installing or Updating

CLI

To install or update just run the following command in the directory you want the binary:

curl https://glaredb.com/install.sh | sh

Python

pip install glaredb

Node.js

npm install @glaredb/glaredb

What's Changed

• feat: Notices on transactions by @scsmithr in #2443
• chore(ci): pin just version by @tychoish in #2442
• feat: copy to support for lance by @tychoish in #2342
• feat: pg_catalog.version(), standard_conforming_strings session var by @scsmithr in #2448
• chore(deps): Bump rust deps by @scsmithr in #2459
• chore(deps): bump actions/cache from 3 to 4 by @dependabot in #2449
• fix: $(find . -name "*.rs" | xargs chmod 0644) by @tychoish in #2463
• chore: Remove timeouts from CLI tests that don't need them by @scsmithr in #2469
• chore: Split out term width function into its own crate by @scsmithr in #2471
• chore: Get most of the rpc tests working by @scsmithr in #2472
• chore: Remove unneeded create credential message by @scsmithr in #2474
• fix: psql tab completion by @universalmind303 in #2464
• chore(deps): bump shlex from 1.2.0 to 1.3.0 by @dependabot in #2477
• fix: actually drop delta tables by @universalmind303 in #2415
• fix: rename "pg_table" to "pg_tables" by @universalmind303 in #2480
• chore: update contributing with pytest info by @tychoish in #2479
• ci: fix timings.csv by @universalmind303 in #2460
• fix: Execute clickhouse query in stream (during execution) by @vrongmeal in #2482
• chore: Upgrade arrow/datafusion/deltalake/lance by @vrongmeal in #2429
• test(python): create table from pandas df by @greyscaled in #2488
• fix: add auth for cassandra by @tychoish in #2478
• fix: download protoc without just during the release by @tychoish in #2476
• chore: forbid wildcard imports + additional rustfmt by @tychoish in #2466
• fix: ci task names by @tychoish in #2497
• chore: Move distexec into a different crate by @vrongmeal in #2489
• chore(deps): Bump rust deps by @scsmithr in #2513
• chore(deps): bump actions/cache from 3 to 4 by @dependabot in #2498
• chore: Add npm badge to readme by @scsmithr in #2515
• chore: downsize some ci builders by @tychoish in #2514
• chore: Run more prql integration tests by @scsmithr in #2516
• fix: add pg_matview table by @universalmind303 in #2523
• ci: explicit caching and even smaller runners by @tychoish in #2518
• fix: bson should handle nulls by @tychoish in #2485
• chore: skip sqlserver+flight combination. by @tychoish in #2526
• release: v0.8.2 by @greyscaled in #2532

Full Changelog: v0.8.1...v0.8.2

v0.8.1

v0.8.1 is a point release of fixes and chores. Refer to the commits below for more detail.

Installing or Updating

CLI

To install or update just run the following command in the directory you want the binary:

curl https://glaredb.com/install.sh | sh

Python

pip install glaredb

Node.js

npm install @glaredb/glaredb

What's Changed

• feat: Support Boolean datatypes after klickhouse switch by @vrongmeal in #2407
• fix: Pushdown SQL Server WHERE clause by @vrongmeal in #2316
• chore: Schedule python/node bindings builds by @scsmithr in #2404
• add commit-sha to cli by @universalmind303 in #2411
• chore(deps): Bump deps by @scsmithr in #2426
• Fix : Adding keywords for PRQL for higlighting by @hemanth94 in #2413
• update readme by @universalmind303 in #2397
• chore: move slt runner into main binary by @tychoish in #2401
• fix: Infer and scan BSON files from file path by @vrongmeal in #2410
• chore: Remove --exclude testing from justfile by @scsmithr in #2428
• fix: Error instead of panic when converting pg timestamps to arrow by @vrongmeal in #2430
• implement pg_catalog.pg_type table by @universalmind303 in #2392
• chore: Remove openssh from slt crate cargo.toml by @scsmithr in #2432
• chore: Post release build results to slack by @scsmithr in #2433
• ci: move minio into its own job by @universalmind303 in #2435
• release: v0.8.1 by @greyscaled in #2439

Full Changelog: v0.8.0...v0.8.1

v0.8.0

Highlights

Cassandra Support

Cassandra is now supported as an external data source.

-- external table
CREATE EXTERNAL TABLE <table_name> FROM cassandra OPTIONS (
    host = '<host>',
    keyspace = '<keyspace>',
    table = '<table>'
);

-- external database
CREATE EXTERNAL DATABASE <db_name> FROM cassandra OPTIONS (
    host = '<host>',
);

-- table function
SELECT * FROM read_cassandra('<host>', '<keyspace>', '<table>')

Clickhouse Support

GlareDB now supports ClickHouse as an external data source. Learn more in our ClickHouse docs.

Table Functions

• read_sqlserver
• read_bson
• read_clickhouse

Initial FlightSQL Protocol Support

glaredb server now includes support for the experimental FlightSQL protocol.

Enable it with:

glaredb server --enable-flight-api

then use your favorite FlightSQL client to connect to the server.

Installing or Updating

CLI

To install or update just run the following command in the directory you want the binary:

curl https://glaredb.com/install.sh | sh

Python

pip install glaredb

Node.js

npm install @glaredb/glaredb

What's Changed

• fix: Hybrid planning when the root plan has local preference by @vrongmeal in #2183
• chore(ci): nodejs bindings yarn version mismatch by @greyscaled in #2212
• chore: consolidating path validation logic by @seve-martinez in #2200
• chore: Fix error message for invalid copy-to by @vrongmeal in #2216
• chore: remove py-glaredb cruft by @tychoish in #2215
• chore: refactor udfs into sqlbuiltins by @universalmind303 in #2214
• feat: add --logs flag to local command by @Lilit0x in #2187
• chore(deps): Bump deps by @scsmithr in #2233
• chore: small cleanup to ComputeServer by @universalmind303 in #2232
• chore: Add go package option to protobuf definitions by @scsmithr in #2241
• chore: Remove logical plan serialization by @scsmithr in #2244
• chore(deps): bump actions/setup-python from 4 to 5 by @dependabot in #2229
• fix: Properly exit SimpleExecuteQueryStream on stream end by @scsmithr in #2243
• Better formatting for FixedSizeList (#2058) by @hemanth94 in #2223
• feat: flightsql initial support by @universalmind303 in #2219
• fix: some broken tests by @universalmind303 in #2249
• feat: allow for tests over flightsql by @universalmind303 in #2240
• fix: truncate long floats in cli formatting for display by @hemanth94 in #2253
• fix: cases sensitivity for statement options keywords by @seve-martinez in #2231
• chore: Detect client disconnect in pg proxy by @scsmithr in #2256
• chore(deps): Bump deps by @scsmithr in #2271
• feat: high level integration testing framework by @tychoish in #2258
• chore(pg proxy): update cloud cert by @greyscaled in #2272
• chore: Bump rust to 1.74.1 by @scsmithr in #2275
• fix: Skip external SLTs for external contributors by @vrongmeal in #2247
• feat(bson): add COPY TO support for writing bson files by @tychoish in #2166
• chore: isolate execution of snowflake integration tests by @tychoish in #2278
• fix: Don't parse version-hint as a number for iceberg tables by @vrongmeal in #2281
• chore(deps): upgrade actions by @tychoish in #2282
• chore(deps): bump zerocopy from 0.7.18 to 0.7.31 by @dependabot in #2257
• chore(deps): bump rustls-pemfile from 1.0.4 to 2.0.0 by @dependabot in #2190
• chore(deps): bump webpki-roots from 0.25.3 to 0.26.0 by @dependabot in #2191
• fix: filter unknown oids for Postgres types by @hemanth94 in #2283
• feat: encrypted flight api via cloud proxy by @universalmind303 in #2279
• chore: Default to compact logs by @scsmithr in #2287
• chore(deps): Bump deps by @scsmithr in #2297
• feat(bson): add table function for bson files by @tychoish in #2202
• fix: Hacky fix for running the experimental scheduler when spawning task by @vrongmeal in #2284
• chore: Remote fn runtime_preference() from TableFunc trait. by @vrongmeal in #2288
• feat: Add read_sqlserver table returning function by @vrongmeal in #2310
• chore: Add .clang-format by @scsmithr in #2312
• fix: bson type handling cleanup by @tychoish in #2313
• feat: cli highlighting improvements. by @universalmind303 in #2307
• feat: Support virtual listing for SQL server. by @vrongmeal in #2311
• feat: Clickhouse data source by @scsmithr in #2300
• fix: scalar udf function argument handling by @tychoish in #2304
• feat: add non-cryptographic hash function by @tychoish in #2298
• feat: partition results helper by @tychoish in #2299
• test: Add datatypes test for SQL Server by @vrongmeal in #2317
• feat: Add support for Azure data dir by @vrongmeal in #2319
• test: Add SLTs for sqlserver <> virtual listing by @vrongmeal in #2320
• chore(deps): Bump deps by @scsmithr in #2331
• chore: Bump rust to 1.75 by @scsmithr in #2335
• chore: Make bucket name required in storage config by @vrongmeal in #2336
• Revert "chore: Make bucket name required in storage config" by @vrongmeal in #2337
• fix: bson builder should handle schema flexibly by @tychoish in #2334
• fix: bson type inference fixes by @tychoish in #2339
• chore: mongo -> mongodb in symbol naming by @tychoish in #2343
• fix: complex bson types by @tychoish in #2341
• feat: read_cassandra table function by @universalmind303 in #2344
• chore(deps): Bump deps by @scsmithr in #2359
• chore: Allow specifying aliases for table functions by @scsmithr in #2364
• feat: Allow specifying alt delimiter when reading csv by @scsmithr in #2365
• chore: Misc builtin function refactoring by @scsmithr in #2367
• feat: Clickhouse followup items by @vrongmeal in #2340
• feat: external table for cassandra by @universalmind303 in #2361
• chore: Add alias map for functions by @scsmithr in #2370
• ci: speed up ci by introducing precompiled slt binary by @universalmind303 in #2363
• fix: add signature for read_excel by @universalmind303 in #2376
• feat: cassandra external database by @universalmind303 in #2374
• fix: excel options by @universalmind303 in #2377
• feat: Allow reading csv files without a header by @scsmithr in #2380
• release: v0.8.0 by @tychoish in #2381
• Revert "release: v0.8.0" by @greyscaled in #2383
• chore: Add slt for asserting builtin tables by @scsmithr in #2387
• chore: Switch to klickhouse by @scsmithr in #2388
• release: v0.8.0 by @greyscaled in #2391
• chore: add check for dynamic linking on linux by @tychoish in #2386
• Revert "release: v0.8.0" by @greyscaled in #2398
• ci: downgrade upload-artifact v4->v3 by @greyscaled in https://github.com/Glar...

v0.7.1

Highlights

Lance Support

GlareDB now offers support for Lance.

Lance is a Modern columnar data format optimized for ML and LLM workflows.

SELECT * FROM lance_scan('path/to/file.lance');

Documentation Improvements

We have made it easier to find the information you need, where you need it most.
Our functions catalog now contain 2 additional columns description and example.
These contain a short description of the function and an example of how to use it.

SELECT
  function_name,
  example,
  description
FROM glare_catalog.functions

Installing or Updating

CLI

To install or update just run the following command in the directory you want
the binary:

curl https://glaredb.com/install.sh | sh

Python

pip install glaredb

What's Changed

New Features

• lance_scan function by @universalmind303 in #2108
• bump prql version by @universalmind303 in #2111
• Allow trailing commas in select list by @scsmithr in #2117
• Remove 'parquet' feature from deltalake dep by @scsmithr in #2121
• add lance as external table by @universalmind303 in #2120
• Bump deps by @scsmithr in #2131
• Send metrics for hybrid exec queries (rpc) by @vrongmeal in #2103
• make tokio a workspace dependency by @tychoish in #2139
• Enable corepack before setting up node by @scsmithr in #2146
• Send user_id in RPC execute request to collect metrics by @vrongmeal in #2140
• Bump deps by @scsmithr in #2159
• Add 'access_mode' column for databases and tables by @scsmithr in #2162
• Remove background jobs by @vrongmeal in #2148
• Split session catalog out into its own crate by @scsmithr in #2167
• Scheduling by @scsmithr in #1924
• Expose catalog through table func context by @scsmithr in #2172
• rename CREATE CREDENTIALS to CREATE CREDENTIAL by @Lilit0x in #2180
• Send query text when executing query over RPC by @vrongmeal in #2179
• alias read_parquet, read_njson, read_csv to their scan counterpart by @Lilit0x in #2185
• builtin descriptions & examples by @universalmind303 in #2178
• Simple query interface by @scsmithr in #2201
• clean up lint target by @tychoish in #2204
• bump google-github-actions/auth from 1 to 2 by @dependabot in #2189
• Bump deps by @scsmithr in #2206
• kdl_match function by @tychoish in #2145
• Move table func trait by @scsmithr in #2208
• improve+cleanup func param processing by @tychoish in #2203
• Data source cache by @scsmithr in #2164

Bug Fixes

• add better error message for invalid "version" cmd, and invalid sql files by @universalmind303 in #2110
• stdin piping by @universalmind303 in #2137
• case sensitivity in create table by @universalmind303 in #2141
• improvements to schema inference by @tychoish in #2143
• Properly compute stats for globbed paths by @tychoish in #2171
• Mark 'default' database as READ_WRITE by @vrongmeal in #2173
• create or replace for credentials by @universalmind303 in #2134
• handle integer overflows by @tychoish in #2205

New Contributors

• @Lilit0x made their first contribution in #2180

Full Changelog: v0.7.0...v0.7.1