Skip to content


Choose a tag to compare
@bluestreak01 bluestreak01 released this 29 May 14:11
· 6 commits to master since this release

This one's a big one!

In brief, to get you to the goods quicker:

  • New VARCHAR data type over STRING default for ++compression & ++performance
  • 50% improvement over already fast SQL query performance
  • Data compression via system-level ZFS, previously Enterprise-only

And more...

Upgrade as soon as you can. Or start fresh and build out your use case.

If you're still reading, onwards for more detail, breaking changes, plus the PR breakdown.

New features 🐣

  • VARCHAR: Introduces the VARCHAR data type and a new type driver API. VARCHAR offers UTF8 encoded strings of variable length as opposed to the UTF16 and fixed length encoding found in STRING, for better compression and significantly improved performance.
    • Maximum size of a single VARCHAR field size is 268 MB.
    • A size of a VARCHAR column in a single partition is limited by 218 TB or disk size, whichever is smaller.
    • STRING is now considered a legacy type.
  • Data compression: Data compression for OSS! Full support for the ZFS file system and system-level data compression.
  • ALTER COLUMN type: Use ALTER TABLE SQL to convert column types.
    • Migrate between STRING, SYMBOL, and VARCHAR data types.

Breaking changes 💥

  • GLIBC upgrade: Upgraded GLIBC to a minimum of 2.28-amd64 and 2.33-aarch64.
  • ILP and TextImport changes: ILP and TextImport now create VARCHAR instead of STRING columns.
  • Jemalloc allocator: The default allocator is now jemalloc. Later updates include disabling jemalloc as the default allocator and removing jemalloc from no-jre builds.

Performance 🚀

  • SQL performance improvement: Average SQL performance improves by 50%. 🔥
  • Memory allocation limit: New memory allocation limit prevents OS OOM handler terminations in many cases.

Bug fixes 🥾

  • SQL: Fixed issues with SAMPLE BY and GROUP BY not grouping when used with column aliases. Fixed WHERE filters with FLOAT columns null comparison.
  • Core: Addressed trailing boundary processing of multipart HTTP requests. Fixed hour parsing and formatting in the 1-12 and 1-24 conventions.

Pull requests :octocat:

  • perf(sql): improve performance of length(VARCHAR) by @mtopolnik in #4442
  • perf(sql): reduce query latency when querying tables under heavy writing by @ideoma in #4500
  • perf(sql): statistics-based map pre-sizing for parallel group by by @puzpuzpuz in #4517
  • perf(sql): speed up GROUP BY queries with ORDER BY and LIMIT by @puzpuzpuz in #4523
  • perf(sql): optimize ORDER BY with LIMIT by @jerrinot in #4529
  • perf(sql): adaptive work stealing for parallel query execution on multicore machines by @puzpuzpuz in #4542
  • feat(core): breaking change💥 - upgrade GLIBC to minimum of 2.28-amd64 and 2.33-aarch64 by @bluestreak01 in #4449
  • feat(core): use jemalloc allocator by default by @bluestreak01 in #4451
  • feat(sql): implement level 2 order book price function by @nwoolmer in #4462
  • feat(core): add ZFS support by @puzpuzpuz in #4507
  • feat(sql): support usage of ALTER table SQL to convert column types by @ideoma in #4413
  • feat(ilp): breaking change💥 - ILP and TextImport create VARCHAR instead of STRING columns by @jerrinot in #4476
  • feat(build): new ami based on amazon linux 2023 💥 - breaking change by @mariusgheorghies in #4519
  • feat(core): resume all suspended WAL tables on start up by @ideoma in #4513
  • feat(core): add RSS memory limit to prevent OOM kills from the OS by @mtopolnik in #4480
  • feat(sql): introduce global select query cache per interface by @puzpuzpuz in #4525
  • feat(core): improve SQL error logging by @bluestreak01 in #4532
  • fix(sql): fix sample by query compilation error when using as-of join by @amunra in #4452
  • fix(core): fix trailing boundary processing of multipart HTTP requests by @glasstiger in #4466
  • fix(http): remove ddl:ok response for insert query by @siddharth0815 in #4472
  • fix(ui): web console to load regardless of pgwire disabled by @glasstiger in #4484
  • fix(sql): fix WHERE filters with FLOAT columns null comparison by @ideoma in #4487
  • fix(sql): minor bug fixes in ExpressionParser by @sivukhin in #4441
  • fix(sql): IN with NULLs and STRINGs sometimes may return the wrong result by @nwoolmer in #4499
  • fix(sql): fix SAMPLE BY and GROUP BY not grouping when used with column aliases by @ideoma in #4503
  • fix(core): fix hour parsing and formatting in the 1-12 and 1-24 conventions by @jerrinot in #4506
  • fix(sql): fix several bugs in aggregation (avg/sum/first_value) window functions by @sivukhin in #4429
  • fix(sql): split_part returning old data when no match is found by @nwoolmer in #4493
  • fix(sql): breaking💥 fix coalesce function correctness by @bluestreak01 in #4490
  • fix(core): fix potential data loss when non-WAL table dropped and created with same name by @ideoma in #4533
  • fix(sql): breaking change💥 - fix expression parser operator precedence by @sivukhin in #4443
  • fix(core): fix transient query failure because of missing partition directory by @ideoma in #4611

New Contributors

Full Changelog: 7.4.2...8.0.0