[BREAKING] Switch to NULLS LAST as default null sorting order, instead of NULLS FIRST #7174
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR changes the default null sorting order from NULLS FIRST to NULLS LAST. That means that when ordering values without a specified null ordering clause, NULL values always end up after all other values:
NULLS LAST
is more intuitive when usingORDER BY
in conjunction withLIMIT
. WithNULLS FIRST
, the following queries returnNULL
values instead of the desired top-n results:With
NULLS LAST
, the desired values are returned instead:This fixes a common footgun/source of confusion for users.
SQL Standard & Other Systems
Unfortunately null ordering is extremely inconsistent across systems, and is not defined in the SQL standard if
NULLS FIRST
orNULLS LAST
are not explicitly provided.There are four different options for
NULL ordering
:NULLS FIRST
whenASC
,NULLS LAST
whenDESC
NULLS LAST
whenASC
,NULLS FIRST
whenDESC
NULLS FIRST
NULLS LAST
Previously we implemented option (3) - this PR switches us to option (4). This option seems the least problematic since it avoids the
top-n
pitfall described above for bothASC/DESC
ordering - and is consistent with more other systems, particularly systems in the analytics space.Other Systems
(1)
NULLS FIRST
whenASC
,NULLS LAST
whenDESC
MySQL, SQLite, BigQuery
(2)
NULLS LAST
whenASC
,NULLS FIRST
whenDESC
PostgreSQL, Snowflake
(3) Always
NULLS FIRST
DuckDB (current)
(4) Always
NULLS LAST
DuckDB (new), Presto/Trino, Pandas, ClickHouse
default_null_order
This PR also expands the
default_null_order
option so that it can correctly model all of these settings:In addition to the currently available settings:
In order to revert to the previous behavior of the system, the following setting can be used: