docs: add tutorial with examples of sql null handling

docs/querying/sql-data-types.md

@@ -140,23 +140,38 @@ as regular JSON arrays instead of in stringified form.
 
 ## NULL values
 
+By default, Druid treats NULL values closely to the SQL standard.
+In the default mode:
+- numeric NULL is permitted
+- NULL values and empty strings are not equal.
+
+This manner of null handling applies to both storage and queries.
 The [`druid.generic.useDefaultValueForNull`](../configuration/index.md#sql-compatible-null-handling)
-runtime property controls Druid's NULL handling mode. For the most SQL compliant behavior, set this to `false` (the default).
-
-When `druid.generic.useDefaultValueForNull = false` (the default), NULLs are treated more closely to the SQL standard. In this mode,
-numeric NULL is permitted, and NULLs and empty strings are no longer treated as interchangeable. This property
-affects both storage and querying, and must be set on all Druid service types to be available at both ingestion time
-and query time. There is some overhead associated with the ability to handle NULLs; see
-the [segment internals](../design/segments.md#handling-null-values) documentation for more details.
-
-When `druid.generic.useDefaultValueForNull = true` (deprecated legacy mode), Druid treats NULLs and empty strings
-interchangeably, rather than according to the SQL standard. In this mode Druid SQL only has partial support for NULLs.
-For example, the expressions `col IS NULL` and `col = ''` are equivalent, and both evaluate to true if `col` contains
-an empty string. Similarly, the expression `COALESCE(col1, col2)` returns `col2` if `col1` is an empty string. While
-the `COUNT(*)` aggregator counts all rows, the `COUNT(expr)` aggregator counts the number of rows where `expr` is
-neither null nor the empty string. Numeric columns in this mode are not nullable; any null or missing values are
-treated as zeroes. This was the default prior to Druid 28.0.0, but will be removed in a future release so that Druid
-always behaves in an SQL compatible manner.
+runtime property controls Druid's NULL handling mode. For the most SQL compliant behavior, maintain the default value of `false`.
+
+There is some performance impact for null handling. see [segment internals](../design/segments.md#handling-null-values) for more information.
+For examples of null handling, see the [null handling tutorial](../tutorials/tutorial-ansi-sql-null.md).
+
+### Legacy null handling mode
+
+:::info
+To ensure Druid always behaves in an SQL compatible manner, this mode will be removed in a future release.
+:::
+
+You can set `druid.generic.useDefaultValueForNull = true` to revert to Druid's deprecated legacy null handling mode, the default for Druid 27.0.0 and prior releases.
+
+When running in the deprecated legacy mode, Druid treats NULL values and empty strings interchangeably.
+In this mode:
+- Druid does not distinguish between empty strings and nulls.
+- Druid SQL only has partial support for NULLs.
+- Numeric columns are not nullable; null or missing values are treated as 0. 
+
+For example, the following expressions are equivalent:
+- col IS NULL
+- col = ''
+Both evaluate to true if col contains an empty string.
+Similarly, the expression COALESCE(col1, col2) returns col2 if col1 is an empty string.
+The COUNT(*) aggregator counts all rows but the COUNT(expr) aggregator counts the number of rows where expr is neither null nor the empty string.
 
 ## Boolean logic
 

docs/tutorials/tutorial-ansi-sql-null.md

@@ -0,0 +1,205 @@
+---
+id: tutorial-ansi-sql-null
+title: Null handling tutorial
+sidebar_label: Handling null values
+description: Introduction to three-valued null handling
+---
+
+<!--
+  ~ Licensed to the Apache Software Foundation (ASF) under one
+  ~ or more contributor license agreements.  See the NOTICE file
+  ~ distributed with this work for additional information
+  ~ regarding copyright ownership.  The ASF licenses this file
+  ~ to you under the Apache License, Version 2.0 (the
+  ~ "License"); you may not use this file except in compliance
+  ~ with the License.  You may obtain a copy of the License at
+  ~
+  ~   http://www.apache.org/licenses/LICENSE-2.0
+  ~
+  ~ Unless required by applicable law or agreed to in writing,
+  ~ software distributed under the License is distributed on an
+  ~ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+  ~ KIND, either express or implied.  See the License for the
+  ~ specific language governing permissions and limitations
+  ~ under the License.
+  -->
+
+This tutorial introduces the basic concepts of three-valued null handling in Apache Druid.
+The three-valued logic primarily affects filters using the logical NOT operation on columns with NULL values.
+This applies to both query and ingestion time filtering.
+
+Druid differentiates between the following:
+- `null` values
+- empty strings: `""`
+- a record with no data
+- empty numerical records
+- 0
+
+## Prerequisites
+
+Before starting this tutorial, download and run Apache Druid on your local machine as described in
+the [single-machine quickstart](index.md).
+
+The tutorial assumes you are familiar with using the [Query view](./tutorial-sql-query-view.md) to ingest and query data.
+
+## Load data with null values
+
+The tutorial loads some data with null values for various data types as follows:
+
+```json
+{"date": "1/1/2024 1:02:00","title": "example_1","string_value": "some_value","numeric_value": 1,"boolean_value_1": true,"boolean_value_2": "1"}
+{"date": "1/1/2024 1:03:00","title": "example_2","string_value": "another_value","numeric_value": 2,"boolean_value_1": false,"boolean_value_2": "0"}
+{"date": "1/1/2024 1:04:00","title": "example_3","string_value": "", "numeric_value": "", "boolean_value_1": "","boolean_value_2":""}
+{"date": "1/1/2024 1:05:00","title": "example_4","string_value": null, "numeric_value": null, "boolean_value_1": null,"boolean_value_2":null}
+```
+
+Run the following query in the Druid Console to load the data:
+
+```sql
+REPLACE INTO "inline_data--abcdef" OVERWRITE ALL
+WITH "ext" AS (
+  SELECT *
+  FROM TABLE(
+    EXTERN(
+      '{"type":"inline","data":"{\"date\": \"1/1/2024 1:02:00\",\"title\": \"example_1\",\"string_value\": \"some_value\",\"numeric_value\": 1,\"boolean_value_1\": true,\"boolean_value_2\": \"1\"}\n{\"date\": \"1/1/2024 1:03:00\",\"title\": \"example_2\",\"string_value\": \"another_value\",\"numeric_value\": 2,\"boolean_value_1\": false,\"boolean_value_2\": \"0\"}\n{\"date\": \"1/1/2024 1:04:00\",\"title\": \"example_3\",\"string_value\": \"\", \"numeric_value\": \"\", \"boolean_value_1\": \"\",\"boolean_value_2\":\"\"}\n{\"date\": \"1/1/2024 1:05:00\",\"title\": \"example_4\",\"string_value\": null, \"numeric_value\": null, \"boolean_value_1\": null,\"boolean_value_2\":null}"}',
+      '{"type":"json"}'
+    )
+  ) EXTEND ("date" VARCHAR, "title" VARCHAR, "string_value" VARCHAR, "numeric_value" BIGINT, "boolean_value_1" VARCHAR, "boolean_value_2" BIGINT)
+)
+SELECT
+  TIME_PARSE("date", 'd/M/yyyy H:mm:ss') AS "__time",
+  "title",
+  "string_value",
+  "numeric_value",
+  CAST("boolean_value_1" AS BOOLEAN) AS "boolean_value_1",
+  CAST("boolean_value_2" AS BOOLEAN) AS "boolean_value_2"
+FROM "ext"
+PARTITIONED BY DAY
+```
+
+After Druid finishes loading the data, run the following query to see the data in Druid:
+
+```sql
+SELECT * FROM null_data_example
+```
+
+|`__time`|`title`|`string_value`|`numeric_value`|`boolean_value_1`|`boolean_value_2`|
+|---|---|---|---|---|---|
+|`"2024-01-01T01:02:00.000Z"`|`"example_1`|`"some_value"`|1|1|1|
+|`"2024-01-01T01:03:00.000Z"`|`"example_2`|`"another_value"`|2|0|0|
+|`"2024-01-01T01:04:00.000Z"`|`"example_3"`|`empty`|`null`|`null`|`null`|
+|`"2024-01-01T01:05:00.000Z"`|`"example_4"`|`null`|`null`|`null`|`null`|
+
+Note the follwoing about the result set:
+- Druid stores the empty string value for `example_3` as an empty, but the other values are null.
+- Druid uses `0/1` for both BOOLEAN type columns.
+
+### Compare to two-valued logic
+
+Before the 28.0 release, Apache Druid used two-valued logic for null handling. The following query shows the results for `SELECT * FROM null_data_example` using the legacy logic.
+
+|`__time`|`title`|`string_value`|`numeric_value`|`boolean_value_1`|`boolean_value_2`|
+|---|---|---|---|---|---|
+|`"2024-01-01T01:02:00.000Z"`|`"example_1`|`"some_value"`|1|1|1|
+|`"2024-01-01T01:03:00.000Z"`|`"example_2`|`"another_value"`|2|0|0|
+|`"2024-01-01T01:04:00.000Z"`|`"example_3"`|`""`|0|0|0|
+
+Note that the `null` values for the BIGINT type column and for both BOOLEAN type columns is stored as `0`.
+
+## String query example
+
+
+
+```sql
+SELECT COUNT(*)
+FROM "null_data_example2"
+WHERE "string_value" != 'some_value'
+```
+
+```sql
+SELECT "string_value",
+      COUNT(*) AS count_all_rows,
+      COUNT("string_value") AS count_values
+FROM "inline_datafix"
+GROUP BY 1
+```
+
+If you dont want to count empty strings or nulls, filter them out. For example:
+
+
+
+### Compare to two-valued logic
+
+The same query using two valued logic returns a value of 2.
+
+## Numeric query example
+
+Using three-valued logic, Druid treats `null` values as "unknown" and does not count them in comparisons.
+
+```sql
+SELECT COUNT(*)
+FROM "null_data_example"
+WHERE "numeric_value" < 2
+```
+
+Druid returns 1. The `null` value for `example_3` is excluded.
+
+### Compare to two-valued logic
+
+The same query using two-valued logic returns a value of 2.
+
+## Boolean query examples 
+
+```sql
+SELECT "boolean_value_1",
+COUNT(*) AS count_of
+FROM "null_data_example"
+GROUP BY 1
+```
+
+Druid returns the following:
+
+|`boolean_value_1`|`count_of`|
+|---|---|
+|`null`|1|
+|0|1|
+|1|1|
+
+
+```sql
+SELECT "boolean_value_2",
+COUNT(*) AS count_of
+FROM "null_data_example"
+GROUP BY 1
+```
+
+Druid returns the following:
+
+|`boolean_value_1`|`count_of`|
+|---|---|
+|`null`|1|
+|0|1|
+|1|1|
+
+### Compare to two-valued logic
+
+These queries return the following under two-valued logic:
+
+|`boolean_value_1`|`count_of`|
+|---|---|
+|0|2|
+|1|1|
+
+|`boolean_value_2`|`count_of`|
+|---|---|
+|0|2|
+|1|1|
+
+
+
+
+
+
+
+
+