Skip to content

feat: implement ScalarFunction#set_special_handling and null_handling: keyword#1177

Merged
suketa merged 5 commits intomainfrom
feature/scalar-function-set-special-handling
Mar 24, 2026
Merged

feat: implement ScalarFunction#set_special_handling and null_handling: keyword#1177
suketa merged 5 commits intomainfrom
feature/scalar-function-set-special-handling

Conversation

@suketa
Copy link
Copy Markdown
Owner

@suketa suketa commented Mar 24, 2026

Summary

Closes #1176 (sub-issue of #1122).

Wraps duckdb_scalar_function_set_special_handling so Ruby scalar functions can receive nil for NULL inputs instead of DuckDB short-circuiting and returning NULL.

Changes

ext/duckdb/scalar_function.c

  • Add thin C wrapper _set_special_handling (private, 0-arg) — same pattern as _set_varargs

lib/duckdb/scalar_function.rb

  • Add public set_special_handling method (calls C wrapper, returns self for chaining)
  • Add null_handling: false keyword to ScalarFunction.create — when true, calls set_special_handling automatically
  • Add :any to SUPPORTED_TYPES so varargs_type= DuckDB::LogicalType::ANY is accepted

lib/duckdb/converter/int_to_sym.rb

  • Add 34 => :any to HASH_TYPES (was simply missing, caused Unknown type: 34 errors)

CHANGELOG.md

  • Document all three additions under # Unreleased

Usage

# Using the method directly
sf = DuckDB::ScalarFunction.new
sf.name = 'null_as_zero'
sf.add_parameter(DuckDB::LogicalType::INTEGER)
sf.return_type = DuckDB::LogicalType::INTEGER
sf.set_special_handling
sf.set_function { |v| v.nil? ? 0 : v }

# Using ScalarFunction.create
sf = DuckDB::ScalarFunction.create(
  name: :null_as_zero,
  return_type: :integer,
  parameter_type: :integer,
  null_handling: true
) { |v| v.nil? ? 0 : v }

Behaviour

Without set_special_handling With set_special_handling
Non-NULL input block called, receives value block called, receives value
NULL input block not called, returns NULL block called, receives nil

Tests

8 tests added covering:

  • Returns self for fluent chaining
  • Block receives nil for NULL inputs (single param, two params, varargs)
  • Non-NULL inputs still work normally
  • Block can explicitly return nil even with special handling
  • Canonical DuckDB C API null-count pattern with INTEGER varargs
  • Canonical DuckDB C API null-count pattern with ANY varargs
  • ScalarFunction.create with null_handling: false (default)
  • ScalarFunction.create with null_handling: true
  • ScalarFunction.create with null_handling: true + varargs

Summary by CodeRabbit

  • New Features

    • Scalar functions can opt into null-handling via a new null_handling option so registered blocks receive nil for NULL inputs.
    • Added a public method to enable special NULL input behavior (returns self for chaining).
    • varargs support extended to the ANY logical type.
  • Tests

    • Added coverage for ANY varargs, mixed-type varargs, and null-handling behaviors (default short-circuit vs. opt-in invocation).

suketa and others added 4 commits March 24, 2026 21:30
Add eight skipped test cases for the planned set_special_handling wrapper
around duckdb_scalar_function_set_special_handling (duckdb.h:3719).

The method marks a scalar function to receive NULL inputs directly,
bypassing DuckDB's default SQL NULL propagation that would otherwise
short-circuit the callback and return NULL for any NULL argument.

Tests cover:
- returns self for fluent chaining
- block receives nil for a single nullable parameter (coalesce pattern)
- block receives nil for any combination of two nullable parameters
- varargs + special handling with NULL coalescing
- non-NULL inputs still work correctly after set_special_handling
- block can explicitly return nil even with special handling enabled
- null_count function with INTEGER varargs — mirrors the canonical DuckDB
  C API test (capi_scalar_functions.cpp) using concrete types:
    my_null_count(40, 1, 3) → 0
    my_null_count(1, 42, NULL) → 1
    my_null_count(NULL, NULL, NULL) → 3
    my_null_count() → 0
- null_count function with ANY varargs — exact C API test replica,
  skipped until DuckDB::LogicalType::ANY is also exposed (issue #1122)

All tests are skipped with a message referencing issue #1122 until the
C extension method is implemented.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Wraps duckdb_scalar_function_set_special_handling (duckdb.h:3719).

By default DuckDB short-circuits scalar function callbacks and returns
NULL whenever any input row contains a NULL argument. Calling
set_special_handling disables that behaviour: the block is invoked even
for NULL inputs, receiving nil for each NULL argument, so the function
can implement its own NULL semantics (coalescing, counting NULLs, etc).

Changes:
- ext/duckdb/scalar_function.c: add thin C wrapper
  _set_special_handling (0-arg private method, same pattern as
  _set_varargs / _add_parameter)
- lib/duckdb/scalar_function.rb: add public set_special_handling that
  calls _set_special_handling and returns self; add :any to
  SUPPORTED_TYPES so varargs_type= accepts DuckDB::LogicalType::ANY
- lib/duckdb/converter/int_to_sym.rb: add 34 => :any to HASH_TYPES
  (the entry was simply missing, causing Unknown type: 34 errors)
- test/duckdb_test/scalar_function_test.rb: activate all 8 skeleton
  tests added in the previous commit (all skips removed)

Closes part of #1122.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
When null_handling: true is passed, set_special_handling is called on
the function so the block receives nil for NULL inputs instead of DuckDB
short-circuiting and returning NULL.

  sf = DuckDB::ScalarFunction.create(
    name: :null_as_zero,
    return_type: :integer,
    parameter_type: :integer,
    null_handling: true
  ) { |v| v.nil? ? 0 : v }

Defaults to false (standard SQL NULL propagation unchanged).

Tests added:
- default is false (NULL input still returns NULL)
- null_handling: true coalesces NULL to non-NULL
- null_handling: true works with varargs (null_count pattern)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@coderabbitai
Copy link
Copy Markdown

coderabbitai bot commented Mar 24, 2026

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 667021c0-51b3-4727-a7ef-51817cd16f63

📥 Commits

Reviewing files that changed from the base of the PR and between 41a7b14 and be653df.

📒 Files selected for processing (1)
  • test/duckdb_test/scalar_function_test.rb
🚧 Files skipped from review as they are similar to previous changes (1)
  • test/duckdb_test/scalar_function_test.rb

📝 Walkthrough

Walkthrough

This PR adds an opt-in null-handling mode to ScalarFunction (Ruby + C binding) so registered blocks receive nil for NULL inputs, and extends varargs support with :any logical type. Tests exercise varargs/ANY and null-handling behavior.

Changes

Cohort / File(s) Summary
Changelog
CHANGELOG.md
Documents null_handling: parameter, set_special_handling, and :any varargs support.
Native C binding
ext/duckdb/scalar_function.c
Adds private binding _set_special_handling -> duckdb_scalar_function_set_special_handling and registers it in init.
Ruby type mapping
lib/duckdb/converter/int_to_sym.rb
Adds mapping 34 => :any to HASH_TYPES so LogicalType::ANY is recognized.
ScalarFunction API
lib/duckdb/scalar_function.rb
Adds null_handling: keyword to .create, public set_special_handling wrapper, and includes :any in SUPPORTED_TYPES; calls set_special_handling when requested.
Tests
test/duckdb_test/scalar_function_test.rb
Adds tests for ANY varargs, null-handling default short-circuit and opt-in behavior, block invocation with nil, chainability, and mixed-type varargs cases.

Sequence Diagram

sequenceDiagram
    participant User as User Code
    participant RubySF as DuckDB::ScalarFunction (Ruby)
    participant CExt as ext/duckdb (C binding)
    participant DuckDB as DuckDB Engine
    participant Block as Ruby Block

    rect rgba(220, 240, 220, 0.5)
    Note over User,RubySF: default (null_handling: false)
    User->>RubySF: .create(..., null_handling: false)
    RubySF->>CExt: register_callback
    User->>DuckDB: invoke with NULL
    DuckDB->>DuckDB: short-circuit -> return NULL
    DuckDB-->>User: NULL (block not called)
    end

    rect rgba(200, 220, 255, 0.5)
    Note over User,RubySF: opt-in (null_handling: true)
    User->>RubySF: .create(..., null_handling: true)
    RubySF->>RubySF: set_special_handling
    RubySF->>CExt: _set_special_handling -> duckdb_scalar_function_set_special_handling
    RubySF->>CExt: register_callback
    User->>DuckDB: invoke with NULL
    DuckDB->>CExt: call registered callback
    CExt->>Block: invoke Ruby block with nil for NULL args
    Block-->>CExt: return custom result
    CExt->>DuckDB: return result
    DuckDB-->>User: custom result
    end
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related issues

Possibly related PRs

Poem

🐰 Hopping through NULLs, I peek and see,

I nibble :any and let callbacks be;
No short-circuit stops my curious trail,
Nil becomes input — let custom logic prevail! ✨

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 14.29% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately describes the main changes: implementing ScalarFunction#set_special_handling method and null_handling: keyword parameter.
Linked Issues check ✅ Passed All coding requirements from #1176 are met: C wrapper for duckdb_scalar_function_set_special_handling is implemented, public set_special_handling method is added, null_handling keyword enables the feature, and type support for ANY is extended.
Out of Scope Changes check ✅ Passed All changes are scoped to #1176 objectives: C binding wrapper, Ruby method wrapper, type mapping addition, and comprehensive tests validating NULL handling behavior.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feature/scalar-function-set-special-handling

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
test/duckdb_test/scalar_function_test.rb (1)

1143-1164: Consider removing the stale skip in test_scalar_function_with_varargs_any_type.

This test uses DuckDB::LogicalType::ANY without a skip, which is correct now that this PR adds :any support. However, there's an existing test at line 865 (test_scalar_function_with_varargs_any_type) that still has a stale skip comment stating ANY type isn't exposed yet.

While that test isn't part of this PR's changes, you may want to unskip it in a follow-up or within this PR to maintain test consistency.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@test/duckdb_test/scalar_function_test.rb` around lines 1143 - 1164, The test
suite contains a stale skip for ANY type in the test named
test_scalar_function_with_varargs_any_type; remove the skip/skip comment that
asserts DuckDB::LogicalType::ANY isn't exposed anymore (or change it to noop) so
the test runs like test_set_special_handling_null_count_any_varargs; locate the
test named test_scalar_function_with_varargs_any_type and delete or update the
skip/skip-related conditional, then run the test suite to ensure it passes with
the newly added :any support.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Nitpick comments:
In `@test/duckdb_test/scalar_function_test.rb`:
- Around line 1143-1164: The test suite contains a stale skip for ANY type in
the test named test_scalar_function_with_varargs_any_type; remove the skip/skip
comment that asserts DuckDB::LogicalType::ANY isn't exposed anymore (or change
it to noop) so the test runs like
test_set_special_handling_null_count_any_varargs; locate the test named
test_scalar_function_with_varargs_any_type and delete or update the
skip/skip-related conditional, then run the test suite to ensure it passes with
the newly added :any support.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: ac36a1cc-23bf-427b-8634-b8a35f07687f

📥 Commits

Reviewing files that changed from the base of the PR and between 811a7de and 41a7b14.

📒 Files selected for processing (5)
  • CHANGELOG.md
  • ext/duckdb/scalar_function.c
  • lib/duckdb/converter/int_to_sym.rb
  • lib/duckdb/scalar_function.rb
  • test/duckdb_test/scalar_function_test.rb

DuckDB::LogicalType::ANY is now fully supported after adding
34 => :any to HASH_TYPES and :any to ScalarFunction::SUPPORTED_TYPES.

Replace the skip with a count_args function that accepts ANY varargs
and returns the number of arguments — confirming mixed-type calls work.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@suketa suketa merged commit d71811e into main Mar 24, 2026
41 checks passed
@suketa suketa mentioned this pull request Mar 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

set_special_handling — wrap duckdb_scalar_function_set_special_handling. Marks the function to receive NULL inputs.

1 participant