Skip to content

Add scalar functions: math, IP address, and string utilities#18409

Merged
mayankshriv merged 1 commit into
apache:masterfrom
mayankshriv:scalar-functions
May 4, 2026
Merged

Add scalar functions: math, IP address, and string utilities#18409
mayankshriv merged 1 commit into
apache:masterfrom
mayankshriv:scalar-functions

Conversation

@mayankshriv
Copy link
Copy Markdown
Contributor

@mayankshriv mayankshriv commented May 4, 2026

Summary

Adds 23 new @ScalarFunction methods. No new dependencies — all implementations use libraries already on the classpath. All functions are stateless, thread-safe, and auto-registered via FunctionRegistry.

Category Function Description
Math cbrt(x) Cube root
exp2(x) 2^x (convenience over power(2,x))
exp10(x) 10^x (convenience over power(10,x))
log1p(x) Numerically stable log(1+x)
sigmoid(x) Logistic function 1/(1+e^-x)
pi() Returns π
e() / euler() Returns Euler's number
bitCount(x) Popcount (set bits)
IP isIPv4String(s) Validate IPv4 address
isIPv6String(s) Validate IPv6 address
ipv4ToLong(s) IPv4 → integer
longToIpv4(x) Integer → IPv4
ipv6ToBytes(s) IPv6 → 16 bytes
bytesToIpv6(b) 16 bytes → IPv6
ipv4ToIpv6(s) IPv4 → IPv4-mapped IPv6
ipv4CIDRToRange(s) CIDR → [min, max] range
String ascii(s) ASCII code of first char
space(n) N spaces (convenience over repeat(' ',n))
substringIndex(s,d,n) Everything before/after Nth delimiter
firstLine(s) Text before first newline
startsWithCaseInsensitive(s,p) Case-insensitive prefix check
endsWithCaseInsensitive(s,p) Case-insensitive suffix check
isValidASCII(s) All chars are ASCII

Test plan

  • 56 unit tests across ArithmeticFunctionsTest (new), IpAddressFunctionsTest, StringFunctionsTest — edge cases include NaN, Infinity, boundary values, round-trip conversions, invalid inputs
  • 27 transform tests in ScalarTransformFunctionWrapperTest — verifies each function works end-to-end through ScalarTransformFunctionWrapper with segment data
  • All tests passing, zero compiler deprecation warnings in changed files
  • Spotless, checkstyle, and license checks pass

Documentation

Documentation updates to pinot-docs will follow once this PR is merged.

🤖 Generated with Claude Code

@codecov-commenter
Copy link
Copy Markdown

codecov-commenter commented May 4, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 63.51%. Comparing base (fa91518) to head (9038639).

Additional details and impacted files
@@             Coverage Diff              @@
##             master   #18409      +/-   ##
============================================
- Coverage     63.54%   63.51%   -0.04%     
  Complexity     1709     1709              
============================================
  Files          3254     3254              
  Lines        199181   199244      +63     
  Branches      30851    30863      +12     
============================================
- Hits         126567   126546      -21     
- Misses        62498    62612     +114     
+ Partials      10116    10086      -30     
Flag Coverage Δ
custom-integration1 100.00% <ø> (ø)
integration 100.00% <ø> (ø)
integration1 100.00% <ø> (ø)
integration2 0.00% <ø> (ø)
java-21 63.51% <100.00%> (-0.04%) ⬇️
temurin 63.51% <100.00%> (-0.04%) ⬇️
unittests 63.51% <100.00%> (-0.04%) ⬇️
unittests1 55.47% <100.00%> (+0.04%) ⬆️
unittests2 34.98% <0.00%> (-0.09%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Math (8 functions): cbrt, exp2, exp10, log1p, sigmoid, pi, e, bitCount.
IP address (8 functions): isIPv4String, isIPv6String, ipv4ToLong,
longToIpv4, ipv6ToBytes, bytesToIpv6, ipv4ToIpv6, ipv4CIDRToRange.
String (7 functions): ascii, space, substringIndex, firstLine,
startsWithCaseInsensitive, endsWithCaseInsensitive, isValidASCII.

Includes unit tests and transform function wrapper tests for all 23 functions.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@xiangfu0 xiangfu0 added query Related to query processing functions Related to scalar or aggregation functions labels May 4, 2026
@mayankshriv mayankshriv merged commit 40dd5f2 into apache:master May 4, 2026
11 checks passed
@mayankshriv mayankshriv deleted the scalar-functions branch May 4, 2026 03:19
@xiangfu0
Copy link
Copy Markdown
Contributor

xiangfu0 commented May 4, 2026

Opened the matching docs PR: pinot-contrib/pinot-docs#789

xiangfu0 added a commit to pinot-contrib/pinot-docs that referenced this pull request May 4, 2026
## Summary

- documents the new scalar math, string, and IP address functions added
by apache/pinot#18409 on the existing category pages
- adds the same functions to the query function index so readers can
discover them from the A-Z reference

## Source cross-checks

- verified signatures, aliases, and return types against `pinot-common`
scalar function implementations
- verified examples and edge-case wording against
`ArithmeticFunctionsTest`, `IpAddressFunctionsTest`,
`StringFunctionsTest`, and `ScalarTransformFunctionWrapperTest` in the
local `apache/pinot` checkout

## Validation

- `git diff --check`
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

functions Related to scalar or aggregation functions query Related to query processing

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants