chore: fix some scalar function docs by Jefffrey · Pull Request #24134 · apache/datafusion

Jefffrey · 2026-08-06T06:10:30Z

Fixing some typos, errors, and consolidating some parts

Jefffrey

i might consider creating tracking issues for some of these to see if theres a way to automate some of this, but dont really have a clear idea of how to go about it as such

Jefffrey · 2026-08-06T06:10:51Z

 regular expression [syntax](https://docs.rs/regex/latest/regex/#syntax)
 (minus support for several features including look-around and backreferences).
+
+The following flags are optionally supported in functions:


instead of having this defined per regex function, better to have it in one place they can then refer to

Jefffrey · 2026-08-06T06:11:11Z

-| true                                       |
-+--------------------------------------------+
+---------------------------------------------+
+| true                                        |


i do wonder if we can find a way to format these tables? theres more out there that were unaligned

Maybe in a followup we can use a simple script to check if they are valid, similar to the markdown ones. Here are the invalid ones:

datafusion/functions/src/core/arrow_field.rs: line 39 datafusion/functions/src/core/nvl.rs: line 34 datafusion/functions/src/core/with_metadata.rs: line 33 datafusion/functions/src/core/with_metadata.rs: line 39 datafusion/functions/src/datetime/date_trunc.rs: line 191 datafusion/functions/src/datetime/date_trunc.rs: line 197 datafusion/functions/src/datetime/to_date.rs: line 45 datafusion/functions/src/datetime/to_date.rs: line 51 datafusion/functions/src/datetime/to_timestamp.rs: line 274 datafusion/functions/src/math/round.rs: line 166 datafusion/functions-aggregate/src/approx_percentile_cont_with_weight.rs: line 94 datafusion/functions-aggregate/src/array_agg.rs: line 80 datafusion/functions-aggregate/src/average.rs: line 86 datafusion/functions-aggregate/src/bool_and_or.rs: line 100 datafusion/functions-aggregate/src/bool_and_or.rs: line 229 datafusion/functions-aggregate/src/count.rs: line 141 datafusion/functions-aggregate/src/grouping.rs: line 48 datafusion/functions-aggregate/src/median.rs: line 75 datafusion/functions-aggregate/src/min_max.rs: line 87 datafusion/functions-aggregate/src/min_max.rs: line 459 datafusion/functions-aggregate/src/percentile_cont.rs: line 103 datafusion/functions-aggregate/src/regr.rs: line 183 datafusion/functions-aggregate/src/stddev.rs: line 56 datafusion/functions-aggregate/src/stddev.rs: line 160 datafusion/functions-aggregate/src/string_agg.rs: line 77 datafusion/functions-aggregate/src/sum.rs: line 156 datafusion/functions-nested/src/array_avg.rs: line 52 datafusion/functions-nested/src/array_filter.rs: line 58 datafusion/functions-nested/src/array_has.rs: line 1002 datafusion/functions-nested/src/array_has.rs: line 1070 datafusion/functions-nested/src/array_normalize.rs: line 55 datafusion/functions-nested/src/array_scale.rs: line 53 datafusion/functions-nested/src/array_sum.rs: line 52 datafusion/functions-nested/src/array_transform.rs: line 57 datafusion/functions-nested/src/cosine_distance.rs: line 54 datafusion/functions-nested/src/except.rs: line 55 datafusion/functions-nested/src/except.rs: line 61 datafusion/functions-nested/src/extract.rs: line 794 datafusion/functions-nested/src/extract.rs: line 978 datafusion/functions-nested/src/inner_product.rs: line 54 datafusion/functions-nested/src/position.rs: line 69 datafusion/functions-nested/src/remove.rs: line 61 datafusion/functions-nested/src/remove.rs: line 171 datafusion/functions-nested/src/remove.rs: line 300 datafusion/functions-nested/src/string.rs: line 173 datafusion/functions-nested/src/string.rs: line 179

Jefffrey · 2026-08-06T06:11:32Z

    ),
    argument(
-        name = "sub-array",
+        name = "sub_array",


maybe we need a way to standardize the names? sub-array doesnt render as well on the site, so prefer sticking to underscores

Jefffrey · 2026-08-06T06:11:55Z

 #[user_doc(
    doc_section(label = "Array Functions"),
    description = "Returns the number of dimensions of the array.",
-    syntax_example = "array_ndims(array, element)",


is there some way to automate checking the arguments match? not likely 🤔

Jefffrey · 2026-08-06T06:12:34Z

        description = "Array expression. Can be a constant, column, or function, and any combination of array operators."
    ),
-    argument(name = "dimension", description = "Array dimension.")
+    argument(name = "dimension", description = "Array dimension. Default is 1")


i was thinking of modifying the doc macro to be able to specific optional args, and then possibly create the syntax_example automatically, but not sure how to get the name of the function from the macro

codecov-commenter · 2026-08-06T06:38:48Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 81.04%. Comparing base (4fcbfb1) to head (9382ddb).
⚠️ Report is 3 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #24134      +/-   ##
==========================================
- Coverage   81.04%   81.04%   -0.01%     
==========================================
  Files        1105     1105              
  Lines      380163   380163              
  Branches   380163   380163              
==========================================
- Hits       308110   308103       -7     
- Misses      53834    53840       +6     
- Partials    18219    18220       +1

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

nuno-faria

Thanks @Jefffrey, LGTM.

nuno-faria · 2026-08-07T19:18:10Z

-| true                                       |
-+--------------------------------------------+
+---------------------------------------------+
+| true                                        |


Maybe in a followup we can use a simple script to check if they are valid, similar to the markdown ones. Here are the invalid ones:

datafusion/functions/src/core/arrow_field.rs: line 39 datafusion/functions/src/core/nvl.rs: line 34 datafusion/functions/src/core/with_metadata.rs: line 33 datafusion/functions/src/core/with_metadata.rs: line 39 datafusion/functions/src/datetime/date_trunc.rs: line 191 datafusion/functions/src/datetime/date_trunc.rs: line 197 datafusion/functions/src/datetime/to_date.rs: line 45 datafusion/functions/src/datetime/to_date.rs: line 51 datafusion/functions/src/datetime/to_timestamp.rs: line 274 datafusion/functions/src/math/round.rs: line 166 datafusion/functions-aggregate/src/approx_percentile_cont_with_weight.rs: line 94 datafusion/functions-aggregate/src/array_agg.rs: line 80 datafusion/functions-aggregate/src/average.rs: line 86 datafusion/functions-aggregate/src/bool_and_or.rs: line 100 datafusion/functions-aggregate/src/bool_and_or.rs: line 229 datafusion/functions-aggregate/src/count.rs: line 141 datafusion/functions-aggregate/src/grouping.rs: line 48 datafusion/functions-aggregate/src/median.rs: line 75 datafusion/functions-aggregate/src/min_max.rs: line 87 datafusion/functions-aggregate/src/min_max.rs: line 459 datafusion/functions-aggregate/src/percentile_cont.rs: line 103 datafusion/functions-aggregate/src/regr.rs: line 183 datafusion/functions-aggregate/src/stddev.rs: line 56 datafusion/functions-aggregate/src/stddev.rs: line 160 datafusion/functions-aggregate/src/string_agg.rs: line 77 datafusion/functions-aggregate/src/sum.rs: line 156 datafusion/functions-nested/src/array_avg.rs: line 52 datafusion/functions-nested/src/array_filter.rs: line 58 datafusion/functions-nested/src/array_has.rs: line 1002 datafusion/functions-nested/src/array_has.rs: line 1070 datafusion/functions-nested/src/array_normalize.rs: line 55 datafusion/functions-nested/src/array_scale.rs: line 53 datafusion/functions-nested/src/array_sum.rs: line 52 datafusion/functions-nested/src/array_transform.rs: line 57 datafusion/functions-nested/src/cosine_distance.rs: line 54 datafusion/functions-nested/src/except.rs: line 55 datafusion/functions-nested/src/except.rs: line 61 datafusion/functions-nested/src/extract.rs: line 794 datafusion/functions-nested/src/extract.rs: line 978 datafusion/functions-nested/src/inner_product.rs: line 54 datafusion/functions-nested/src/position.rs: line 69 datafusion/functions-nested/src/remove.rs: line 61 datafusion/functions-nested/src/remove.rs: line 171 datafusion/functions-nested/src/remove.rs: line 300 datafusion/functions-nested/src/string.rs: line 173 datafusion/functions-nested/src/string.rs: line 179

nuno-faria · 2026-08-07T19:24:23Z

 #[user_doc(
    doc_section(label = "Array Functions"),
    description = "Sort array.",
-    syntax_example = "array_sort(array, desc, nulls_first)",
+    syntax_example = "array_sort(array[, desc[, nulls_first]])",
    sql_example = r#"```sql
 > select array_sort([3, 1, 2]);
 +-----------------------------+
 | array_sort(List([3,1,2]))   |
 +-----------------------------+
 | [1, 2, 3]                   |
 +-----------------------------+


I'm not sure about the name of the arguments, since the argument desc can be 'asc' and nulls_first can be 'nulls last'. What about order and nulls or something like that?

Also, should we also add an example using the additional arguments?

chore: fix some scalar function docs

e8ea274

github-actions Bot added documentation Improvements or additions to documentation functions Changes to functions implementation labels Aug 6, 2026

Jefffrey commented Aug 6, 2026

View reviewed changes

Jefffrey marked this pull request as ready for review August 6, 2026 06:15

Merge branch 'main' into fixup-scalar-function-docs

9382ddb

nuno-faria approved these changes Aug 7, 2026

View reviewed changes

Navigation Menu

Uh oh!

chore: fix some scalar function docs - #24134

chore: fix some scalar function docs#24134
Jefffrey wants to merge 2 commits into
apache:mainfrom
Jefffrey:fixup-scalar-function-docs

Jefffrey commented Aug 6, 2026

Uh oh!

Jefffrey left a comment

Uh oh!

Jefffrey Aug 6, 2026

Uh oh!

Jefffrey Aug 6, 2026

Uh oh!

nuno-faria Aug 7, 2026

Uh oh!

Jefffrey Aug 6, 2026

Uh oh!

Jefffrey Aug 6, 2026

Uh oh!

Jefffrey Aug 6, 2026

Uh oh!

codecov-commenter commented Aug 6, 2026 •

edited

Loading

Uh oh!

nuno-faria left a comment

Uh oh!

nuno-faria Aug 7, 2026

Uh oh!

nuno-faria Aug 7, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

Jefffrey commented Aug 6, 2026

Uh oh!

Jefffrey left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

codecov-commenter commented Aug 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

nuno-faria left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

codecov-commenter commented Aug 6, 2026 •

edited

Loading