Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HIVE-24908: Adding Respect/Ignore nulls as a UDAF parameter is ambiguous #2091

Merged

Conversation

kasakrisz
Copy link
Contributor

What changes were proposed in this pull request?

  1. Add respectNulls boolean property to GenericUDAFParameterInfo.
  2. Map this property to Windowing specification/definitions objects.
  3. Introduce property to WindowFunctionDescription which show whether the function supports handling null treatment settings.
  • If the supportNullTreatment property is not set explicitly the UDAF does not support null treatment and the compiler throws SemanticException if specified.
  • If the supportNullTreatment property is set to true but the function call does not specify null treatment the default null treatment is used: RESPECT NULLS
  • The specified null treatment is used othervise.

Why are the changes needed?

This patch is a follow-up of #2060. See jira for details.

Does this PR introduce any user-facing change?

Yes. When calling lead or lag functions by specifying RESPECT/IGNORE NULLS

  • and no default value parameter is given RESPECT/IGNORE NULLS are not passed as default value parameter: NULL is returned if rows has no corresponding following/preceding rows
  • and default value parameter is given RESPECT/IGNORE NULLS are not passed as default value parameter: no Exception is thrown

How was this patch tested?

mvn test -Dtest.output.overwrite -DskipSparkTests -Dtest=TestNegativeLlapLocalCliDriver -Dqfile=nulltreatment.q -pl itests/qtest -Pitests
mvn test -Dtest.output.overwrite -DskipSparkTests -Dtest=TestMiniLlapLocalCliDriver -Dqfile=windowing_navfn.q -pl itests/qtest -Pitests

@kasakrisz kasakrisz merged commit cdd647b into apache:master Mar 23, 2021
@kasakrisz kasakrisz deleted the HIVE-24908-master-ignore-nulls-windowspec branch May 4, 2021 07:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants