Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support to pass dynamic values to timestamp Extract function #15586

Merged
merged 7 commits into from Dec 21, 2023

Conversation

AlbericByte
Copy link
Contributor

@AlbericByte AlbericByte commented Dec 16, 2023

Fixes #15072

Description

Before this modification , the third parameter (timezone) require to be a Literal, it will throw a error when this parameter is column Identifier.

Release note


Key changed/added classes in this PR
  • Updated the validation
  • Updated TimestampExtractExprMacro class
  • Added a unit test in TimestampExtractExprMacroTest

This PR has:

  • been self-reviewed.
  • using the concurrency checklist (Remove this item if the PR doesn't have any relation to concurrency.)
  • added documentation for new or modified features or behaviors.
  • a release note entry in the PR description.
  • added Javadocs for most classes and all non-trivial methods. Linked related entities via Javadoc links.
  • added or updated version, license, or notice information in licenses.yaml
  • added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader.
  • added unit tests or modified existing tests to cover new code paths, ensuring the threshold for code coverage is met.
  • added integration tests.
  • been tested in a test Druid cluster.

Comment on lines +1847 to +1856
testHelper.testExpressionString(
new TimeExtractOperatorConversion().calciteOperator(),
ImmutableList.of(
testHelper.makeInputRef("t"),
testHelper.makeLiteral("DAY"),
testHelper.makeInputRef("timezone")
),
makeExpression(ColumnType.LONG, "timestamp_extract(\"t\",'DAY',\"timezone\")"),
2L
);

Check notice

Code scanning / CodeQL

Deprecated method or constructor invocation Note

Invoking
ExpressionTestHelper.testExpressionString
should be avoided because it has been deprecated.
testHelper.makeLiteral("DAY"),
testHelper.makeInputRef("timezone")
),
makeExpression(ColumnType.LONG, "timestamp_extract(\"t\",'DAY',\"timezone\")"),

Check notice

Code scanning / CodeQL

Deprecated method or constructor invocation Note

Invoking
CalciteTestBase.makeExpression
should be avoided because it has been deprecated.
}

final Expr arg = args.get(0);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you could still keep it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no reference for this arg, so I removed it.

if (val == null) {
// Return null if the argument if null.
return ExprEval.of(null);
}
final ISOChronology chronology = chronologyOptional.orElseGet(() -> {
Optional<String> timeZoneOptional = args.size() > 2
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could avoid this check in eval and set zone to UTC outside eval itself is args size is less than or equal to 2

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I also think the usage of Optional is unnecessary here given that the usage is contained within this class. So you could also do good-ol if-null-then-x-else-y style of code.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed in the latest commit. @abhishekagarwal87 Thanks for you suggestion and time

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the test error "Error while executing SQL "SELECT * FROM INFORMATION_SCHEMA.COLUMNS": Remote driver error: Unauthorized" should be irrelevant to my modification.

final ISOChronology chronology = ISOChronology.getInstance(timeZone);

class TimestampExtractExpr extends ExprMacroTable.BaseScalarUnivariateMacroFunctionExpr
class TimestampExtractExpr extends ExprMacroTable.BaseScalarMacroFunctionExpr
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

drive by comment: because eval is called every row, we often will split up implementations of Expr when one form has a dynamic argument that needs evaluated every row, see TimestampFloorExprMacro for example. This allows cases where the argument is a constant to skip some work.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good catch and suggestion @clintropolis followed the example and fixed in the latest commit.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@clintropolis and @abhishekagarwal87 the unit test cases("Remote driver error: Unauthorized" and "leadership changed") failed which should be not relative to my modification.

@abhishekagarwal87 abhishekagarwal87 merged commit a2e65e6 into apache:master Dec 21, 2023
82 of 83 checks passed
@abhishekagarwal87
Copy link
Contributor

thank you for your contribution @AlbericByte

@LakshSingla LakshSingla added this to the 29.0.0 milestone Jan 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Unable to pass dynamic values to time function
4 participants