[SPARK-22263][SQL]Refactor deterministic as lazy value

[gengliangwang]

What changes were proposed in this pull request?

The method deterministic is frequently called in optimizer.
Refactor deterministic as lazy value, in order to avoid redundant computations.

How was this patch tested?

Simple benchmark test over TPC-DS queries, run time from query string to optimized plan(continuous 20 runs, and get the average of last 5 results):
Before changes: 12601 ms
After changes: 11993ms
This is 4.8% performance improvement.

Also run test with Unit test.

[SparkQA]

Test build #82664 has finished for PR 19478 at commit bdeea55.

• This patch fails to build.
• This patch merges cleanly.
• This patch adds no public classes.

[viirya]

I doubt how much time this can save. But why won't just:

lazy val deterministic: Boolean = children.forall(_.deterministic)

I think it is equal to this change.

[hvanhovell]

+1

[hvanhovell]

@gengliangwang do you have any benchmark that shows that this is a performance bottleneck?

[SparkQA]

Test build #82667 has finished for PR 19478 at commit 8143ee1.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[gatorsmile]

Compare the total optimization time for the TPC-DS queries?

[gengliangwang]

@viirya @hvanhovell @gatorsmile

Thanks, I have attached the performance result in the description in this PR.
Overall I don't see any downside of the code change.
Also it is possible to take much more time to get the deterministic of UDF, while making it lazy value can avoid that.

[gatorsmile]

LGTM cc @JoshRosen

[SparkQA]

Test build #82698 has finished for PR 19478 at commit 03e88b0.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[viirya]

Ok. The performance result looks good. LGTM.

[gatorsmile]

Thanks! Merged to master.