HIVE-24286: Render date and time with progress of Hive on Tez

[okumin]

What changes were proposed in this pull request?

Add date and time to the progress log when hive.server2.in.place.progress is disabled.
https://issues.apache.org/jira/browse/HIVE-24286

Why are the changes needed?

With hive.server2.in.place.progress disabled, Hive prints progress periodically. We want to add date and time to each line so that we can review what phase took time or where the job got stuck easily. As I mentioned in HIVE-24286, Hive on MR or Hive on Spark add date and time.

INFO  : Map 1: -/-	Reducer 2: 0/1	
INFO  : Map 1: 0/1	Reducer 2: 0/1	
INFO  : Map 1: 0(+1)/1	Reducer 2: 0/1	
INFO  : Map 1: 1/1	Reducer 2: 0(+1)/1	
INFO  : Map 1: 1/1	Reducer 2: 1/1	

Does this PR introduce any user-facing change?

The log format can change if a user configures hive.server2.in.place.progress=false.

Is the change a dependency upgrade?

No

How was this patch tested?

I tested the patch on my local machine.

INFO  : 2023-06-15 14:07:44,590	Map 1: -/-	Reducer 2: 0/1	
INFO  : 2023-06-15 14:07:45,605	Map 1: 0/1	Reducer 2: 0/1	
INFO  : 2023-06-15 14:07:48,135	Map 1: 0(+1)/1	Reducer 2: 0/1	
INFO  : 2023-06-15 14:07:49,658	Map 1: 1/1	Reducer 2: 1/1	

[SourabhBadhya]

LGTM +1 (pending tests).

[aturoczy]

LocalDateTime won't be confusing? Just thinking how it will be look like if a user will be EST but the server time will be PST.
Just try to imagine the output.

Overall the code is OK. Could you please move the jira in-progress state?

[okumin]

I followed the behavior of MapReduce since I actually wrote this patch while I was migrating Hive on MR on Hive on Tez.

• MR: It uses the default Calendar which follows the default time zone

Looking at other usages, it could be kinder to use HiveConf#getLocalTimeZone. Let me check and try it.

[okumin]

@aturoczy I updated it. I think it is reasonable since it is consistent with other places such as CURRENT_DATE UDF.

I have also verified the property takes effect.

By default, the default one, UTC in this case, is used.

$ beeline -e 'SELECT sum(price) FROM orders' --hiveconf hive.server2.in.place.progress=false
...
INFO  : 2023-06-16 16:56:55,735	Map 1: -/-	Reducer 2: 0/1	
INFO  : 2023-06-16 16:56:56,240	Map 1: 0/1	Reducer 2: 0/1	

The same offset is used when we explicitly give hive.local.time.zone=UTC.

$ beeline -e 'SELECT sum(price) FROM orders' --hiveconf hive.server2.in.place.progress=false --hiveconf hive.local.time.zone=UTC
...
INFO  : 2023-06-16 16:59:44,985	Map 1: -/-	Reducer 2: 0/1	
INFO  : 2023-06-16 16:59:45,995	Map 1: 0/1	Reducer 2: 0/1	

Configuring other time zones, we see a date and time with the offset.

$ beeline -e 'SELECT sum(price) FROM orders' --hiveconf hive.server2.in.place.progress=false --hiveconf hive.local.time.zone=America/Los_Angeles
...
INFO  : 2023-06-16 10:02:17,700	Map 1: -/-	Reducer 2: 0/1	
INFO  : 2023-06-16 10:02:18,712	Map 1: 0/1	Reducer 2: 0/1	
...
$ beeline -e 'SELECT sum(price) FROM orders' --hiveconf hive.server2.in.place.progress=false --hiveconf hive.local.time.zone=Asia/Tokyo
...
INFO  : 2023-06-17 02:03:21,041	Map 1: -/-	Reducer 2: 0/1	
INFO  : 2023-06-17 02:03:22,050	Map 1: 0/1	Reducer 2: 0/1	

[sonarqubecloud]

Kudos, SonarCloud Quality Gate passed!   

0 Bugs
0 Vulnerabilities
0 Security Hotspots
0 Code Smells

No Coverage information
No Duplication information

[ayushtkn]

LGTM

[okumin]

Thank you!