Time shift occurring between spark and h2o frame

[exalate-issue-sync]

Customer reported that h2o is adding a timeshift (+7 hours, likely from timezone) while pandas dataframe does not.

{code:python}h2o.cluster().timezone{code}

{{'UTC'}}

{code:python}df = spark.read.parquet('hdfs://' + input_path + '/' + input_data)[keep_list]
df1 = df.limit(10).select('snapshot_date')
df1.cache()
df1.show(){code}

{quote}+-------------+

|snapshot_date|

+-------------+

| 2015-01-01|

| 2015-01-01|

| 2015-01-01|

| 2015-01-01|

| 2015-01-01|

| 2015-01-01|

| 2015-01-01|

| 2015-01-01|

| 2015-01-01|

| 2015-01-01|

+-------------+{quote}

{code:python}#h2o.cluster().timezone = "America/Phoenix"
hc.as_h2o_frame(df1)

#Method 'as_h2o_frame' is deprecated and will be removed in release 3.30. Please use method 'asH2OFrame' instead!{code}

||snapshot_date||
|2015-01-01 07:00:00|
|2015-01-01 07:00:00|
|2015-01-01 07:00:00|
|2015-01-01 07:00:00|
|2015-01-01 07:00:00|
|2015-01-01 07:00:00|
|2015-01-01 07:00:00|
|2015-01-01 07:00:00|
|2015-01-01 07:00:00|
|2015-01-01 07:00:00|

{noformat}df1.schema
StructType(List(StructField(snapshot_date,DateType,true)))
h2o.H2OFrame(df1.toPandas())
Parse progress: |█████████████████████████████████████████████████████████| 100%{noformat}

||snapshot_date||
|2015-01-01 00:00:00|
|2015-01-01 00:00:00|
|2015-01-01 00:00:00|
|2015-01-01 00:00:00|
|2015-01-01 00:00:00|
|2015-01-01 00:00:00|
|2015-01-01 00:00:00|
|2015-01-01 00:00:00|
|2015-01-01 00:00:00|
|2015-01-01 00:00:00|

[DinukaH2O]

JIRA Issue Migration Info

Jira Issue: SW-2070
Assignee: Marek Novotny
Reporter: Neema Mashayekhi
State: Resolved
Fix Version: 3.28.1.2-1
Attachments: N/A
Development PRs: N/A

[hasithjp]

JIRA Issue Migration Info Cont'd

Jira Issue Created Date: 2020-03-10T15:32:23.593-0700