Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SPARK-20456][PYTHON][FOLLOWUP] Fix timezone-dependent doctests in unix_timestamp and from_unixtime #18597

Closed
wants to merge 1 commit into from

Conversation

HyukjinKwon
Copy link
Member

What changes were proposed in this pull request?

This PR proposes to simply ignore the results in examples that are timezone-dependent in unix_timestamp and from_unixtime.

Failed example:
    time_df.select(unix_timestamp('dt', 'yyyy-MM-dd').alias('unix_time')).collect()
Expected:
    [Row(unix_time=1428476400)]
Got:unix_timestamp
    [Row(unix_time=1428418800)]
Failed example:
    time_df.select(from_unixtime('unix_time').alias('ts')).collect()
Expected:
    [Row(ts=u'2015-04-08 00:00:00')]
Got:
    [Row(ts=u'2015-04-08 16:00:00')]

How was this patch tested?

Manually tested and ./run-tests --modules pyspark-sql.

@HyukjinKwon
Copy link
Member Author

cc @ueshin, could you take a look when you have some time?

@SparkQA
Copy link

SparkQA commented Jul 11, 2017

Test build #79500 has finished for PR 18597 at commit 5c5f405.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@ueshin
Copy link
Member

ueshin commented Jul 11, 2017

How about surrounding with SQLConf spark.sql.session.timeZone as America/Los_Angeles ?

@HyukjinKwon
Copy link
Member Author

Sounds good.

@SparkQA
Copy link

SparkQA commented Jul 11, 2017

Test build #79505 has finished for PR 18597 at commit 31fa09b.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@ueshin
Copy link
Member

ueshin commented Jul 11, 2017

LGTM.

@ueshin
Copy link
Member

ueshin commented Jul 11, 2017

Thanks! merging to master.

@asfgit asfgit closed this in d4d9e17 Jul 11, 2017
@HyukjinKwon
Copy link
Member Author

Thank you @ueshin.

@HyukjinKwon HyukjinKwon deleted the SPARK-20456 branch January 2, 2018 03:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants