Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support binary operation with Timedelta in Pyspark backend #2873

Merged
merged 4 commits into from
Jul 22, 2021
Merged

Support binary operation with Timedelta in Pyspark backend #2873

merged 4 commits into from
Jul 22, 2021

Conversation

emilyreff7
Copy link
Contributor

This PR adds support in the Pyspark backend for doing binary operations (e.g. add/subtract) on timestamp columns using Pandas Timedeltas, for example:

dt['timestamp_col'] + pd.Timedelta(days=5)

@datapythonista datapythonista added pyspark The Apache PySpark backend feature Features or general enhancements labels Jul 21, 2021
@emilyreff7 emilyreff7 changed the title Support binary operation with Timedelta in Pyspark Support binary operation with Timedelta in Pyspark backend Jul 21, 2021
icexelloss
icexelloss suggested changes Jul 22, 2021
View reviewed changes
Copy link
Contributor

@icexelloss icexelloss left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Request for small change to simply the logic

icexelloss
icexelloss reviewed Jul 22, 2021
View reviewed changes
Copy link
Contributor

@icexelloss icexelloss left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. +1

@jreback jreback added this to the Next release milestone Jul 22, 2021
@jreback jreback merged commit c9cb307 into ibis-project:master Jul 22, 2021
@jreback
Copy link
Contributor

jreback commented Jul 22, 2021

thanks @emilyreff7

@emilyreff7 emilyreff7 deleted the support-timestamp-binop-spark branch July 22, 2021 21:23
@cpcloud cpcloud removed this from the Next release milestone Jan 7, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@icexelloss icexelloss icexelloss approved these changes

Assignees
No one assigned
Labels
feature Features or general enhancements pyspark The Apache PySpark backend
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@emilyreff7 @jreback @icexelloss @cpcloud @datapythonista @emilyreff