Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add tolerance parameter to asofJoin #304

Merged
merged 9 commits into from
Apr 17, 2023

Conversation

nina-hu
Copy link
Contributor

@nina-hu nina-hu commented Apr 5, 2023

Add a tolerance parameter to as_of_join() in tempo. When this parameter is set, the as_of_join will only pull in rows from the right df that are within the tolerance band. If no such values exist in the right df, then null values will be returned.

Design details: https://docs.google.com/document/d/1EoFqCf_sXgANnRNXnAn88FzbrEN5UJOFYlCk7l02dyg/edit#heading=h.bsuf6cksdce9

@tnixon
Copy link
Contributor

tnixon commented Apr 6, 2023

Hi @nina-hu - I see that you've added a parameter to several methods, are you going to provide an implementation, too?

@codecov
Copy link

codecov bot commented Apr 10, 2023

Codecov Report

Merging #304 (8fe8d2f) into master (e6ef8b6) will increase coverage by 0.18%.
The diff coverage is n/a.

❗ Current head 8fe8d2f differs from pull request most recent head 9d06de8. Consider uploading reports for the commit 9d06de8 to get more accurate results

@@            Coverage Diff             @@
##           master     #304      +/-   ##
==========================================
+ Coverage   85.46%   85.65%   +0.18%     
==========================================
  Files           6        6              
  Lines         757      767      +10     
  Branches      156      159       +3     
==========================================
+ Hits          647      657      +10     
  Misses         92       92              
  Partials       18       18              

see 3 files with indirect coverage changes

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

python/tempo/tsdf.py Outdated Show resolved Hide resolved
python/tempo/tsdf.py Outdated Show resolved Hide resolved
python/tempo/tsdf.py Outdated Show resolved Hide resolved
Copy link
Contributor

@tnixon tnixon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

Copy link
Contributor

@tnixon tnixon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@nina-hu - can you use Black to reformat tsdf.py?

@tnixon
Copy link
Contributor

tnixon commented Apr 14, 2023

@nina-hu - can you use Black to reformat tsdf.py?

this can be done with the command:
black ./python/tempo/tsdf.py

@nina-hu nina-hu requested a review from tnixon April 17, 2023 16:39
@tnixon tnixon merged commit 8270aa6 into databrickslabs:master Apr 17, 2023
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants