Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Feature Store] Add Dask and Spark feature mergers (get_offline_features) #1544

Merged
merged 23 commits into from
Jan 2, 2022
Merged

[Feature Store] Add Dask and Spark feature mergers (get_offline_features) #1544

merged 23 commits into from
Jan 2, 2022

Conversation

yaronha
Copy link
Collaborator

@yaronha yaronha commented Dec 5, 2021

No description provided.

yaron haviv added 21 commits December 2, 2021 00:48
add dask and spark mergers
400cce4
add dask and spark mergers
83aef03
Merge remote-tracking branch 'remotes/origin/development' into new_fe…
1d750b9 
…ature_mergers
fix spark
845cb4b
fix spark
a359384
fix spark
8342c6a
fix spark
f0d293b
fix spark
1c7928b
fix spark
a6bdbe0
fix spark
36b1cc1
cleanup spark
092b856
cleanup spark
3857fac
fix test
c9bddf3
Merge remote-tracking branch 'remotes/origin/development' into new_fe…
6e7fdd8 
…ature_mergers

# Conflicts:
#	requirements.txt
fix test
b79360d
cleanup
e648ed4
Merge remote-tracking branch 'remotes/origin/development' into new_fe…
b9638a6 
…ature_mergers

# Conflicts:
#	requirements.txt
lint
a71515f
lint
3637b65
fixes
ce6b37c
Merge remote-tracking branch 'remotes/origin/development' into new_fe…
b72363d 
…ature_mergers
@yaronha yaronha requested a review from gtopper December 29, 2021 11:16
gtopper
gtopper reviewed Dec 29, 2021
View reviewed changes
Copy link
Collaborator

@gtopper gtopper left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor comments

mlrun/datastore/base.py Outdated Show resolved Hide resolved
mlrun/feature_store/feature_set.py Outdated Show resolved Hide resolved
mlrun/feature_store/retrieval/job.py Outdated Show resolved Hide resolved
mlrun/feature_store/retrieval/base.py Show resolved Hide resolved
mlrun/feature_store/retrieval/spark_merger.py Outdated Show resolved Hide resolved
gtopper
gtopper approved these changes Dec 30, 2021
View reviewed changes
Copy link
Collaborator

@gtopper gtopper left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Maybe we can use TestFeatureStoreSparkEngine to test the spark merger?

@Hedingber Hedingber changed the title [Feature store] add Dask and Spark feature mergers (get_offline_features) [Feature Store] Add Dask and Spark feature mergers (get_offline_features) Jan 2, 2022
@Hedingber Hedingber merged commit 9abcdd5 into mlrun:development Jan 2, 2022
@yaronha yaronha deleted the new_feature_mergers branch January 14, 2022 14:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@gtopper gtopper gtopper approved these changes

@Hedingber Hedingber Hedingber approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@yaronha @gtopper @Hedingber