Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Pre fetch openml data for pytest #112

Merged
merged 3 commits into from Mar 1, 2021
Merged

Pre fetch openml data for pytest #112

merged 3 commits into from Mar 1, 2021

Conversation

franchuterivera
Copy link
Contributor

It prefetches the openml data on a need basis. With pytest -s option one would see:

test/test_data/test_validation.py::test_data_validation_for_classification[True-2] DEBUG:openml.datasets.dataset:Data pickle file already exists and is up to date.
DEBUG:openml.datasets.dataset:Data pickle file already exists and is up to date.
DEBUG:openml.datasets.dataset:Data pickle file already exists and is up to date.
DEBUG:openml.datasets.dataset:Data pickle file already exists and is up to date.
DEBUG:openml.datasets.dataset:Data pickle file already exists and is up to date.
DEBUG:openml.datasets.dataset:Data pickle file already exists and is up to date.
DEBUG:openml.datasets.dataset:Data pickle file already exists and is up to date.
DEBUG:openml.datasets.dataset:Data pickle file already exists and is up to date.
DEBUG:openml.datasets.dataset:Data pickle file already exists and is up to date.
DEBUG:openml.datasets.dataset:Data pickle file already exists and is up to date.
DEBUG:openml.datasets.dataset:Data pickle file already exists and is up to date.
DEBUG:openml.datasets.dataset:Data pickle file already exists and is up to date.
DEBUG:openml.datasets.dataset:Data pickle file already exists and is up to date.
DEBUG:openml.datasets.dataset:Data pickle file already exists and is up to date.
DEBUG:openml.datasets.dataset:Data pickle file already exists and is up to date.
DEBUG:openml.datasets.dataset:Data pickle file already exists and is up to date.
DEBUG:openml.datasets.dataset:Data pickle file already exists and is up to date.
DEBUG:openml.datasets.dataset:Data pickle file already exists and is up to date.
DEBUG:openml.datasets.dataset:Data pickle file already exists and is up to date.
DEBUG:openml.datasets.dataset:Data pickle file already exists and is up to date.
DEBUG:openml.datasets.dataset:Data pickle file already exists and is up to date.
DEBUG:openml.datasets.dataset:Data pickle file already exists and is up to date.
DEBUG:openml.datasets.dataset:Data pickle file already exists and is up to date.
DEBUG:openml.datasets.dataset:Data pickle file already exists and is up to date.
DEBUG:openml.datasets.dataset:Data pickle file already exists and is up to date.
DEBUG:openml.datasets.dataset:Data pickle file already exists and is up to date.
DEBUG:openml.datasets.dataset:Data pickle file already exists and is up to date.
DEBUG:openml.datasets.dataset:Data pickle file already exists and is up to date.
Pre-Fetch of 14 tasks took 2.0051114559173584 seconds...

Addresses #109

@franchuterivera franchuterivera added this to In progress in Roadmap via automation Feb 23, 2021
@franchuterivera franchuterivera added the enhancement New feature or request label Feb 23, 2021
@franchuterivera franchuterivera marked this pull request as draft February 23, 2021 20:39
@franchuterivera franchuterivera marked this pull request as ready for review March 1, 2021 14:26
Roadmap automation moved this from In progress to Reviewer approved Mar 1, 2021
@ravinkohli ravinkohli merged commit 55ec853 into automl:refactor_development Mar 1, 2021
Roadmap automation moved this from Reviewer approved to Done Mar 1, 2021
github-actions bot pushed a commit that referenced this pull request Mar 1, 2021
Ravin Kohli: Merge pull request #112 from franchuterivera/refactor_de…
2aa06dd 
…velopment_ADD_109
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ravinkohli ravinkohli ravinkohli approved these changes

Assignees
No one assigned
Labels
enhancement New feature or request
Projects
Roadmap
Done
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
2 participants
@franchuterivera @ravinkohli