Improve dataset test infrastructure #3450
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
pmeier
changed the title
Codecov Report
@@ Coverage Diff @@
## master #3450 +/- ##
=======================================
Coverage 76.11% 76.11%
=======================================
Files 105 105
Lines 9697 9697
Branches 1556 1556
=======================================
Hits 7381 7381
Misses 1836 1836
Partials 480 480 Continue to review full report at Codecov.
|
facebook-github-bot
pushed a commit
that referenced
this pull request
Mar 4, 2021
Summary: * always use default config as base * fix test_all_configs decorator * lint * add a utility function to create a random string * move output check of inject_fake_data to dedicated method * always disable download and extract functionality Reviewed By: fmassa Differential Revision: D26756261 fbshipit-source-id: a720dce48148d4d69678d43a2c5ec50ac92d69a0
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Currently, we only use the default config if the user didn't specify any
CONFIGS. This means on the other hand that if he specifies a config, only the specified values are forwarded toinject_fake_data.Since it may be the case that the user only specifies a part of available parameters and wants to leave the others on their default value, we should include them. With this PR the default config is always used as base and simply updated by the config of the user.
Sometimes the image or video filename is a random string. This PR adds a
create_random_string()function for a convenient interface.The output checking of the
inject_fake_data()method takes up most of the body of thecreate_dataset()and thus drowning the actual important parts. In order to make the base class more maintainable, this PR moves the checking into a dedicated function.This PR splits the patching of the download / extract and check logic and only makes the patching of the check logic optional. Before we might run into scenarios where the
test_not_found_or_corruptedtest actually tried to download the dataset. This happens for example fordatasets.SBUsince it usesdownload=Trueas default and in contrast to most other datasets.