WIP: DON'T MERGE: Ultimate test setup #1899
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…ge. These are global options
- Implement ItemCommand, ItemCommit - Implement first test repos: BasicGit, BasicMixed - several little fixes and changes - add TODOs
- Fix addurl - BasicMixed now works too and is the same as old BasicAnnexTestRepo
RF: introduce helper fucntions for command calls
…to ItemFile, let TestRepo check for independent repos
ENH: represent remotes and branches ENH: Implement physical assertions for ItemRepo TST: Test recursive execution of everything for TestRepo class and BasicGit respectively
yarikoptic
reviewed
Oct 13, 2017
|
|
||
|
|
||
| @auto_repr | ||
| class ItemInfoFile(ItemFile): |
There was a problem hiding this comment.
invaluable concern: use "Item" as a suffix (if at all) not as a prefix, to stay inline with other places (AnnexRepo, etc)
There was a problem hiding this comment.
here is an example for attrs
>>> import attr
>>> @attr.s
... class C(object):
... x = attr.ib(default=42)
... y = attr.ib(default=attr.Factory(list))so we could do just that from testrepos import items, and use items.File, or if really valuing our bytes, from testrepos import items as it; it.File
datalad/tests/testrepos/items.py
Outdated
|
|
||
| if self.branches: | ||
| assert self.commits | ||
| # TODO: Not necessarily vice versa? Could be just detached HEAD, I guess |
There was a problem hiding this comment.
make sure that it also asserts on the states of the branches
although git-annex branch might be tricky, since many git-annex operations just adjust the timestamp within the log of the git-annex although no content being dropped etc. E.g. 'annex fsck' would change git-annex branch.
|
the failing test in the env with lots of debugging etc |
…from the one used for instantiation BF: Use OrderedDict for TestRepo's items ENH: Allow for other TestRepos to be used within definitions like Items ENH/TST: Add a rudimentary MixedSubmodules TestRepo to test enhancement - MixedSubmodules itself remains TODO!
Codecov Report
@@ Coverage Diff @@
## master #1899 +/- ##
==========================================
+ Coverage 85.72% 89.74% +4.01%
==========================================
Files 265 287 +22
Lines 31598 32936 +1338
==========================================
+ Hits 27088 29558 +2470
+ Misses 4510 3378 -1132
Continue to review full report at Codecov.
|
…_class__ inplace instead. TST: Use several BasicMixed in MixedSubmodules to test enahncement
BF: use relative paths for submodule-add ENH: Have MixedSubmodules almost resemble old SubmoduleDataset Add(and remove) some TODOs plus minor fixes
BF: Let ItemRepo know if datalad special remote was enabled
ENH: Lazily provide persistent TestRepos and files ENH: Use cloning in MixedSubmodules to fully resemble old SubmoduleDataset
RF: Move code into a helper function
…n additional items that were instantiated during creation of that item
RF: Something to clone from needs to be an ItemRepo ENH: Start accounting for non-initialized submodules TST: Last 'old' TestRepo almost done
BF: When reading remotes from config, do the right thing if we are in a submodule with .git file ENH: Get branch (and commit) information from cloning source RF: Make annex-init a method of ItemRepo to be called by its create() as well as when initializing a submodule ENH: Add command 'submodule update' ENH: Finish 'old' TestRepo with nested submodules BF: When calling super().__init__ for items, pass 'check_definition'
We need to reference persistent ones in other TestRepos using them. Therefore the laziness wasn't real, since the simple import of repos.py instantiated all of them, that were referenced this way. Now, the 'src' parameter of an ItemRepo isn't required to be an ItemRepo anymore, but is allowed to be a callable. get_persistent_testrepo(cls, attr) now delivers a callable, that will create the instance if called and it wasn't created yet. In addition this function now allows to not only return the instance of 'cls', but its attribute 'attr' instead.
bpoldrack
force-pushed
the
rf-testrepos
branch
from
486e133
to
dd19a8a
Compare
…apt _update_from_src accordingly
bpoldrack
force-pushed
the
rf-testrepos
branch
from
dd19a8a
to
115993f
Compare
Closed
…ble types in PY3 and we don'T actually need a set here
V6 BF: Clean up failed attempt to create a particular persistent TestRepo instance
…pt decorators and tests accordingly. This is needed, since we have TestRepos (MixedSubmodulesOld*), that are invalid in V6 right from the beginning. So it's not about skipping the actual tests, that are supposed to operate on several TestRepos. This might be somewhat confusing - don't throw away this approach easily!
11 tasks
|
Don't see myself finishing this anytime soon. Thus closing. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
You may look, but don't touch!
TestRepobase classItemsubclassesTestRepo's,ItemRepo's andItemFile'sassert_intactmethod (mostly done)with_testreposTestRepowith meaningful messages to ease doing so for everybodytestrepos.utils.pyGitRepo/AnnexRepo) to show offItemCommands and some properties until they are actually required. Do this while adapting tests and creating additionalTestRepos. It's important to make use of what's already available ASAP.Provide new concept for running unit tests in different setups.
short overview of the goals
New
TestReposubclasses are capable of setting up pretty much everything you can think of, including untracked files, independent repos (or even an entire forrest of hierarchies at once), staged and unstaged modifications, ... everything. This used to be not possible, since we relied on them being cloneable repositories.They are delivered by
with_testreposas an object that contains all information about the SHOULD-BE state of the entire setup. Tests now can base their assertions on properties of theTestRepo. Thereby it's easy to write tests that can actually run on anything being thrown at them. They don't need (hard-)coded assumptions about the tesrepos that were known to the developer when writing the test. And they also don't need to read properties from FS, which basically means to make the IS state the SHOULD-BE state.They provide full automatic integrity testing. No need for a test developer to each time think about, what could possibly have changed and come up with a complete set of assertions to make sure nothing has changed.
This also means there's now the possibility to just define intended changes and automatically run a full set of assertions making sure that those changes and those changes ONLY actually happened. Again: No need by the developer to always come up with the necessary assertions to make that sure.
If used via new
with_testreposdecorator it is ensured that testrepos can't be left messed up by tests and still being used by other tests, too. However, if the test requests a "read-only" instance of aTestRepothey will get a persistent one, that isn't always recreated and used by others, too. Those persistent ones will check themselves after such a test (and raise of course if check failed) and will be automatically recreated from scratch for the next test that wants to use them.Finally, the definition of new test setups is pretty easy, I think. To help with that even further there are supposed to be extensive checks on the definitions, generating pretty precise messages, that are telling what's wrong with the definition.
Implicit feature: There is basically no need for additional unit tests on
TestRepoand the subclasses ofItem. They implicitly test themselves as soon as they are used in a subclass ofTestRepo. This subclass just needs to be instantiated. The "object consistency" part ofassert_intact(which is autmatically and recursively called for all item of aTestRepo) does that. Whatever unit test you can come up with that actually contains assertions that are not tested anyway, implies that either those assertions should be added in a general way in the appropriateassert_intactmethod or it's about a not yet tested setup, meaning you should simply define a newTestRepoinstead. The latter will automatically test the code intestrepomodule against that situation and furthermore this also leads to us testing DATALAD against that untested situation!