Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TST: annexrepo: Update test for "no commits" sub-repo fix in Git #3493

Closed
wants to merge 4 commits into from

Conversation

kyleam
Copy link
Contributor

@kyleam kyleam commented Jun 26, 2019

As of Git 2.22.0, specifically b22827045e (dir: do not traverse
repositories with no commits, 2019-04-09), 'git ls-files' now treats a
repository on an unborn branch as a repository rather than a
directory.

Fixes #3490.

kyleam added a commit that referenced this pull request Jun 26, 2019
Note this is doing some ugly things to the upstream git setup because
of unresolved issues with running the upstream git build on Xenial
rather than Trusty.
@kyleam
Copy link
Contributor Author

kyleam commented Jun 26, 2019

A Git v2.22.0 build (with this PR and gh-3492) is set up at: https://travis-ci.org/datalad/datalad/builds/550858203

@kyleam
Copy link
Contributor Author

kyleam commented Jun 26, 2019

The updated test fails in the 2.22.0 Travis build. I'm afraid this is due to the same issue I haven't figured out in gh-3476. When I run it locally, both Git 2.21.0 and 2.22.0 behave as expected. For 2.22.0, this means that "submod/" is included in the untracked files, the same result that the azure build posted in gh-3490 shows.

So my local tests and conda's azure tests agree. There seems to be something wonky with the upstream Git setup on Travis.

@kyleam
Copy link
Contributor Author

kyleam commented Jun 26, 2019

So my local tests and conda's azure tests agree. There seems to be something wonky with the upstream Git setup on Travis.

Or, somehow the bundled Git is getting used despite setting DATALAD_USE_DEFAULT_GIT=1. I switched to git-annex-standalone from neurodebian, and I can trigger this. That gives me something to dig into locally.

failure
% DATALAD_USE_DEFAULT_GIT=1 python -m nose datalad/support/tests/test_annexrepo.py:test_AnnexRepo_status
F
======================================================================
FAIL: datalad.support.tests.test_annexrepo.test_AnnexRepo_status
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/kyle/src/python/venvs/datalad/lib/python3.5/site-packages/nose/case.py", line 198, in runTest
    self.test(*self.arg)
  File "/home/kyle/src/python/datalad/datalad/tests/utils.py", line 615, in newfunc
    return t(*(arg + (filename,)), **kw)
  File "/home/kyle/src/python/datalad/datalad/tests/utils.py", line 615, in newfunc
    return t(*(arg + (filename,)), **kw)
  File "/home/kyle/src/python/datalad/datalad/support/tests/test_annexrepo.py", line 2153, in test_AnnexRepo_status
    _test_status(ar)
  File "/home/kyle/src/python/datalad/datalad/support/tests/test_annexrepo.py", line 1996, in _test_status
    eq_(stat, ar.get_status())
AssertionError: {'type_changed': [], 'added': [], 'untracked': ['fifth', 'sub/third', 'submod/'], 'modified': ['first', 'second'], 'deleted': []} != {'type_changed': [], 'added': [], 'untracked': ['fifth', 'sub/third'], 'modified': ['first', 'second'], 'deleted': []}
-------------------- >> begin captured logging << --------------------
datalad.utils: Level 5: Importing datalad.utils
datalad.utils: DEBUG: Maximal length of cmdline string (adjusted for safety margin): 1252864
datalad.utils: Level 5: Done importing datalad.utils
datalad.cmd: Level 9: Will use default git /home/kyle/.guix-profile/bin/git
datalad.cmd: Level 9: Running: ['git', 'version']
datalad.cmd: Level 8: Finished running ['git', 'version'] with status 0
datalad.cmd: Level 9: Running: ['git', 'config', '-z', '-l', '--show-origin']
datalad.cmd: Level 8: Finished running ['git', 'config', '-z', '-l', '--show-origin'] with status 0
datalad.ui: Level 5: Starting importing ui
datalad.ui.dialog: Level 5: Starting importing ui.dialog
datalad.ui.dialog: Level 5: Done importing ui.dialog
datalad.ui: Level 5: Initiating UI switcher
datalad.ui: DEBUG: UI set to SilentConsoleLog(out=<StringIO>)
datalad.ui: Level 5: Done importing ui
--------------------- >> end captured logging << ---------------------

----------------------------------------------------------------------
Ran 1 test in 1.797s

FAILED (failures=1)

@kyleam kyleam force-pushed the annexrepo-status-nocommits branch from 615e1dd to c4bb132 Compare June 26, 2019 18:53
@kyleam
Copy link
Contributor Author

kyleam commented Jun 26, 2019 via email

@kyleam kyleam force-pushed the annexrepo-status-nocommits branch from c4bb132 to ff2bc90 Compare June 26, 2019 19:13
@kyleam
Copy link
Contributor Author

kyleam commented Jun 27, 2019

The appveyor failures are the familiar test_install.test_datasets_datalad_org failures.

@kyleam kyleam added the WIP work in progress label Jun 28, 2019
We'll use this to add the bundle git version to external_versions.
Even when DATALAD_USE_DEFAULT_GIT is set, there are some commands that
will use the bundled Git (e.g., "git annex status").  In these cases,
inspecting cmd:git isn't relevant, and we need a way to check the
bundled version.
As of Git 2.22.0, specifically b22827045e (dir: do not traverse
repositories with no commits, 2019-04-09), 'git ls-files' now treats a
repository on an unborn branch as a repository rather than a
directory.

Fixes datalad#3490.
@kyleam kyleam force-pushed the annexrepo-status-nocommits branch from ff2bc90 to 1cd955d Compare June 28, 2019 12:40
@codecov
Copy link

codecov bot commented Jun 28, 2019

Codecov Report

❗ No coverage uploaded for pull request base (0.11.x@27b9aa4). Click here to learn what that means.
The diff coverage is 76.92%.

Impacted file tree graph

@@            Coverage Diff            @@
##             0.11.x    #3493   +/-   ##
=========================================
  Coverage          ?   77.41%           
=========================================
  Files             ?      252           
  Lines             ?    33507           
  Branches          ?        0           
=========================================
  Hits              ?    25941           
  Misses            ?     7566           
  Partials          ?        0
Impacted Files Coverage Δ
datalad/support/external_versions.py 75.55% <100%> (ø)
datalad/support/tests/test_annexrepo.py 96% <66.66%> (ø)
datalad/cmd.py 85.61% <88.88%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 27b9aa4...1cd955d. Read the comment docs.

1 similar comment
@codecov
Copy link

codecov bot commented Jun 28, 2019

Codecov Report

❗ No coverage uploaded for pull request base (0.11.x@27b9aa4). Click here to learn what that means.
The diff coverage is 76.92%.

Impacted file tree graph

@@            Coverage Diff            @@
##             0.11.x    #3493   +/-   ##
=========================================
  Coverage          ?   77.41%           
=========================================
  Files             ?      252           
  Lines             ?    33507           
  Branches          ?        0           
=========================================
  Hits              ?    25941           
  Misses            ?     7566           
  Partials          ?        0
Impacted Files Coverage Δ
datalad/support/external_versions.py 75.55% <100%> (ø)
datalad/support/tests/test_annexrepo.py 96% <66.66%> (ø)
datalad/cmd.py 85.61% <88.88%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 27b9aa4...1cd955d. Read the comment docs.

@kyleam kyleam removed the WIP work in progress label Jun 28, 2019
@kyleam
Copy link
Contributor Author

kyleam commented Jun 28, 2019

Test with upstream Git looks good. AppVeyor failure is due to test_install.test_datasets_datalad_org. Will merge, dropping the last commit.

@kyleam kyleam closed this Jun 28, 2019
@yarikoptic yarikoptic added this to the Release 0.11.6 milestone Jul 26, 2019
yarikoptic added a commit that referenced this pull request Jul 31, 2019
0.11.6 (Jul 30, 2019) -- am I the last of 0.11.x?

Primarily bug fixes to achieve more robust performance

Fixes

- Our tests needed various adjustments to keep up with upstream
  changes in Travis and Git. ([#3479][]) ([#3492][]) ([#3493][])

- `AnnexRepo.is_special_annex_remote` was too selective in what it
  considered to be a special remote.  ([#3499][])

- We now provide information about unexpected output when git-annex is
  called with `--json`.  ([#3516][])

- Exception logging in the `__del__` method of `GitRepo` and
  `AnnexRepo` no longer fails if the names it needs are no longer
  bound.  ([#3527][])

- [addurls][] botched the construction of subdataset paths that were
  more than two levels deep and failed to create datasets in a
  reliable, breadth-first order.  ([#3561][])

- Cloning a `type=git` special remote showed a spurious warning about
  the remote not being enabled.  ([#3547][])

Enhancements and new features

- For calls to git and git-annex, we disable automatic garbage
  collection due to past issues with GitPython's state becoming stale,
  but doing so results in a larger .git/objects/ directory that isn't
  cleaned up until garbage collection is triggered outside of DataLad.
  Tests with the latest GitPython didn't reveal any state issues, so
  we've re-enabled automatic garbage collection.  ([#3458][])

- [rerun][] learned an `--explicit` flag, which it relays to its calls
  to [run][[]].  This makes it possible to call `rerun` in a dirty
  working tree ([#3498][]).

- The [metadata][] command aborts earlier if a metadata extractor is
  unavailable.  ([#3525][])

* tag '0.11.6': (56 commits)
  [DATALAD RUNCMD] make update-changelog
  finalize CHANGELOG.md entry and boost version
  BF(DOC): close [create] with [] to not cause WARNING by md-strict pandoc
  CHANGELOG.md: Link entry from b3e8adb
  CHANGELOG.md: Add entry for gh-3547
  CHANGELOG.md: Add entry for gh-3561
  CHANGELOG.md: Add link for addurls
  RF: inform about special remotes based on autoenable config
  CHANGELOG.md: Second batch for 0.11.6
  BF: addurls: Process datasets in a stable, breadth-first order
  BF: addurls: Fix construction of nested subpaths
  TST: addurls: Don't hard-code path separator
  BF(TST): skip test_v7_detached_get in direct mode - fails to annex upgrade
  TST: benchmark-travis-pr: Swap 'pip install' and 'git show'
  TST: benchmark-travis-pr: Move repeated logic to run_asv()
  TST: benchmark-travis-pr: Support other bases
  TST: benchmark-travis-pr: Tweak message about current HEAD
  TST: benchmark-travis-pr: Simplify two git commands into one
  TST: benchmark-travis-pr: Reorder and break up lines
  TST: benchmark-travis-pr: Move command for running asv into function
  ...
@kyleam kyleam deleted the annexrepo-status-nocommits branch February 11, 2020 20:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants