Windows: annex has issues with links in submodules so do not code testing get/install in submodule repos for now :-/

[yarikoptic]

http://git-annex.branchable.com/bugs/downloads_load___40__from_url__41___to_incorrect_directory_in_a_submodule/

> git clone --recurse git://github.com/yarikoptic/datalad -b nf-test-repos
...
> cd datalad/datalad/tests/testrepos/basic/r1
> ls -l test-annex.dat
lrwxrwxrwx 1 yoh yoh 186 Feb 23 15:56 test-annex.dat -> .git/annex/objects/zk/71/SHA256E-s4--181210f8f9c779c26da1d9b2075bde0127302ee0e3fca38c9a83f5b1dd8e5d3b.dat/SHA256E-s4--181210f8f9c779c26da1d9b2075bde0127302ee0e3fca38c9a83f5b1dd8e5d3b.dat
> git co master    
Switched to branch 'master'

$> git annex get test-annex.dat
(merging origin/git-annex into git-annex...)
(recording state in git...)
get test-annex.dat (from web...) 
../../../../../.git/modules/datalad/te 100%[=============================================================================>]       4  --.-KB/s   in 0s     
ok
(recording state in git...)

# indeed upstairs
$> ls -l ../../../../../.git/modules/datalad/tests/testrepos/modules/basic/r1/annex/objects/zk/71/SHA256E-s4--181210f8f9c779c26da1d9b2075bde0127302ee0e3fca
38c9a83f5b1dd8e5d3b.dat/SHA256E-s4--181210f8f9c779c26da1d9b2075bde0127302ee0e3fca38c9a83f5b1dd8e5d3b.dat
-r-------- 1 yoh yoh 4 Feb 23 15:57 ../../../../../.git/modules/datalad/tests/testrepos/modules/basic/r1/annex/objects/zk/71/SHA256E-s4--181210f8f9c779c26da1d9b2075bde0127302ee0e3fca38c9a83f5b1dd8e5d3b.dat/SHA256E-s4--181210f8f9c779c26da1d9b2075bde0127302ee0e3fca38c9a83f5b1dd8e5d3b.dat

$> acpolicy git-annex
git-annex:
  Installed: 5.20150205+git57-gc05b522-1~nd80+1
  Candidate: 5.20150205+git57-gc05b522-1~nd1
  Version table:
 *** 5.20150205+git57-gc05b522-1~nd80+1 0
        500 http://neuro.debian.net/debian-devel/ jessie/main amd64 Packages
        100 /var/lib/dpkg/status

$> acpolicy git      
git:           
  Installed: 1:2.1.4-2.1
  Candidate: 1:2.1.4-2.1
  Version table:
     1:2.1.4+next.20141218-2 0
        300 http://http.debian.net/debian/ experimental/main amd64 Packages
 *** 1:2.1.4-2.1 0
        900 http://http.debian.net/debian/ jessie/main amd64 Packages
        600 http://http.debian.net/debian/ sid/main amd64 Packages
        100 /var/lib/dpkg/status

[yarikoptic]

actually workaround for testing would be to clone that submodule into another temp directory -- then annex works properly

[joeyh]

http://git-annex.branchable.com/bugs/Git_annexed_files_symlink_are_wrong_when_submodule_is_not_in_the_same_path/

Due to the way git submodules work, if you intend to use a git-annex
repository as a submodule, you can only use it as a submodule, and it
has to always be located in the same place relative to its parent
module.

I do not see a way around this. (Other than using direct mode.)

see shy jo

[yarikoptic]

ah right -- just stoke me that indeed if repository is initialized as an existing submodule (not added anew) its .git is just a file containing a pointer to "upstairs" .git/ directory ... lame :-/ And the only way around this was if annex was not hidden under .git/annex but rather was an independent directory in the root of the repository (e.g. as .annex).

So is important to keep it in mind as a limitation of git-annex then (existing independent annex'ed repos can't be used as submodules), heh heh

[yarikoptic]

@joeyh but how much git-annex relies on hardcoded .git/annex path? ;) i.e. may be some time there could be an option to initialize it outside of the .git/? ;)

[yarikoptic]

Even though I have marked it as 'wontfix' for now: I have initiated a new milestone (Git-annex: big refactoring 1) where we could collate those issues we identified, you agree that they are worth tackling, but it would require a massive refactoring of annex code/structure. For now I have placed there #32 since you are already dealing with it somewhat, and it might be worth moving this #44 there?

[joeyh]

Yaroslav Halchenko wrote:

ah right -- just stoke me that indeed if repository is initialized as an
existing submodule (not added anew) its .git is just a file containing a
pointer to "upstairs" .git/ directory ... lame :-/

It might work to replace the "gitlink" file with a symlink.

see shy jo

[yarikoptic]

replacing .git with a symlink sounds like begging for a trouble ;) but I could be proven wrong (that git folks envisioned this crippled .git file as a cross-platform symlink replacement and otherwise test proper git functioning whenever .git is a symlink to somewhere high up ;))

[yarikoptic]

BTW -- in #40 we were considering submodules to mitigate large collections... with #44 I guess it wouldn't be the best option really since would disallow dealing with a particular subset of annexes

[joeyh]

(My email reply was rejected by github for some reason. Repost.)

replacing .git with a symlink sounds like begging for a trouble ;)

I tried it, and it seems to work ok, at least for the limited number of
things I know to do with submodules. Note that after converting .git to
a symlink, I had to edit .git/config and remove the core.worktree
setting.

See:
http://git-annex.branchable.com/bugs/Git_annexed_files_symlink_are_wrong_when_su
bmodule_is_not_in_the_same_path/#comment-8eeb2ec8891cf95e9d4e7623000299e4

BTW -- in #40 we were considering submodules to mitigate large collections...
with #44 I guess it wouldn't be the best option really since would disallow
dealing with a particular subset of annexes

It would be ok using the above approach, a submodule then uses the same
symlink paths as does a regular repository.

[joeyh]

I've added submodule support to git-annex

http://git-annex.branchable.com/submodules/

Still needs testing, and it's known to not work in FAT or Windows yet,
although I think I can get it working in those situations too.

see shy jo

[yarikoptic]

cool, thank you @joeyh . tested with a fresh build of git-annex(avail from neurodebian -devel now for jessie/sid) and indeed works smoothly. I will wait for your go on windows/OSX to consider it resolved fully. Cheers!

[joeyh]

Yaroslav Halchenko wrote:

cool, thank you @joeyh . tested with a fresh build of git-annex(avail from
neurodebian -devel now for jessie/sid) and indeed works smoothly. I will wait
for your go on windows/OSX to consider it resolved fully. Cheers!

Submodules should work portably since cf903d5a3c42afd569ff0c1a2202c0e0168b6f8f

see shy jo

[yarikoptic]

splendid. thank you @joeyh. I will close it then

[yarikoptic]

not yet fully resolved, thus will reopen for now
http://git-annex.branchable.com/submodules/#comment-9d969fcaa6b5276b21338905d55c7cd7

[joeyh]

Yaroslav Halchenko wrote:

not yet fully resolved, thus will reopen for now
http://git-annex.branchable.com/submodules/#
comment-9d969fcaa6b5276b21338905d55c7cd7

I'm sorry, but as a bug report, this is completely incoherent.

Happy to help if I can get a textual description of how to reproduce
the problem, reported as a bug and not an image in a forum comment.

(Also, why reopen a bug report? Bug numbers are not a scarce resource,
while time to read through a bunch of messages to get to the
problem-de-jour that is somehow releated to the original bug report is a
scarce resource.)

see shy jo