Possible git-receive-pack.exe installation issue #16

Closed
pleriche opened this Issue May 26, 2012 · 28 comments

5 participants

@pleriche

Hi,

The installer (Git-1.7.10-preview20120409.exe) installs an executable file for libexec\git-core\git-receive-pack.exe on Windows XP, but under Vista and later the installed git-receive-pack.exe is a shortcut to git.exe instead.

The shortcut mechanism works fine under a local command prompt, but seems to cause a problem when attempting to push to the repository via a non-interactive ssh session, e.g. when using TortoiseGit:

The command:
git.exe push --progress "origin" master:master

Returns this error:
git: 'c:/myrepos/reponame.git' is not a git command. See 'git --help'.
fatal: The remote end hung up unexpectedly

It appears that under a non-interactice ssh session that the shortcut does not pass "receive-pack" as first parameter to git.exe, but the name of the repository instead.

As a workaround, the following can be run on the local computer to fix the issue for TortoiseGit:
git config --global remote.origin.receivepack "git receive-pack"

As mentioned earlier, this workaround is not required on Windows XP, since the installer installs an executable and not a shortcut to git.exe.

More on this issue here:
http://stackoverflow.com/questions/2550422/git-failed-at-pushing-to-remote-server-repository-path-is-not-a-git-comma

Is this an SSH issue, a TortoiseGit issue, or something that should be addressed in the installer?

Thanks!

Pierre

@hvoigt
MSysGit - the development behind Git for Windows member

Could you provide a short script snippet (sequence of commands) that creates such a setup in a directory? Then we have a basis on which to decide and know the precise setup you are talking about. Currently you description leaves to many information open for me to decide on the question.

@pleriche

I've done some more digging on this issue, and I believe that the installer is installing a symbolic link to git.exe for git-receive-pack.exe on Vista and later, but on XP it installs a physical executable.

I am using TortoiseGit and CopSSH 3.03. From what I can tell it looks like CopSSH 3.03 does not support symbolic links, because when TortoiseGit invokes "git-receive-pack blah" CopSSH executes "git blah" instead of "git receive-pack blah" on the remote server.

If this is in fact true then it would be unfortunate, because CopSSH 3.03 is the last free version available. I have not tested other SSH implementations, so CopSSH could not be unique in this regard. I wonder if it would not be possible to have an option at installation to install the physical files instead of symbolic links, even on Vista and later?

FWIW here are the steps I follow when I install a server and workstation:

On the server:
1) Install MsysGit, selecting the "Run Git and included Unix tools from the Windows Command Prompt" when asked.
2) Add the git-core folder to the environment path (typically C:\Program Files\Git\libexec\git-core).
3) Install CopSSH 3.03
4) Go to the administrative control panel and create Windows users for all git users. Give them passwords.
5) Activate all the Windows users under the CopSSH "01. Activate a User" option. Untick the "Create link to user's real home directory" option when doing so. Create new public and private keys if you don't have keys already.
6) If you already have private and public keys, copy them to the ...Program Files\ICW\home{username} folder and call them {username}.key and {username}.key.pub. Also copy the public key to ...ICW\home{username}.ssh\ and rename it to "authorized_keys".
7) Reboot the server so the changes to the path can take effect.
8) Create your Git repositories and make sure that you grant appropriate access rights to the repository folders for all users created in step 4.
9) If you need remote access to the server, forward the SSH protocol (port 22, UDP and TCP) to the machine.

On the workstation:
10) Copy the private key generated in 5 (or the one you used in 6) to the .ssh subfolder in your user profile (typically c:\Users\Administrator.ssh) and rename it to "id_rsa".
11) If you've generated the private key with a passphrase and you wish to remove it, run "ssh-keygen -p" in a command prompt and specify a blank new passphrase.
12) If you're using TortoiseGit, remember to specify in the installer that you're using OpenSSH.
13) To check out a repository: git clone {username}@{servername}:{repodriveletter}:/{repopath}

These steps work fine when the server runs XP, but if it runs Vista or later any attempt to push via Tortoisegit will fail with "git: 'c:/myrepos/reponame.git' is not a git command. See 'git --help'."

Thanks!

Pierre

@sschuberth
MSysGit - the development behind Git for Windows member

AFAIK there should not be any application-level support for symbolic links necessary, they should be completely transparent to the application.

Moreover, if the installer cannot create symbolic links it falls back to creating hard links before trying copies. On your XP machine, please check whether you are having hard links or real copies. Unfortunately, Windows Explorer has no way of telling you this by default. I highly recommend installing [1] which gives you overlay icons for hard links / symbolic links and also some information in the file's properties dialog.

Another option to check whether hard link are used would be to re-run the installer on your XP machine as "Git-1.7.10-preview20120409.exe /log c:\git.log" and look at the log file for messages like "Creating symbolic link failed, will try a hard link" but no messages like "Creating hardlink failed, will try a copy".

If it turns out that you in fact have hard links in place, and CopSSH can deal with them but not with symbolic links, I have no problem with dropping symbolic links in favor of hard links on Vista and above as they only give little benefit over symbolic links.

[1] http://schinagl.priv.at/nt/hardlinkshellext/hardlinkshellext.html

@pleriche

Thanks for looking into this. I can confirm that git-receive-pack.exe is a hard link on the Windows XP machine.

When I copied across the libexec\git-core folder to the Windows 2008R2 server from the XP machine I believe it created normal files, which fixed the problem for me.

As an experiment I have now deleted the git-receive-pack.exe file on the server and created a hard link with the following command:
C:\Program Files (x86)\Git\libexec\git-core>mklink /H git-receive-pack.exe ..\..\bin\Git.exe

With the above hard link in place I can push to the repository successfully. However, if I create a symbolic link instead by omitting the /H parameter, then an attempt to push to the repository fails with the error:
'c:/repos/test.git' is not a git command. See 'git --help'.

@sschuberth
MSysGit - the development behind Git for Windows member

Thanks for testing. If there are no objections to remove symbolic link support, I'll some up with a patch that removes it from the installer in favor using hard links if available, and copies of not.

@dscho
MSysGit - the development behind Git for Windows member

I'd like to understand this issue a bit better first. My hunch is that git.c's main() function does not get a correct argv when it is called via CopSSH as a symbolic link. If that is the case, I do not think it is CopSSH's fault and I think that in that case symbolic links must be removed in order to ensure a working setup.

@pleriche

As a test I've modified git.c's main() function to dump the input parameters to a text file.

With a hard link the following parameter values are logged when attempting to do a push via TortoiseGit over SSH:
1 of 2: C:\Program Files (x86)\Git\libexec\git-core\git-receive-pack.exe
2 of 2: c:/repos/test.git

With a symbolic link the following parameters are logged:
1 of 2: C:\Program Files (x86)\Git\libexec\git-core\git.exe
2 of 2: c:/repos/test.git

It seems that in the case of a hard link that the full path of the hard link is passed as first parameter (as expected), whereas with a symbolic link that the full path to the link target is passed instead.

Something else that I have noticed is that if I run git-receive-pack locally from a command prompt or bash that the correct parameter values are logged for both the hard link and symbolic link. I therefore suspect that SSH (or CopSSH in particular) may be a contributing factor.

@dscho
MSysGit - the development behind Git for Windows member

Great analysis!

That also proves that we must revert the symbolic links in the installer. Apparently the person who came up with the idea of using symbolic links did not really test things...

@kusma
MSysGit - the development behind Git for Windows member

While reverting the patch might very well be the solution, it seems that something out of the ordinary is going on here. I just wrote a small test-program that prints argv[0], compiled it to a.exe, set up a symlink to a.exe called b.exe, and executed b.exe. It printed "b.exe" when invoked through the symlink, and "a.exe" when not.

Could it be that CopSSH somehow tries to resolve symlinks themselves or something?

@dscho
MSysGit - the development behind Git for Windows member

FWIW I tried a simple "git revert 7fdd58f" but it conflicts rather heavily...

@dscho
MSysGit - the development behind Git for Windows member

I guess that we should hard-link git-receive-pack and git-upload-pack in the least, to avoid surprises with CopSSH and similar programs.

@dscho
MSysGit - the development behind Git for Windows member

Okay, in case we want to shed all the symbolic link stuff: https://github.com/msysgit/msysgit/commits/dont-symlink (I personally do not think it is worth the hassles, and I have to admit that I am disappointed by the quality of the research by the person who suggested using symbolic links in the first place.)

@dscho
MSysGit - the development behind Git for Windows member

@kusma I just realized why I did not like CopSSH in the first place: it is obviously based on Open Source software but is itself so closed you cannot even diagnose what it really does with symbolic links.

@pleriche

Results from using "ssh -T user@server" and running "git-receive-pack test.git" from the prompt:

Hard link:
1 of 2: C:\Program Files (x86)\Git\libexec\git-core\git-receive-pack.exe
2 of 2: test.git

Symbolic link:
1 of 2: C:\Program Files (x86)\Git\libexec\git-core\git.exe
2 of 2: test.git

Since both produce the same results when run from a local command prompt, I think the blame lies with CopSSH. However, a cursory web search did not yield any mention of a CopSSH bug in the handling of symbolic links, so the observed behaviour may well be intentional.

@hvoigt
MSysGit - the development behind Git for Windows member

@pleriche after this analysis do we agree that this issue is mainly a CopSSH one? Since filing issues with them is customer only could you please file an issue there (probably referencing this issue) and we can close it here?

Since their site is so closed I think we should not even try to support use cases with their software. It seems they make heavy use of open source software add a little bit to it to provide a software you pay for. Since we have no means to investigate properly and we are providing volunteer work it does not seem appropriate for me to support them. So for all problems resulting from uses with their software I think they should also do the support for it.

@hvoigt
MSysGit - the development behind Git for Windows member

IOW I vote for not removing the symbolic link support but just leaving everything as is. What do you think?

@pleriche

@hvoigt Unfortunately I am not a customer either... well not in the sense that I've bought anything from them. The version of CopSSH that I'm using is 3.0.3, which is the last free version. I am fairly certain that even if this was brought to their attention that they probably won't fix the free version (which is already a few years old).

That said, I believe that the free version of CopSSH is still fairly popular, and in my experience is ridiculously easy to configure for use with msysGit. Apart from this one problem, it also works very well. When I searched the net for step by step instructions on how to configure a Git server for Windows I found two guides and both mentioned this version of CopSSH, which is how I came to use it.

@dscho
MSysGit - the development behind Git for Windows member

@pleriche IIRC the instructions how to use CopSSH are somewhere on the Wiki, right? If so, the resolution should be a matter of telling people to

cd /bin/
rm git-receive-pack.exe
ln git.exe git-receive-pack.exe
rm git-upload-pack.exe
ln git.exe git-upload-pack.exe

and enjoy, no? Can you please

1) verify that that works, and
2) change the instructions?

Thanks!

@pleriche

The following works for me, and has the added advantage that you don't have to change the Windows path manually (provided you pick the "Run Git and included Unix tools from the Windows Command Prompt" in the msysGit installer):

cd /bin
ln -f ../libexec/git-core/git-upload-pack.exe git-upload-pack.exe
ln -f git.exe git-receive-pack.exe

Now that I know about this issue (and the appropriate workaround) it is no longer a problem for me. Nevertheless, it remains a bit of a "gotcha" and I'm sure there will be others that will also struggle with this: Even if the wiki is updated (will take a look at that next), there are other guides on the net that won't be. Personally, I believe that having the installer create those two hard links would be the best solution.

@dscho
MSysGit - the development behind Git for Windows member

@pleriche that's great!

Could you also have a look at /share/WinGit/install.iss whether you can come up with a patch to always hard-link upload-pack and receive-pack? As you provided the Wiki page, it is not overly pressing, but as you said, it would be nice to have that change in the installer.

@sschuberth
MSysGit - the development behind Git for Windows member

@pleriche Also thanks from my side for the Wiki entry.

As I see it, we have 3 options to fix this:

1) Completely remove Symlinks in favor of Hardlinks form the installer.
2) Add a command line option to the installer ("/nosymlinks") to force falling back to Hardlinks.
3) Always hardlink upload-pack and receive-pack as Dscho suggested.

While 2) is a flexible solution, its drawback is that is has to be documented. I don't like 3) as it adds magic to the installer. Both 2) and 3) slightly increase the installer's complexity. So I'd vote for 1) as it

a) removes complexity from the installer,
b) Symlinks can be created by Admins only,
c) Symlinks have very limited advantages over Hardlinks in our case.

@pleriche pleriche added a commit to pleriche/msysgit that referenced this issue Jun 1, 2012
@pleriche pleriche Issue #16 workaround: Added steps to the installer to create git-uplo…
…ad-pack.exe and git-receive-pack.exe hard links in the \bin folder. This fixes the error that may occur when attempting to push to a repository hosted on a Windows Vista (or later) server due to an apparent compatibility issue between CopSSH and symbolic links.

Signed-off-by: Pierre le Riche <github@pleasedontspam.me>
58d7212
@pleriche

@dscho I've submitted a pull request for the updated install.iss that creates hard links for git-receive-pack and git-upload-pack. I've tested it and it works.

@sschuberth Switching to hard links will solve the problem too, so either way I will be happy.

@dscho
MSysGit - the development behind Git for Windows member

IIRC the original reasoning for using symlinks was that most backup software is stupid and does not realize when files are hardlinked...

So I'd like to submit

4) hard-link receive-pack and upload-pack as required for the remote side of pushing/fetching, but remove all other hard-linked .exe files, keeping only git.exe.

@pleriche

If you're considering removing all the other symbolic/hard links, then very few files remain in the libexec/git-core folder and it begs the question: Why not just move the remaining files into /bin and do away with libexec/git-core? None of the installer options adds libexec/git-core to the path, so I would assume the executables in there are the less frequently used ones and moving them will probably not cause major disruption for most users.

With everything in /bin you could get away with just a single hard link (or even just a plain copy of git.exe) for git-receive-pack.exe. Git-upload-pack.exe is already a standalone executable, but since it currently resides in libexec/git-core it is not on the path.

@dscho
MSysGit - the development behind Git for Windows member

@pleriche usually I would agree with doing away the libexec/git-core/ directory. But that is a decision we would have to get accepted upstream, chances for which I estimate to be very close to 0. So while I agree with you, pragmatically we'll probably have to keep it.

@sschuberth sschuberth added a commit that referenced this issue Jun 3, 2012
@sschuberth sschuberth Installer: Do not try to create Symbolic Links anymore (GitHub issue #16
)

This manually reverts 7fdd58f partly as
some third-party software, like CopSSH, does not work correctly with
Symbolic Links. Making CopSSH work is more important than making Windows
Explorer display the correct directory size, which was the main original
reason to use Symbolic Links.

The confusion about the directory size when using Hardlinks can be
reduced in a future commit that only creates Hardlinks for
git-receive-pack and git-upload-pack, but no other built-ins.

Signed-off-by: Sebastian Schuberth <sschuberth@gmail.com>
da509c4
@sschuberth
MSysGit - the development behind Git for Windows member

Fixed in da509c4 by not trying to create Symbolic Links anymore, effectively falling back to Hardlinks. Dscho's suggestion number 4) may be additionally implemented in a future commit.

@sschuberth sschuberth closed this Jun 3, 2012
@dscho
MSysGit - the development behind Git for Windows member

@sschuberth I agree, it is not my itch at this time and I have bigger fish to fry. And that would be a new issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment