Provide a working out of the box instaweb

[ciaranj]

Related to issue #9 and in particular comment #9 (comment) . It would be great if msysgit supported (out of the box) an implementation of the instaweb command that worked without external (where external is, external to the provided git installation) dependencies.

There are it seems various instructions on the web as to how one might set about achieving this, for example ( http://asimilatorul.com/index.php/2009/10/12/git-instaweb-using-mongoose-and-msysgit/ ) however looking at the documentation for instaweb it looks as though we should just provide a msys-happy install of lighthttpd, for example: https://github.com/mkottman/lighttpd-1.5-mingw looks like a potential candidate for inclusion.

I could probably take a look at this, but it may turn out to be beyond my means (hopefully not though.)

[ciaranj]

lighttpd would appear to be a royal PiTA to get compiled (requires cmake which is a 10 meg set of binaries I figure won't be popular, unless I'm missing something) will look at mongoose which seems to have less dependencies, but would change msysgit's git's instaweb to default to use mongoose rather than lighttpd ?

[hvoigt]

Yes if you can get it working with mongoose more easily I do not see a downside. You probably want to make it the default configuration on msysgit using /etc/gitconfig if you do so though.

[ciaranj]

Ah, ok if this is possible then yes that is a much better solution, I'm currently in some sort of file-path hell, I can fire up instaweb running mongoose which I compiled successfully (and trivially) from within the msysgit environment, but I can't seem to make the file munging play nicely... I'm fairly close, but not without changing the way some things are passed to mongoose (likely *nix incompatible) ... also remembering how bad I am at perl :(

[ciaranj]

Sweet, thanks to some help from @harlandpaul (some abs_path calls seem to be required) I've got a working git instaweb using mongoose with no special magic.

So I can do it ;) I'm just not sure how to package things successfully! I need to add the CGI perl module (as per Issue #9) think I can figure this one out, but I'm stuck on 3 other issues :

1. I had to copy libiconv-2.dll to the git-core folder (there are various incidents on google suggesting this 'fix', but I'm hopeful for a better path-based solution.. not sure yet)
2. I need to pull in the source code to mongoose into msysgit/msysgit which is a mercurial repository. What is the preference for doing this, should I just take a snapshot and document the current tagname/commit hash and commit that, or should I commit the whole repo yadda, yadda.
3. I need to make a small change to Git's instaweb scripts, so presumably I also need to change msysgit/git, if I do do these changes do I just submit two pull requests for the separate projects, wait for the first to be consumed, before updating the second's request with the relevant submodule change? (Sorry if this is dumb)

[hvoigt]

Some answers:

1. Yes there should be another solution
2. Just write the release.sh script that downloads, compiles and installs the binaries into msysgit and commit the script (like e.g. /src/openssl/)
3. Yes that should be submitted as a seperate pull request on msysgit/git possibly referencing this issue.

[dscho]

As for Hg repository: we now have a working git remote-hg, so in theory you can call "git clone hg::http://wherever.com/hg/mongoose". I say "in theory" because it requires a working Python, something I did not manage to build for MinGW (except cross-compiling from Linux).

You can see that things are working beautifully here: http://github.com/dscho/hg (updated via a cron job on a Linux machine I have access to).

As to 3) yes, two pull requests are appropriate, the msysgit one including an update to the /git submodule pointer (which might need to be updated if the pull request for the git repository needs to be adjusted).

[ciaranj]

@dscho Hi, sorry, I've done a stab at following the existing patterns that I can see for importing 3rd party dependencies in ( ciaranj/msysgit@e32bca6 ) but i'm not really quite sure how to proceed. ...

1. Other dependencies appear to get checked in at the end of their release.sh script, is this the correct way to do it in this case as well?
2. (possibly related to 1) I presume if 1 doesn't happen then I need to add a call to mongoose/release.sh to the WinGit/release.sh file, otherwise I just rely on the WinGit/copy_files.sh copying this (mongoose.exe) across. Which is the
   preferred approach? (presumably this depends on whether you want local 'instaweb' support working for msysgit's Git, as well as for the produced Git artefact?)

[dscho]

1. yes!
2. you need to adjust copy-files, too.

Great work!

[ciaranj]

So, Progress (but I'm afraid that I need help to even get close to finishing this off);

So, starting with a positive :) An example (no smoke or mirrors) release can be found here: https://docs.google.com/folder/d/0B44eAAsUHKHPMWJ1TnVrZE1NaEE/edit (you will still need to use git instaweb --httpd mongoose to launch it at the moment)

So there's a few steps outstanding;

1. This pull request on msysgit/msysgit needs to be reviewed, refined and accepted : Added CGI.pm (3.59) to the default perl install msysgit#15 (I think this is required for Issue add support for gitweb integration with gerrit without apache #9 too)
2. I need help to review the work in https://github.com/ciaranj/msysgit/tree/add_mongoose to determine if I'm doing things 'the right way'
3. I need help to review the work in https://github.com/ciaranj/git/commits/instaweb_msysgit_changes (one of these commits may need to go upstream as it updates the mongoose config file syntax to the latest version, alternatively we could figure out (and package) whatever older version was supported) and the other commits are basically doing stuff to handle msysgit path mangling, so hopefully someone better than me can figure out a better/neater solution now I've a working proof-of-concept :)

I've create a branch https://github.com/ciaranj/msysgit/tree/issue_11 where the mongoose + cgi branches have been merged and I've re-enabled instaweb (Which is what I used to generate the binary above)

Other than the review process required, there are some functional issues remaining;

1. The forking/process identifier handling appears broken as re-launching mongoose doesn't work
2. The snapshot feature of gitweb appears not to do anything

But surprisingly thats all I've noticed :)

[dscho]

Probably forking Mongoose requires a properly working fork()? Windows does not have that.

Maybe 2) is exactly the same issue?

[dscho]

IOW I think that this code is failing: http://code.google.com/p/mongoose/source/browse/mongoose.c#1178

I could imagine that it tries to execute ".perl" scripts directly, which works on Unix-like systems like Linux because those systems interpret the first line of scripts when they begin with a so-called 'hash-bang', i.e. "#!/usr/bin/perl".

If that is indeed the case, we might need to interpret that line ourselves. Since Mongoose is BSD and Git is GPLv2, we cannot simply copy the relevant code from our compat/mingw.c, unfortunately (Hannes Sixt wrote the useful parse_interpreter() function which would come in handy here).

[ciaranj]

Help! (please) :) I've thrashed about for a few hours on this, but I'm struggling. I can get mongoose launching as is (not sure if this is related to the above 2 comments or not) when running as part of WinGit, and I can get it launching happily as part of MsysGit if I copy libiconv-2.dll from /mingw/bin to /bin ... but if I do this I can no longer switch msysgit branches as the git that executes has 'found' libiconv-2.dll in the /bin folder, and locked it so the checkout can't complete :(

So :) I figured I had to somehow get /mingw/bin on the path that belongs to the environment that git.exe is being executed in by gitweb.exe (when doing instaweb from within msysgit)

Using process explorer (the old sysinternals tool) I can see that I can succesfully pass the path with /mingw/bin pre-pended to the front into mongoose which then happily passes that onto the CGI script (gitweb.cgi) .. which then appears to re-execute perl (succesfully passing in the path) which then executes git.exe .. however at this point the PATH is set to '' (blank)

I know nothing about perl :( but I figure it is something to do with the lines that look like:

open my $fd, "-|", git_cmd(), "cat-file", "blob", $hash

Where git_cmd() returns the path to the git.exe and a git directory location. I guess my question is, how do get 'PATH' so it is set correctly when 'git_cmd()' is executed? .. or am I (still) barking up the wrong tree !

[dscho]

@ciaranj thank you for the thorough analysis! However, I am not quite sure where exactly PATH is ''. Is it already when the .cgi is run, or when the .cgi calls cat-file, or when Git starts up? I do not see any PATH mangling in gitweb.cgi here... so I suspect that something fishy is going on either in our compat/mingw.c or in Mongoose.

[ciaranj]

So, when I use process explorer ( http://technet.microsoft.com/en-us/sysinternals/bb896653.aspx ) and select the 'git.exe' process that is eventually launched by cgiweb and I inspect the environment through the tool I see that 'PATH' is set to blank (but other environemntals such as GITWEB_CONF are set!)

There is some path mangling (I think) in git-instaweb.sh where it executes '. git-sh-setup' ... but the git.exe process is '2 levels' in/down from that script and in-between scripts seem to have the correct path .. I imagine this will turn out to be something simple, but perl/cgi is not something I have much (any!) familiarity with sorry :(