[Enhancement] Replace Batch Redirector with Shims

[ferventcoder]

Use a shim generator to create shim executable files instead of batch redirect files.

Shims are like symlinks except they work better. They are an executable that just runs another executable (or command). These shims are generated during the install and point only to the relative path of the executable. They are tiny at 11K.

[Redsandro]

You have my interest. I have no idea what you are saying, but I don't like batch redirects.

What are shim files in this scope?

[ferventcoder]

@Redsandro sorry for not catching you on this here - shims are exe files which simply act to pass through information.

[AaronFriel]

Is there any update on this?

[gep13]

@ferventcoder will correct me if I am wrong, but I think this is in the latest alpha release.

Try

cup chocolatey -pre

Gary

[ferventcoder]

Yeah, sorry I meant to put a note back in here when I released the alpha. What Gary said.

[mlt]

I noticed that portable version of SumatraPDF requires double quotation (actually triple as typed) to open files with spaces in path. Is it shimgen at fault? The real SumatraPDF.exe opens everything just fine with ordinary quotation for files with spaces.

[ferventcoder]

@mlt This is interesting. It should not be the case. It should be the same amount of quotation as normal from command line calls to sumatra. What do you need for that from your shell and what are you seeing that you need when you call the shimgen version?

[mlt]

I feel like it removes one quotation level. For instance, I'd call real SumatraPDF "some file.pdf" however I have to SumatraPDF """some file.pdf""" with shim when running, e.g., from Far file manager in either case. It should be similar to regular cmd.exe . I guess "" is treated as a single quote when passed down. I noticed that something is not right when pdf association quit working in my Emacs Org mode setup with Chocolatey version of Sumatra.

[ferventcoder]

Well that is definitely incorrect behavior. Let's get to the bottom of
this. Can you tell me what your output is when you type

where sumatrapdf

On Thursday, March 13, 2014, Mikhail Titov notifications@github.com wrote:

I feel like it removes one quotation level. For instance, I'd call real
SumatraPDF "some file.pdf" however I have to SumatraPDF """some file.pdf"""
with shim when running, e.g., from Far file manager in either case. It
should be similar to regular cmd.exe . I guess "" is treated as a single
quote when passed down. I noticed that something is not right when pdf
association quit working in my Emacs Org mode setup with Chocolatey version
of Sumatra.

Reply to this email directly or view it on GitHubhttps://github.com//issues/372#issuecomment-37603055
.

Rob
"Be passionate in all you do"

http://devlicio.us/blogs/rob_reynolds
http://ferventcoder.com
http://twitter.com/ferventcoder

[ferventcoder]

My wording in the first comment didn't make a whole lot of sense. I think you answered correctly though.

[ferventcoder]

@mlt I'm thinking you may have Emacs looking for SumatraPDF before SumatraPDF.exe, which could cause the need for double quotation. A simple where/which query where sumatrapdf could prove this out followed by SumatraPDF.exe "one set.pdf". Can you provide those things?

[mlt]

Nope, didn't fix it, if I tested it correctly. I did cup -pre chocolatey. It looks like updated things but gave some (partial?) warning that I ignored.

C:\Chocolatey>dir chocolateyinstall\tools\shimgen.exe
 Volume in drive C is Win7
 Volume Serial Number is 986F-B84D

 Directory of C:\Chocolatey\chocolateyinstall\tools

04/03/2014  11:41 AM            42,496 shimgen.exe
               1 File(s)         42,496 bytes
               0 Dir(s)  787,509,374,976 bytes free

C:\Chocolatey>where sumatrapdf
C:\Chocolatey\bin\SumatraPDF.exe

C:\Chocolatey>SumatraPDF.exe "C:\Users\user\Documents\work\Side Inlets\slides\detection\detection.pdf"

And it failed by opening 2 SumatraPDF windows one claiming it failed to open C:\Chocolatey\lib\sumatrapdf.commandline.2.4\tools\Inlets\slides\detection\detection.pdf and another one for C:\Users\user\Documents\work\Side

[ferventcoder]

@mlt I see it stripping off quotes. Let me see what I can do.

[marcelklehr]

Could you release this as open source? I could really use something like this.

[mlt]

I think in 99% cases NTFS is used that supports links or whatever it is called on Windows. Shall we use that by default and let OS and FS handle all instead of a separate EXE and just fall back to shimgen if no link functionality is available?

Mikhail

On Apr 5, 2014, at 10:35 AM, Marcel Klehr notifications@github.com wrote:

Could you release this as open source? I could really use something like this.

—
Reply to this email directly or view it on GitHub.

[TomOne]

Shall we use that by default and let OS and FS handle all instead of a separate EXE and just fall back to shimgen if no link functionality is available?

Do you mean symbolic links? The support for symbolic links on Windows is very naive. If you make for example a symbolic link to an EXE and that EXE requires an external library, the symbolic link won’t resolve the relative path to it. This makes symbolic links on Windows unusable and useless in many cases, because many programs require external libraries. Unix-like systems don’t have this limitation. I think this is also the reason because shims are preferred here.

[mlt]

Oops didn't think of that a I used mostly with non-executables. Can we make it as an option if there is no such dependencies? E.g. create SumatraPDF.exe.gui containing "ln" would create a symlink if FS supports it, if the file is empty, then shimgen.

Mikhail

On Apr 5, 2014, at 1:02 PM, TomOne notifications@github.com wrote:

Shall we use that by default and let OS and FS handle all instead of a separate EXE and just fall back to shimgen if no link functionality is available?

Do you mean symbolic links? The support for symbolic links on Windows is very naive. If you make for example a symbolic link to an EXE and that EXE requires an external library, the symbolic link won’t resolve the relative path to it. This makes symbolic links on Windows unusable and useless in many cases, because many programs require external libraries. Unix-like systems don’t have this limitation. I think this is also the reason because shims are preferred here.

—
Reply to this email directly or view it on GitHub.

[jberezanski]

Another thing to consider is that creating a symlink is a privileged operation on Windows, so it would not work for regular users.

[mlt]

Indeed, symlink creation requires privileges. However I just tried and a hardlink for SumatraPDF was created just fine with an unprivileged user and it works okay as it has no bundled DLLs/resources. I used Far File Manager 3 to create one with ALT+F6. I'm not sure what is an API call for that.

[ferventcoder]

Right, but we need it to work in almost all cases. How will we be able to detect that it works without actually running the application in some controlled state and seeing if it errors due to an unfound library? That's why we're moving to a shim. Still provides really good functionality over batches with minimal issue.

[mlt]

I mean to give an option for packagers and keep the rest as is. For example, if we know that SumatraPDF.exe does not need anything else at all, I'd prefer to simply hardlink it if it is cumbersome to fix quotes unfolding. Placing hardlink into SumatraPDF.exe.gui and letting Chocolatey to respect that seems like a reasonable solutionworkaround.

[ferventcoder]

@mlt we still support windows 2003 (and so does MS until 7/14/2015).

[jberezanski]

@mlt right, I forgot about hard links. Coincidentally, Windows itself uses them for similar purpose, creating links in system32 (and other places) to exes and dlls in WinSxS (the unmanaged assembly cache).

@ferventcoder hard links are fully supported in xp/2003 (actually, even in 2000). The convenient command line tool mklink is only available in Vista+, however, so Chocolatey would need to ship its own simple binary exposing the CreateHardLink API.

Oh, and the logic should fall back to shims in cases when the hard link obviously cannot be used, e.g. $ChocolateyBinRoot is on a different drive than $ChocolateyInstall\bin, or link creation fails (e.g. both variables seem to point to the same drive, but one or more of the directories is actually a NTFS junction pointing to a different drive or even a file share).

[georgehemmings]

@ferventcoder have you made any progress on the stripping of quotes issue?

[ferventcoder]

@georgehemmings This is done based on the call in the Win32 API, meaning Windows strips them out (http://stackoverflow.com/questions/9287812/backslash-and-quote-in-command-line-arguments & http://weblogs.asp.net/jgalloway/archive/2006/09/13/Command-Line-Confusion.aspx) - What I'll need to do is likely detect spaces and strap on quotes when found. Or just add quotes every time around arguments.

[mlt]

Does --gui for shimgen imply no waiting only? I feel like console window still appears. This is somewhat annoying when used often, for example with SumatraPDF and -inverse-search when used with emacsclientw. A window of the latter blinks when jumping from viewer to source.
While hardlinking "solves" the issue, I noticed that apparently SumatraPDF in such case creates sumatrapdfcache folder withint C:\Chocolatey\bin\. I suspect it used to create it in its own folder formerly. I wonder if that might be an issue with multiuser setup when it may lack permission to write.

[ferventcoder]

@mlt thanks for pointing out the console window appears. I think I have fixed that (but not yet released).

[ferventcoder]

@mlt I have the spaces issue fixed. Should be a call like normal to that once you see the next beta/release go out.

[ferventcoder]

This should be ready to go.

[pfmoore]

I don't see the sources for shimgen anywhere. Are they available?

[TomOne]

Unfortunately not. Shimgen is closed-source proprietary software. Here’s the license text: https://github.com/chocolatey/chocolatey/blob/master/src/tools/shimgen.license.txt