Skip to content
This repository has been archived by the owner. It is now read-only.

[Enhancement] Replace Batch Redirector with Shims #372

Closed
ferventcoder opened this issue Nov 17, 2013 · 31 comments

Comments

@ferventcoder
Copy link
Contributor

commented Nov 17, 2013

Use a shim generator to create shim executable files instead of batch redirect files.

Shims are like symlinks except they work better. They are an executable that just runs another executable (or command). These shims are generated during the install and point only to the relative path of the executable. They are tiny at 11K.

@Redsandro

This comment has been minimized.

Copy link
Contributor

commented Nov 18, 2013

You have my interest. I have no idea what you are saying, but I don't like batch redirects.

What are shim files in this scope?

ferventcoder added a commit to ferventcoder/chocolatey that referenced this issue Nov 19, 2013
ferventcoder added a commit to ferventcoder/chocolatey that referenced this issue Nov 19, 2013
ferventcoder added a commit to ferventcoder/chocolatey that referenced this issue Nov 19, 2013
If it fails, it backs up to using the old method.
ferventcoder added a commit to ferventcoder/chocolatey that referenced this issue Nov 19, 2013
ferventcoder added a commit to ferventcoder/chocolatey that referenced this issue Jan 18, 2014
This allows shimgen to treat the entire arg as part of the argument,
even when it includes spaces. This is especially important with
paths that include spaces.
@ferventcoder

This comment has been minimized.

Copy link
Contributor Author

commented Jan 18, 2014

@Redsandro sorry for not catching you on this here - shims are exe files which simply act to pass through information.

@AaronFriel

This comment has been minimized.

Copy link

commented Feb 16, 2014

Is there any update on this?

@gep13

This comment has been minimized.

Copy link
Member

commented Feb 16, 2014

@ferventcoder will correct me if I am wrong, but I think this is in the latest alpha release.

Try

cup chocolatey -pre

Gary

@ferventcoder

This comment has been minimized.

Copy link
Contributor Author

commented Feb 16, 2014

Yeah, sorry I meant to put a note back in here when I released the alpha. What Gary said.

@mlt

This comment has been minimized.

Copy link

commented Mar 13, 2014

I noticed that portable version of SumatraPDF requires double quotation (actually triple as typed) to open files with spaces in path. Is it shimgen at fault? The real SumatraPDF.exe opens everything just fine with ordinary quotation for files with spaces.

@ferventcoder

This comment has been minimized.

Copy link
Contributor Author

commented Mar 13, 2014

@mlt This is interesting. It should not be the case. It should be the same amount of quotation as normal from command line calls to sumatra. What do you need for that from your shell and what are you seeing that you need when you call the shimgen version?

@mlt

This comment has been minimized.

Copy link

commented Mar 14, 2014

I feel like it removes one quotation level. For instance, I'd call real SumatraPDF "some file.pdf" however I have to SumatraPDF """some file.pdf""" with shim when running, e.g., from Far file manager in either case. It should be similar to regular cmd.exe . I guess "" is treated as a single quote when passed down. I noticed that something is not right when pdf association quit working in my Emacs Org mode setup with Chocolatey version of Sumatra.

@ferventcoder

This comment has been minimized.

Copy link
Contributor Author

commented Mar 14, 2014

Well that is definitely incorrect behavior. Let's get to the bottom of
this. Can you tell me what your output is when you type

where sumatrapdf

On Thursday, March 13, 2014, Mikhail Titov notifications@github.com wrote:

I feel like it removes one quotation level. For instance, I'd call real
SumatraPDF "some file.pdf" however I have to SumatraPDF """some file.pdf"""
with shim when running, e.g., from Far file manager in either case. It
should be similar to regular cmd.exe . I guess "" is treated as a single
quote when passed down. I noticed that something is not right when pdf
association quit working in my Emacs Org mode setup with Chocolatey version
of Sumatra.

Reply to this email directly or view it on GitHubhttps://github.com//issues/372#issuecomment-37603055
.

Rob
"Be passionate in all you do"

http://devlicio.us/blogs/rob_reynolds
http://ferventcoder.com
http://twitter.com/ferventcoder

@ferventcoder

This comment has been minimized.

Copy link
Contributor Author

commented Mar 14, 2014

My wording in the first comment didn't make a whole lot of sense. I think you answered correctly though.

@ferventcoder

This comment has been minimized.

Copy link
Contributor Author

commented Apr 3, 2014

@mlt I'm thinking you may have Emacs looking for SumatraPDF before SumatraPDF.exe, which could cause the need for double quotation. A simple where/which query where sumatrapdf could prove this out followed by SumatraPDF.exe "one set.pdf". Can you provide those things?

ferventcoder added a commit to ferventcoder/chocolatey that referenced this issue Apr 3, 2014
@mlt

This comment has been minimized.

Copy link

commented Apr 3, 2014

Nope, didn't fix it, if I tested it correctly. I did cup -pre chocolatey. It looks like updated things but gave some (partial?) warning that I ignored.

C:\Chocolatey>dir chocolateyinstall\tools\shimgen.exe
 Volume in drive C is Win7
 Volume Serial Number is 986F-B84D

 Directory of C:\Chocolatey\chocolateyinstall\tools

04/03/2014  11:41 AM            42,496 shimgen.exe
               1 File(s)         42,496 bytes
               0 Dir(s)  787,509,374,976 bytes free

C:\Chocolatey>where sumatrapdf
C:\Chocolatey\bin\SumatraPDF.exe

C:\Chocolatey>SumatraPDF.exe "C:\Users\user\Documents\work\Side Inlets\slides\detection\detection.pdf"

And it failed by opening 2 SumatraPDF windows one claiming it failed to open C:\Chocolatey\lib\sumatrapdf.commandline.2.4\tools\Inlets\slides\detection\detection.pdf and another one for C:\Users\user\Documents\work\Side

@ferventcoder

This comment has been minimized.

Copy link
Contributor Author

commented Apr 3, 2014

@mlt I see it stripping off quotes. Let me see what I can do.

@marcelklehr

This comment has been minimized.

Copy link

commented Apr 5, 2014

Could you release this as open source? I could really use something like this.

@mlt

This comment has been minimized.

Copy link

commented Apr 5, 2014

I think in 99% cases NTFS is used that supports links or whatever it is called on Windows. Shall we use that by default and let OS and FS handle all instead of a separate EXE and just fall back to shimgen if no link functionality is available?

Mikhail

On Apr 5, 2014, at 10:35 AM, Marcel Klehr notifications@github.com wrote:

Could you release this as open source? I could really use something like this.


Reply to this email directly or view it on GitHub.

@TomOne

This comment has been minimized.

Copy link

commented Apr 5, 2014

Shall we use that by default and let OS and FS handle all instead of a separate EXE and just fall back to shimgen if no link functionality is available?

Do you mean symbolic links? The support for symbolic links on Windows is very naive. If you make for example a symbolic link to an EXE and that EXE requires an external library, the symbolic link won’t resolve the relative path to it. This makes symbolic links on Windows unusable and useless in many cases, because many programs require external libraries. Unix-like systems don’t have this limitation. I think this is also the reason because shims are preferred here.

@mlt

This comment has been minimized.

Copy link

commented Apr 5, 2014

Oops didn't think of that a I used mostly with non-executables. Can we make it as an option if there is no such dependencies? E.g. create SumatraPDF.exe.gui containing "ln" would create a symlink if FS supports it, if the file is empty, then shimgen.

Mikhail

On Apr 5, 2014, at 1:02 PM, TomOne notifications@github.com wrote:

Shall we use that by default and let OS and FS handle all instead of a separate EXE and just fall back to shimgen if no link functionality is available?

Do you mean symbolic links? The support for symbolic links on Windows is very naive. If you make for example a symbolic link to an EXE and that EXE requires an external library, the symbolic link won’t resolve the relative path to it. This makes symbolic links on Windows unusable and useless in many cases, because many programs require external libraries. Unix-like systems don’t have this limitation. I think this is also the reason because shims are preferred here.


Reply to this email directly or view it on GitHub.

@jberezanski

This comment has been minimized.

Copy link
Contributor

commented Apr 5, 2014

Another thing to consider is that creating a symlink is a privileged operation on Windows, so it would not work for regular users.

@mlt

This comment has been minimized.

Copy link

commented Apr 7, 2014

Indeed, symlink creation requires privileges. However I just tried and a hardlink for SumatraPDF was created just fine with an unprivileged user and it works okay as it has no bundled DLLs/resources. I used Far File Manager 3 to create one with ALT+F6. I'm not sure what is an API call for that.

@ferventcoder

This comment has been minimized.

Copy link
Contributor Author

commented Apr 7, 2014

Right, but we need it to work in almost all cases. How will we be able to detect that it works without actually running the application in some controlled state and seeing if it errors due to an unfound library? That's why we're moving to a shim. Still provides really good functionality over batches with minimal issue.

@mlt

This comment has been minimized.

Copy link

commented Apr 7, 2014

I mean to give an option for packagers and keep the rest as is. For example, if we know that SumatraPDF.exe does not need anything else at all, I'd prefer to simply hardlink it if it is cumbersome to fix quotes unfolding. Placing hardlink into SumatraPDF.exe.gui and letting Chocolatey to respect that seems like a reasonable solutionworkaround.

@ferventcoder

This comment has been minimized.

Copy link
Contributor Author

commented Apr 7, 2014

@mlt we still support windows 2003 (and so does MS until 7/14/2015).

@jberezanski

This comment has been minimized.

Copy link
Contributor

commented Apr 7, 2014

@mlt right, I forgot about hard links. Coincidentally, Windows itself uses them for similar purpose, creating links in system32 (and other places) to exes and dlls in WinSxS (the unmanaged assembly cache).

@ferventcoder hard links are fully supported in xp/2003 (actually, even in 2000). The convenient command line tool mklink is only available in Vista+, however, so Chocolatey would need to ship its own simple binary exposing the CreateHardLink API.

Oh, and the logic should fall back to shims in cases when the hard link obviously cannot be used, e.g. $ChocolateyBinRoot is on a different drive than $ChocolateyInstall\bin, or link creation fails (e.g. both variables seem to point to the same drive, but one or more of the directories is actually a NTFS junction pointing to a different drive or even a file share).

@georgehemmings

This comment has been minimized.

Copy link

commented May 8, 2014

@ferventcoder have you made any progress on the stripping of quotes issue?

@ferventcoder

This comment has been minimized.

Copy link
Contributor Author

commented May 8, 2014

@georgehemmings This is done based on the call in the Win32 API, meaning Windows strips them out (http://stackoverflow.com/questions/9287812/backslash-and-quote-in-command-line-arguments & http://weblogs.asp.net/jgalloway/archive/2006/09/13/Command-Line-Confusion.aspx) - What I'll need to do is likely detect spaces and strap on quotes when found. Or just add quotes every time around arguments.

@mlt

This comment has been minimized.

Copy link

commented Jun 1, 2014

Does --gui for shimgen imply no waiting only? I feel like console window still appears. This is somewhat annoying when used often, for example with SumatraPDF and -inverse-search when used with emacsclientw. A window of the latter blinks when jumping from viewer to source.
While hardlinking "solves" the issue, I noticed that apparently SumatraPDF in such case creates sumatrapdfcache folder withint C:\Chocolatey\bin\. I suspect it used to create it in its own folder formerly. I wonder if that might be an issue with multiuser setup when it may lack permission to write.

@ferventcoder

This comment has been minimized.

Copy link
Contributor Author

commented Jun 2, 2014

@mlt thanks for pointing out the console window appears. I think I have fixed that (but not yet released).

@ferventcoder

This comment has been minimized.

Copy link
Contributor Author

commented Jun 2, 2014

@mlt I have the spaces issue fixed. Should be a call like normal to that once you see the next beta/release go out.

ferventcoder added a commit to ferventcoder/chocolatey that referenced this issue Jun 2, 2014
 - shimgen can handle spaces in arguments now
 - shimgen does not open a window for GUI based apps
ferventcoder added a commit to ferventcoder/chocolatey that referenced this issue Jun 2, 2014
* stable:
  (chocolatey-archiveGH-372) shimgen improvements
@ferventcoder

This comment has been minimized.

Copy link
Contributor Author

commented Jun 2, 2014

This should be ready to go.

@pfmoore

This comment has been minimized.

Copy link

commented Aug 4, 2014

I don't see the sources for shimgen anywhere. Are they available?

@TomOne

This comment has been minimized.

Copy link

commented Aug 4, 2014

Unfortunately not. Shimgen is closed-source proprietary software. Here’s the license text: https://github.com/chocolatey/chocolatey/blob/master/src/tools/shimgen.license.txt

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
10 participants
You can’t perform that action at this time.