New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Command 'gem list {{REGEXP}}' fails on Git for Windows on Windows XP #1669

Closed
MarkDBlackwell opened this Issue Jul 19, 2016 · 13 comments

Comments

Projects
None yet
6 participants
@MarkDBlackwell

MarkDBlackwell commented Jul 19, 2016

This issue is related to:

  • [X ] The command line gem

A prerequisite is to do gem install jsonb multi_json.

I'm having a problem. My current problem is:

The command gem list {{REGEXP}} works properly (i.e., it can limit the gems it finds) on my 64-bit Windows 7 system, using the Git Bash prompt of Git for Windows (version 2.8.1):

$ gem list json

*** LOCAL GEMS ***
json (1.8.2, default: 1.8.1)
jsonb (0.0.1)
multi_json (1.12.1)

$ gem list ^json$

*** LOCAL GEMS ***
json (1.8.2, default: 1.8.1)

$ gem list "^json$"

*** LOCAL GEMS ***
json (1.8.2, default: 1.8.1)

However, on my 32-bit Windows XP SP3 system (also using the Git Bash prompt of Git for Windows, version 2.8.1), I haven't found a way that works successfully:

$ gem list json

*** LOCAL GEMS ***
json (1.8.3, 1.8.2, default: 1.8.1)
jsonb (0.0.1)
multi_json (1.12.1)

$ gem list ^json$

*** LOCAL GEMS ***
json (1.8.3, 1.8.2, default: 1.8.1)
multi_json (1.12.1)

$ gem list "^json$"

*** LOCAL GEMS ***
json (1.8.3, 1.8.2, default: 1.8.1)
multi_json (1.12.1)

$ gem list '^json$'

*** LOCAL GEMS ***
json (1.8.3, 1.8.2, default: 1.8.1)
multi_json (1.12.1)

$ gem list \^json\$

*** LOCAL GEMS ***
json (1.8.3, 1.8.2, default: 1.8.1)
multi_json (1.12.1)

Yet, on cmd.exe, it can succeed:

>gem list "^json$"

*** LOCAL GEMS ***
json (1.8.3, 1.8.2, default: 1.8.1)

BTW, gem search {{REGEXP}} has the problem, too.

Here are my current environment details:

On Windows XP (where it fails):

$ gem env
RubyGems Environment:
  - RUBYGEMS VERSION: 2.6.6
  - RUBY VERSION: 2.2.5 (2016-04-26 patchlevel 319) [i386-mingw32]
  - INSTALLATION DIRECTORY: C:/progra/ruby/lib/ruby/gems/2.2.0
  - USER INSTALLATION DIRECTORY: C:/Documents and Settings/Mark/.gem/ruby/2.2.0
  - RUBY EXECUTABLE: C:/progra/ruby/bin/ruby.exe
  - EXECUTABLE DIRECTORY: C:/progra/ruby/bin
  - SPEC CACHE DIRECTORY: C:/Documents and Settings/Mark/.gem/specs
  - SYSTEM CONFIGURATION DIRECTORY: C:/Documents and Settings/All Users/Application Data
  - RUBYGEMS PLATFORMS:
    - ruby
    - x86-mingw32
  - GEM PATHS:
     - C:/progra/ruby/lib/ruby/gems/2.2.0
     - C:/Documents and Settings/Mark/.gem/ruby/2.2.0
  - GEM CONFIGURATION:
     - :update_sources => true
     - :verbose => true
     - :backtrace => false
     - :bulk_threshold => 1000
     - "gem" => "--conservative --no-rdoc --no-ri --run-tests"
     - "rdoc" => "--all --inline-source --line-numbers"
  - REMOTE SOURCES:
     - https://rubygems.org/
  - SHELL PATH:
     - C:\Documents and Settings\Mark\bin
     - C:\Program Files\Git\mingw32\bin
     - C:\Program Files\Git\usr\local\bin
     - C:\Program Files\Git\usr\bin
     - C:\Program Files\Git\usr\bin
     - C:\Program Files\Git\mingw32\bin
     - C:\Program Files\Git\usr\bin
     - C:\Documents and Settings\Mark\bin
     - C:\progra\ruby\bin
     - C:\WINDOWS\system32
     - C:\WINDOWS
     - C:\WINDOWS\System32\Wbem
     - C:\Program Files\Common Files\Adaptec Shared\System
     - C:\Program Files\Common Files\Adobe\AGL
     - C:\progra\gtk\bin
     - C:\Program Files\Intel\DMIX
     - C:\Program Files\Medialink\MWN-USB54G
     - C:\Program Files\Common Files\Teleca Shared
     - C:\Program Files\VDMSound
     - C:\Program Files\7-Zip
     - C:\Program Files\gs\gs9.16\bin
     - C:\progra\lilypond\current\usr\bin
     - C:\progra\nano\bin
     - C:\Program Files\Skype\Phone
     - C:\progra\Python32
     - C:\Documents and Settings\All Users\Application Data\Oracle\Java\javapath
     - C:\Program Files\Inno Setup 5
     - C:\Program Files\Git\usr\bin\vendor_perl
     - C:\Program Files\Git\usr\bin\core_perl
     - C:\Documents and Settings\Mark\.ec2\bin

On Windows 7 (where it succeeds):

$ gem env
RubyGems Environment:
  - RUBYGEMS VERSION: 2.6.6
  - RUBY VERSION: 2.2.5 (2016-04-26 patchlevel 319) [i386-mingw32]
  - INSTALLATION DIRECTORY: C:/progra/ruby/lib/ruby/gems/2.2.0
  - USER INSTALLATION DIRECTORY: C:/Users/Mark/.gem/ruby/2.2.0
  - RUBY EXECUTABLE: C:/progra/ruby/bin/ruby.exe
  - EXECUTABLE DIRECTORY: C:/progra/ruby/bin
  - SPEC CACHE DIRECTORY: C:/Users/Mark/.gem/specs
  - SYSTEM CONFIGURATION DIRECTORY: C:/ProgramData
  - RUBYGEMS PLATFORMS:
    - ruby
    - x86-mingw32
  - GEM PATHS:
     - C:/progra/ruby/lib/ruby/gems/2.2.0
     - C:/Users/Mark/.gem/ruby/2.2.0
  - GEM CONFIGURATION:
     - :update_sources => true
     - :verbose => true
     - :backtrace => false
     - :bulk_threshold => 1000
  - REMOTE SOURCES:
     - https://rubygems.org/
  - SHELL PATH:
     - C:\Users\Mark\bin
     - C:\Program Files\Git\mingw64\bin
     - C:\Program Files\Git\usr\local\bin
     - C:\Program Files\Git\usr\bin
     - C:\Program Files\Git\usr\bin
     - C:\Program Files\Git\mingw64\bin
     - C:\Program Files\Git\usr\bin
     - C:\Users\Mark\bin
     - C:\Windows\system32
     - C:\Windows
     - C:\Windows\System32\Wbem
     - C:\Windows\System32\WindowsPowerShell\v1.0
     - C:\Program Files\Common Files\Microsoft Shared\Windows Live
     - C:\Program Files (x86)\Common Files\Microsoft Shared\Windows Live
     - C:\Program Files (x86)\Windows Live\Shared
     - C:\Program Files (x86)\NVIDIA Corporation\PhysX\Common
     - C:\Program Files (x86)\AMD APP\bin\x86_64
     - C:\Program Files (x86)\Intel\iCLS Client
     - C:\Program Files\Intel\iCLS Client
     - C:\Program Files (x86)\ATI Technologies\ATI.ACE\Core-Static
     - C:\Program Files (x86)\Common Files\Adobe\AGL
     - C:\ProgramData\Oracle\Java\javapath
     - C:\progra\gtk\bin
     - C:\Program Files\Python33
     - C:\progra\ruby\bin
     - C:\Program Files (x86)\archduke\midi
     - C:\Program Files (x86)\gs\gs9.16\bin
     - C:\Program Files (x86)\gs\gs9.16\lib
     - C:\Program Files (x86)\Heroku\bin
     - C:\Program Files (x86)\LilyPond\current\usr\bin
     - C:\Program Files (x86)\Skype\Phone
     - C:\progra\nano\bin
     - C:\Program Files (x86)\Inno Setup 5
     - C:\Program Files\Git\usr\bin\vendor_perl
     - C:\Program Files\Git\usr\bin\core_perl

I will abide by the code of conduct.

@bronzdoc bronzdoc added the triage label Jul 19, 2016

@MarkDBlackwell

This comment has been minimized.

Show comment
Hide comment
@MarkDBlackwell

MarkDBlackwell Jul 19, 2016

On Windows XP, Git for Windows's Bash seems unlikely to be altering the command line, because this script:

$ cat > try-git-for-windows-parameter.bash
case "$1" in
"json")
  message='no anchors';;
"^json")
  message='start anchor';;
"json$")
  message='end anchor';;
"^json$")
  message='both anchors';;
*)
  message='something else';;
esac
echo $message

produces the output:

$ ./try-git-for-windows-parameter.bash ^json$
both anchors

MarkDBlackwell commented Jul 19, 2016

On Windows XP, Git for Windows's Bash seems unlikely to be altering the command line, because this script:

$ cat > try-git-for-windows-parameter.bash
case "$1" in
"json")
  message='no anchors';;
"^json")
  message='start anchor';;
"json$")
  message='end anchor';;
"^json$")
  message='both anchors';;
*)
  message='something else';;
esac
echo $message

produces the output:

$ ./try-git-for-windows-parameter.bash ^json$
both anchors
@raggi

This comment has been minimized.

Show comment
Hide comment
@MarkDBlackwell

This comment has been minimized.

Show comment
Hide comment
@MarkDBlackwell

MarkDBlackwell Aug 27, 2016

http://daviddeley.com/autohotkey/parameters/parameters.htm#WINCRULESCHANGE

@raggi Noted. :) However, on XP 32-bit Git for Windows version 2.9.2.0, plain Ruby picks up both ^ and $:

$ ruby -ve "p ARGV[0]" -- ^json$
ruby 2.2.5p319 (2016-04-26 revision 54774) [i386-mingw32]
"^json$"

and yet with gem list I get:

$ gem list ^json$
*** LOCAL GEMS ***
json (2.0.2, 1.8.3, default: 1.8.1)
multi_json (1.12.1)

MarkDBlackwell commented Aug 27, 2016

http://daviddeley.com/autohotkey/parameters/parameters.htm#WINCRULESCHANGE

@raggi Noted. :) However, on XP 32-bit Git for Windows version 2.9.2.0, plain Ruby picks up both ^ and $:

$ ruby -ve "p ARGV[0]" -- ^json$
ruby 2.2.5p319 (2016-04-26 revision 54774) [i386-mingw32]
"^json$"

and yet with gem list I get:

$ gem list ^json$
*** LOCAL GEMS ***
json (2.0.2, 1.8.3, default: 1.8.1)
multi_json (1.12.1)
@raggi

This comment has been minimized.

Show comment
Hide comment
@raggi

raggi Aug 27, 2016

Contributor

My note was for someone diagnosing the issue. I no longer have access to the rubygems github org, and am not volunteering to take the issue currently.

Contributor

raggi commented Aug 27, 2016

My note was for someone diagnosing the issue. I no longer have access to the rubygems github org, and am not volunteering to take the issue currently.

@RGSS3

This comment has been minimized.

Show comment
Hide comment
@RGSS3

RGSS3 Dec 3, 2016

you need

gem list "^^json$"

because Windows shell(cmd) needs a single "^" as escape character

RGSS3 commented Dec 3, 2016

you need

gem list "^^json$"

because Windows shell(cmd) needs a single "^" as escape character

@MarkDBlackwell

This comment has been minimized.

Show comment
Hide comment
@MarkDBlackwell

MarkDBlackwell Dec 3, 2016

@RGSS3 Following your suggestion, on Windows XP (on Git for Windows) just now I obtain the same result—in other words, the list includes 'multi_json':

$ gem list "^^json$"
*** LOCAL GEMS ***
json (2.0.2, 1.8.3, default: 1.8.1)
multi_json (1.12.1)

MarkDBlackwell commented Dec 3, 2016

@RGSS3 Following your suggestion, on Windows XP (on Git for Windows) just now I obtain the same result—in other words, the list includes 'multi_json':

$ gem list "^^json$"
*** LOCAL GEMS ***
json (2.0.2, 1.8.3, default: 1.8.1)
multi_json (1.12.1)
@segiddins

This comment has been minimized.

Show comment
Hide comment
@segiddins

segiddins Dec 3, 2016

Member

What happens if you invoke gem list from another process, without using a shell?

Member

segiddins commented Dec 3, 2016

What happens if you invoke gem list from another process, without using a shell?

@MarkDBlackwell

This comment has been minimized.

Show comment
Hide comment
@MarkDBlackwell

MarkDBlackwell Dec 3, 2016

@segiddins By "invoke gem list from another process", do you mean:

  1. Invoke some method on some Rubygems object? or
  2. Shell out, without typing directly into the shell?

Here's the latter (with otherwise the same environment)—it gives the same result:

$ ruby -e "s=%Q@gem list ^json\$@; p s; p exec s"
"gem list ^json$"
*** LOCAL GEMS ***
json (2.0.2, 1.8.3, default: 1.8.1)
multi_json (1.12.1)

$ ruby -e "s=%Q@gem list ^^json\$@; p s; p exec s"
"gem list ^^json$"
*** LOCAL GEMS ***
json (2.0.2, 1.8.3, default: 1.8.1)
multi_json (1.12.1)

MarkDBlackwell commented Dec 3, 2016

@segiddins By "invoke gem list from another process", do you mean:

  1. Invoke some method on some Rubygems object? or
  2. Shell out, without typing directly into the shell?

Here's the latter (with otherwise the same environment)—it gives the same result:

$ ruby -e "s=%Q@gem list ^json\$@; p s; p exec s"
"gem list ^json$"
*** LOCAL GEMS ***
json (2.0.2, 1.8.3, default: 1.8.1)
multi_json (1.12.1)

$ ruby -e "s=%Q@gem list ^^json\$@; p s; p exec s"
"gem list ^^json$"
*** LOCAL GEMS ***
json (2.0.2, 1.8.3, default: 1.8.1)
multi_json (1.12.1)
@MarkDBlackwell

This comment has been minimized.

Show comment
Hide comment
@MarkDBlackwell

MarkDBlackwell Dec 3, 2016

@segiddins Here (from the same environment) Ruby invokes the gem program directly, without using the shell. It obtains the same result:

$ ruby -e "s1='gem'; s2='list'; s3='^json\$'; p s1; p s2; p s3; p exec s1, s2, s3"
"gem"
"list"
"^json$"
*** LOCAL GEMS ***
json (2.0.2, 1.8.3, default: 1.8.1)
multi_json (1.12.1)

$ ruby -e "s1='gem'; s2='list'; s3='^^json\$'; p s1; p s2; p s3; p exec s1, s2, s3"
"gem"
"list"
"^^json$"
*** LOCAL GEMS ***
json (2.0.2, 1.8.3, default: 1.8.1)
multi_json (1.12.1)

MarkDBlackwell commented Dec 3, 2016

@segiddins Here (from the same environment) Ruby invokes the gem program directly, without using the shell. It obtains the same result:

$ ruby -e "s1='gem'; s2='list'; s3='^json\$'; p s1; p s2; p s3; p exec s1, s2, s3"
"gem"
"list"
"^json$"
*** LOCAL GEMS ***
json (2.0.2, 1.8.3, default: 1.8.1)
multi_json (1.12.1)

$ ruby -e "s1='gem'; s2='list'; s3='^^json\$'; p s1; p s2; p s3; p exec s1, s2, s3"
"gem"
"list"
"^^json$"
*** LOCAL GEMS ***
json (2.0.2, 1.8.3, default: 1.8.1)
multi_json (1.12.1)
@segiddins

This comment has been minimized.

Show comment
Hide comment
@segiddins

segiddins Dec 3, 2016

Member

Thanks for trying that... that's very weird, I cannot reproduce this on OS X

Member

segiddins commented Dec 3, 2016

Thanks for trying that... that's very weird, I cannot reproduce this on OS X

@bronzdoc

This comment has been minimized.

Show comment
Hide comment
@bronzdoc

bronzdoc Jul 18, 2018

Member

@MarkDBlackwell is this still and issue for you?

Member

bronzdoc commented Jul 18, 2018

@MarkDBlackwell is this still and issue for you?

@MarkDBlackwell

This comment has been minimized.

Show comment
Hide comment
@MarkDBlackwell

MarkDBlackwell Jul 27, 2018

@bronzdoc, I haven't been bothered by the issue, all this time, partly because I haven't used Ruby on Rails much, since then.

gem list --all '^rails$' was annoying, because it failed (per the issue) to exclude related gems.

Amazingly, gem list --remote '^rails*' | wc -l counts 1,273 gems—even without adding --all.

Windows XP usually is considered dead. But who has an old 32-bit computer running Windows 7? On it, the issue might occur. (On my Windows 7 machine, the issue doesn't occur, but it's 64-bit.)

Otherwise, if anyone is running OS X 10.6 Snow Leopard or earlier, they possibly have 32-bit hardware. If so, then installing Windows XP SP3 in a virtual machine, and RubyInstallerForWindows and Git Bash, would be a way to try to experience the issue.

That would be sufficient IMO to rule out interaction with standard Windows XP SP3.

Although cloud servers can run some 32-bit OSs in VMs, precious few actually are 32-bit hardware.

I know where I can borrow a 32-bit Windows XP SP3 computer. This Sunday, I'll see if it experiences the issue.

We know that if one user speaks up—regarding a product awkwardness issue—then many others are noticing the same thing, and silently criticizing. ;-)

MarkDBlackwell commented Jul 27, 2018

@bronzdoc, I haven't been bothered by the issue, all this time, partly because I haven't used Ruby on Rails much, since then.

gem list --all '^rails$' was annoying, because it failed (per the issue) to exclude related gems.

Amazingly, gem list --remote '^rails*' | wc -l counts 1,273 gems—even without adding --all.

Windows XP usually is considered dead. But who has an old 32-bit computer running Windows 7? On it, the issue might occur. (On my Windows 7 machine, the issue doesn't occur, but it's 64-bit.)

Otherwise, if anyone is running OS X 10.6 Snow Leopard or earlier, they possibly have 32-bit hardware. If so, then installing Windows XP SP3 in a virtual machine, and RubyInstallerForWindows and Git Bash, would be a way to try to experience the issue.

That would be sufficient IMO to rule out interaction with standard Windows XP SP3.

Although cloud servers can run some 32-bit OSs in VMs, precious few actually are 32-bit hardware.

I know where I can borrow a 32-bit Windows XP SP3 computer. This Sunday, I'll see if it experiences the issue.

We know that if one user speaks up—regarding a product awkwardness issue—then many others are noticing the same thing, and silently criticizing. ;-)

@MarkDBlackwell

This comment has been minimized.

Show comment
Hide comment
@MarkDBlackwell

MarkDBlackwell Jul 31, 2018

TL;DR The latest release of Rubygems on a "fresh" Windows XP machine didn't experience the issue.

@bronzdoc @segiddins @RGSS3 From Git for Windows and RubyInstaller, I installed the last binary releases which are functional on 32-bit Windows XP.

To test the issue, I followed these steps:

  1. Obtain a 32-bit machine running Windows XP Professional, Service Pack 3 (SP3).

  2. Install RubyInstaller.org's binary for Ruby version 2.0.0-p648 (the last binary version of RubyInstaller to support Windows XP—which version I identified by trial-and-error), by downloading and executing the asset, rubyinstaller-2.0.0-p648.exe from column RubyInstallers on the RubyInstaller archives webpage:

    • Unselect "Install Tcl/Tk support";

    • Select "Add Ruby executables to your PATH"; and

    • Unselect "Associate .rb and .rbw files with this Ruby installation".

  3. Install Git for Windows, by downloading and executing the asset, Git-2.10.0-32-bit.exe containing version 2.10.0. (This is the last binary version to support Windows XP, per the Git for Windows FAQ):

    • Unselect all components on the page, "Which Components should be installed?";

    • Select "Use Git from Git Bash only";

    • Select "Checkout as-is, commit as-is";

    • Select "Use Windows' default console window";

    • Unselect "Enable file system caching"; and

    • Unselect "Enable Git Credential Manager".

Git for Windows installs—into the user's Start menu—two entry points: "Git Bash" and "Git CMD".

  1. Start Git for Windows, using the "Git Bash" entry point, and do:
$ ruby --version
ruby 2.0.0p648 (2015-12-16) [i386-mingw32]
  1. Download the latest version of the gem, rubygems-update (for me, this was version 2.7.7). Then do:
$ gem install --local --no-document rubygems-update-2.7.7.gem
$ update_rubygems --version=2.7.7
  1. Install two gems which have characters either before or after 'json' in their names:
$ gem install --no-document multi_json -v 1.12.1
$ gem install --no-document jsonb -v 0.0.1

Every gem list command (below) emitted this error message:

YAML safe loading is not available.
Please upgrade psych to a version that supports safe loading (>= 2.0).
  1. Still using the "Git Bash" entry point of Git for Windows, do:
$ gem list json

*** LOCAL GEMS ***
json (default: 1.7.7)
jsonb (0.0.1)
multi_json (1.12.1)

$ gem list ^json$

*** LOCAL GEMS ***
json (default: 1.7.7)

So, the issue does not appear.

  1. Start Git for Windows, using its "Git CMD" entry point. Double-quote the argument this time, since it contains a caret ^:
>gem list "^json$"

*** LOCAL GEMS ***
json (default: 1.7.7)

Again, the issue does not appear.

Therefore, on a fresh Windows XP machine with up-to-date software, the results look fine. The test fails to reproduce the issue.

MarkDBlackwell commented Jul 31, 2018

TL;DR The latest release of Rubygems on a "fresh" Windows XP machine didn't experience the issue.

@bronzdoc @segiddins @RGSS3 From Git for Windows and RubyInstaller, I installed the last binary releases which are functional on 32-bit Windows XP.

To test the issue, I followed these steps:

  1. Obtain a 32-bit machine running Windows XP Professional, Service Pack 3 (SP3).

  2. Install RubyInstaller.org's binary for Ruby version 2.0.0-p648 (the last binary version of RubyInstaller to support Windows XP—which version I identified by trial-and-error), by downloading and executing the asset, rubyinstaller-2.0.0-p648.exe from column RubyInstallers on the RubyInstaller archives webpage:

    • Unselect "Install Tcl/Tk support";

    • Select "Add Ruby executables to your PATH"; and

    • Unselect "Associate .rb and .rbw files with this Ruby installation".

  3. Install Git for Windows, by downloading and executing the asset, Git-2.10.0-32-bit.exe containing version 2.10.0. (This is the last binary version to support Windows XP, per the Git for Windows FAQ):

    • Unselect all components on the page, "Which Components should be installed?";

    • Select "Use Git from Git Bash only";

    • Select "Checkout as-is, commit as-is";

    • Select "Use Windows' default console window";

    • Unselect "Enable file system caching"; and

    • Unselect "Enable Git Credential Manager".

Git for Windows installs—into the user's Start menu—two entry points: "Git Bash" and "Git CMD".

  1. Start Git for Windows, using the "Git Bash" entry point, and do:
$ ruby --version
ruby 2.0.0p648 (2015-12-16) [i386-mingw32]
  1. Download the latest version of the gem, rubygems-update (for me, this was version 2.7.7). Then do:
$ gem install --local --no-document rubygems-update-2.7.7.gem
$ update_rubygems --version=2.7.7
  1. Install two gems which have characters either before or after 'json' in their names:
$ gem install --no-document multi_json -v 1.12.1
$ gem install --no-document jsonb -v 0.0.1

Every gem list command (below) emitted this error message:

YAML safe loading is not available.
Please upgrade psych to a version that supports safe loading (>= 2.0).
  1. Still using the "Git Bash" entry point of Git for Windows, do:
$ gem list json

*** LOCAL GEMS ***
json (default: 1.7.7)
jsonb (0.0.1)
multi_json (1.12.1)

$ gem list ^json$

*** LOCAL GEMS ***
json (default: 1.7.7)

So, the issue does not appear.

  1. Start Git for Windows, using its "Git CMD" entry point. Double-quote the argument this time, since it contains a caret ^:
>gem list "^json$"

*** LOCAL GEMS ***
json (default: 1.7.7)

Again, the issue does not appear.

Therefore, on a fresh Windows XP machine with up-to-date software, the results look fine. The test fails to reproduce the issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment