gem_package reports gem as installed when it is not.

[fasmat]

Description

I'm trying to install a gem with chef via gem_package but fail to do so on Ubuntu 22.04

Chef Version

22.5.923

Platform Version

Ubuntu 22.04

Replication Case

I have the following code in one of my recipes:

gem_package "bcrypt_pbkdf" do
    gem_binary "/usr/bin/gem"
    version "~>1.1"
    action :install
end

Client Output

The gem is installed according to chef:

    lin-vm:   * gem_package[bcrypt_pbkdf] action install[2022-05-11T17:21:27+00:00] INFO: Processing gem_package[bcrypt_pbkdf] action install (lin-vm::ruby_gems line 20)
    lin-vm: [2022-05-11T17:21:27+00:00] DEBUG: gem_package[bcrypt_pbkdf] is already installed - nothing to do
    lin-vm:  (up to date)

But checking for it shows that it is not there:

vagrant@test-vm:~$ gem list

*** LOCAL GEMS ***

abbrev (default: 0.1.0)
base64 (default: 0.1.0)
benchmark (default: 0.1.1)
bigdecimal (default: 3.0.0)
bundler (default: 2.2.22)
cgi (default: 0.2.0)
csv (default: 3.1.9)
date (default: 3.1.0)
dbm (default: 1.1.0)
debug (default: 0.1.0)
delegate (default: 0.2.0)
did_you_mean (default: 1.5.0)
digest (default: 3.0.0)
drb (default: 2.0.4)
english (default: 0.7.1)
erb (default: 2.2.0)
etc (default: 1.2.0)
fcntl (default: 1.0.0)
fiddle (default: 1.0.6)
fileutils (default: 1.5.0)
find (default: 0.1.0)
forwardable (default: 1.3.2)
gdbm (default: 2.1.0)
getoptlong (default: 0.1.1)
io-console (default: 0.5.7)
io-nonblock (default: 0.1.0)
io-wait (default: 0.1.0)
ipaddr (default: 1.2.2)
irb (default: 1.3.5)
json (default: 2.5.1)
logger (default: 1.4.3)
matrix (default: 0.3.1)
minitest (5.14.2)
mutex_m (default: 0.1.1)
net-ftp (default: 0.1.2)
net-http (default: 0.1.1)
net-imap (default: 0.1.1)
net-pop (default: 0.1.1)
net-protocol (default: 0.1.0)
net-smtp (default: 0.2.1)
net-telnet (0.1.1)
nkf (default: 0.1.0)
observer (default: 0.1.1)
open-uri (default: 0.1.0)
open3 (default: 0.1.1)
openssl (default: 3.0.0)
optparse (default: 0.1.0)
ostruct (default: 0.3.1)
pathname (default: 0.1.0)
power_assert (1.2.0)
pp (default: 0.1.0)
prettyprint (default: 0.1.0)
prime (default: 0.1.2)
pstore (default: 0.1.1)
psych (default: 3.3.0)
racc (default: 1.5.1)
rake (13.0.6, 13.0.3)
rbs (1.0.4)
rdoc (default: 6.3.1)
readline (default: 0.0.2)
readline-ext (default: 0.1.1)
reline (default: 0.2.5)
resolv (default: 0.2.0)
resolv-replace (default: 0.1.0)
rexml (3.2.5)
rinda (default: 0.1.0)
rss (0.2.9)
rubygems-update (3.3.5)
securerandom (default: 0.1.0)
set (default: 1.0.1)
shellwords (default: 0.1.0)
singleton (default: 0.1.1)
stringio (default: 3.0.0)
strscan (default: 3.0.0)
syslog (default: 0.1.0)
tempfile (default: 0.1.1)
test-unit (3.3.7)
time (default: 0.1.0)
timeout (default: 0.1.1)
tmpdir (default: 0.1.2)
tracer (default: 0.1.1)
tsort (default: 0.1.0)
typeprof (0.12.0)
un (default: 0.1.0)
uri (default: 0.10.1)
weakref (default: 0.1.1)
xmlrpc (0.3.2)
yaml (default: 0.1.1)
zlib (default: 1.1.0)

This issue appears with all gems that I try to install. Chef reports them to be already installed, but they are actually not there.

Any suggestions as to what could be the issue?

[fasmat]

Digging a bit further I found my gems to be installed here: /root/.chefdk/gem/ruby/3.0.0/gems with binaries linked to /root/.chefdk/gem/ruby/3.0.0/bin/

But I would expect them to be installed here: /var/lib/gems/3.0.0/gems/ and the binaries in /usr/local/bin/

[PrajaktaPurohit]

Could you let us know if you are trying to run chef-client from workstation when you see this error or is there a separate install of chef-client, if so what is the version of chef-client that is being run on the system?

Have you tried nuking the /root/.chefdk directory?

[fasmat]

Hi @PrajaktaPurohit

I'm running chef-client from my chef-workstation installation. The version of chef-client is the one shipped with workstation 22.5.923.

Clearing the /root/.chefdk/ directory unfortunately did not work. The next chef run installed the gems again into this directory instead of to /var/lib/gems.

[PrajaktaPurohit]

chef-17.10.0 is the version of chef that is installed with workstation 22.5.923.

[johnmccrae]

@fasmat Can you check both the Gem Path and the hard drive more broadly to see if that gem is lying around somewhere else? I'm curious why chef reports it installed but gem list isn't showing it. This is a new issue so we're a bit confused.

[fasmat]

@johnmccrae

information about used chef versions:

vagrant@test-vm:~$ chef --version
Chef Workstation version: 22.5.923
Test Kitchen version: 3.2.2
Cookstyle version: 7.32.1
Chef Infra Client version: 17.10.0
Chef InSpec version: 4.56.20
Chef CLI version: 5.6.1
Chef Habitat version: 1.6.420

my gem environment looks like this:

vagrant@test-vm:~$ sudo gem environment
RubyGems Environment:
  - RUBYGEMS VERSION: 3.3.5
  - RUBY VERSION: 3.0.2 (2021-07-07 patchlevel 107) [x86_64-linux-gnu]
  - INSTALLATION DIRECTORY: /var/lib/gems/3.0.0
  - USER INSTALLATION DIRECTORY: /root/.local/share/gem/ruby/3.0.0
  - RUBY EXECUTABLE: /usr/bin/ruby3.0
  - GIT EXECUTABLE: /usr/bin/git
  - EXECUTABLE DIRECTORY: /usr/local/bin
  - SPEC CACHE DIRECTORY: /root/.local/share/gem/specs
  - SYSTEM CONFIGURATION DIRECTORY: /etc
  - RUBYGEMS PLATFORMS:
     - ruby
     - x86_64-linux
  - GEM PATHS:
     - /var/lib/gems/3.0.0
     - /root/.local/share/gem/ruby/3.0.0
     - /usr/local/lib/ruby/gems/3.0.0
     - /usr/lib/ruby/gems/3.0.0
     - /usr/lib/x86_64-linux-gnu/ruby/gems/3.0.0
     - /usr/share/rubygems-integration/3.0.0
     - /usr/share/rubygems-integration/all
     - /usr/lib/x86_64-linux-gnu/rubygems-integration/3.0.0
  - GEM CONFIGURATION:
     - :update_sources => true
     - :verbose => true
     - :backtrace => false
     - :bulk_threshold => 1000
  - REMOTE SOURCES:
     - https://rubygems.org/
  - SHELL PATH:
     - /usr/local/sbin
     - /usr/local/bin
     - /usr/sbin
     - /usr/bin
     - /sbin
     - /bin
     - /snap/bin

gem_package will install gems to /root/.chefdk even if gem list already lists those gems. It seems to completely ignore GEM PATHS and only check this location.

I temporarily fixed my issue by replacing the following code in my recipe:

gem_package "bcrypt_pbkdf" do
    gem_binary "/usr/bin/gem"
    version "~>1.1"
    action :install
end

with

execute "installing gem bcrypt_pbkdf" do
    command "gem install bcrpt_pbkdf --version '~>1.1'"
    not_if "gem list bcrypt_pbkdf -i -v '~>1.1'"
end

which does exactly what I want to do: install a gem system wide for all users unless it is already installed.