install fails on PE 3.2.2 due to Puppet Module Tool Bug

[brettswift]

On a clean install of PE 3.2.3, RHEL 6.4, I get an error:

error snippet:

Error: No such file or directory - /etc/puppetlabs/puppet/modules/r10k/spec/fixtures/modules/r10k/files

Full stack:

puppet module install zack/r10k --debug --version 1.0.2

Notice: Preparing to install into /etc/puppetlabs/puppet/modules ...
Notice: Downloading from https://forgeapi.puppetlabs.com ...
Debug: HTTP GET https://forgeapi.puppetlabs.com/v3/releases?module=zack-r10k
Debug: HTTP GET https://forgeapi.puppetlabs.com/v3/releases?module=puppetlabs-stdlib
Debug: HTTP GET https://forgeapi.puppetlabs.com/v3/releases?module=puppetlabs-stdlib&limit=20&offset=20
Notice: Found at least one version of puppetlabs-stdlib compatible with PE (3.2.3);
Notice: Skipping versions which don't express PE compatibility. To install
the most recent version of the module regardless of compatibility
with PE, use the '--ignore-requirements' flag.
Debug: HTTP GET https://forgeapi.puppetlabs.com/v3/releases?module=puppetlabs-ruby
Debug: HTTP GET https://forgeapi.puppetlabs.com/v3/releases?module=puppetlabs-gcc
Debug: HTTP GET https://forgeapi.puppetlabs.com/v3/releases?module=puppetlabs-pe_gem
Debug: HTTP GET https://forgeapi.puppetlabs.com/v3/releases?module=mhuffnagle-make
Debug: HTTP GET https://forgeapi.puppetlabs.com/v3/releases?module=puppetlabs-inifile
Notice: Found at least one version of puppetlabs-inifile compatible with PE (3.2.3);
Notice: Skipping versions which don't express PE compatibility. To install
the most recent version of the module regardless of compatibility
with PE, use the '--ignore-requirements' flag.
Debug: HTTP GET https://forgeapi.puppetlabs.com/v3/releases?module=puppetlabs-vcsrepo
Debug: HTTP GET https://forgeapi.puppetlabs.com/v3/releases?module=puppetlabs-git
Debug: HTTP GET https://forgeapi.puppetlabs.com/v3/releases?module=gentoo-portage
Debug: HTTP GET https://forgeapi.puppetlabs.com/v3/releases?module=puppetlabs-concat
Notice: Found at least one version of puppetlabs-concat compatible with PE (3.2.3);
Notice: Skipping versions which don't express PE compatibility. To install
the most recent version of the module regardless of compatibility
with PE, use the '--ignore-requirements' flag.
Debug: HTTP GET https://forgeapi.puppetlabs.com/v3/releases?module=ripienaar-concat
Info: Resolving dependencies ...
Info: Preparing to install ...
Debug: HTTP GET https://forgeapi.puppetlabs.com/v3/files/zack-r10k-1.0.2.tar.gz
Debug: Executing 'tar xzf /var/opt/lib/pe-puppet/puppet-module/cache/zack-r10k20140513-15637-hrn7bv --no-same-owner -C /var/opt/lib/pe-puppet/puppet-module/cache/tmp-unpacker20140513-15637-v98unf'
Debug: Executing 'find /var/opt/lib/pe-puppet/puppet-module/cache/tmp-unpacker20140513-15637-v98unf -type d -exec chmod 755 {} +'
Debug: Executing 'find /var/opt/lib/pe-puppet/puppet-module/cache/tmp-unpacker20140513-15637-v98unf -type f -exec chmod a-wst {} +'
Debug: Executing 'chown -R 0:0 /var/opt/lib/pe-puppet/puppet-module/cache/tmp-unpacker20140513-15637-v98unf'
Debug: HTTP GET https://forgeapi.puppetlabs.com/v3/files/puppetlabs-ruby-0.1.0.tar.gz
Debug: Executing 'tar xzf /var/opt/lib/pe-puppet/puppet-module/cache/puppetlabs-ruby20140513-15637-m9b7pl --no-same-owner -C /var/opt/lib/pe-puppet/puppet-module/cache/tmp-unpacker20140513-15637-ddp3g4'
Debug: Executing 'find /var/opt/lib/pe-puppet/puppet-module/cache/tmp-unpacker20140513-15637-ddp3g4 -type d -exec chmod 755 {} +'
Debug: Executing 'find /var/opt/lib/pe-puppet/puppet-module/cache/tmp-unpacker20140513-15637-ddp3g4 -type f -exec chmod a-wst {} +'
Debug: Executing 'chown -R 0:0 /var/opt/lib/pe-puppet/puppet-module/cache/tmp-unpacker20140513-15637-ddp3g4'
Debug: HTTP GET https://forgeapi.puppetlabs.com/v3/files/puppetlabs-gcc-0.1.0.tar.gz
Debug: Executing 'tar xzf /var/opt/lib/pe-puppet/puppet-module/cache/puppetlabs-gcc20140513-15637-o9857r --no-same-owner -C /var/opt/lib/pe-puppet/puppet-module/cache/tmp-unpacker20140513-15637-y1vi5s'
Debug: Executing 'find /var/opt/lib/pe-puppet/puppet-module/cache/tmp-unpacker20140513-15637-y1vi5s -type d -exec chmod 755 {} +'
Debug: Executing 'find /var/opt/lib/pe-puppet/puppet-module/cache/tmp-unpacker20140513-15637-y1vi5s -type f -exec chmod a-wst {} +'
Debug: Executing 'chown -R 0:0 /var/opt/lib/pe-puppet/puppet-module/cache/tmp-unpacker20140513-15637-y1vi5s'
Debug: HTTP GET https://forgeapi.puppetlabs.com/v3/files/mhuffnagle-make-0.0.2.tar.gz
Debug: Executing 'tar xzf /var/opt/lib/pe-puppet/puppet-module/cache/mhuffnagle-make20140513-15637-1eypmms --no-same-owner -C /var/opt/lib/pe-puppet/puppet-module/cache/tmp-unpacker20140513-15637-1sa7ryj'
Debug: Executing 'find /var/opt/lib/pe-puppet/puppet-module/cache/tmp-unpacker20140513-15637-1sa7ryj -type d -exec chmod 755 {} +'
Debug: Executing 'find /var/opt/lib/pe-puppet/puppet-module/cache/tmp-unpacker20140513-15637-1sa7ryj -type f -exec chmod a-wst {} +'
Debug: Executing 'chown -R 0:0 /var/opt/lib/pe-puppet/puppet-module/cache/tmp-unpacker20140513-15637-1sa7ryj'
Debug: HTTP GET https://forgeapi.puppetlabs.com/v3/files/puppetlabs-vcsrepo-0.2.0.tar.gz
Debug: Executing 'tar xzf /var/opt/lib/pe-puppet/puppet-module/cache/puppetlabs-vcsrepo20140513-15637-1adawzc --no-same-owner -C /var/opt/lib/pe-puppet/puppet-module/cache/tmp-unpacker20140513-15637-13ktisn'
Debug: Executing 'find /var/opt/lib/pe-puppet/puppet-module/cache/tmp-unpacker20140513-15637-13ktisn -type d -exec chmod 755 {} +'
Debug: Executing 'find /var/opt/lib/pe-puppet/puppet-module/cache/tmp-unpacker20140513-15637-13ktisn -type f -exec chmod a-wst {} +'
Debug: Executing 'chown -R 0:0 /var/opt/lib/pe-puppet/puppet-module/cache/tmp-unpacker20140513-15637-13ktisn'
Debug: HTTP GET https://forgeapi.puppetlabs.com/v3/files/gentoo-portage-2.1.0.tar.gz
Debug: Executing 'tar xzf /var/opt/lib/pe-puppet/puppet-module/cache/gentoo-portage20140513-15637-xlpr8l --no-same-owner -C /var/opt/lib/pe-puppet/puppet-module/cache/tmp-unpacker20140513-15637-l2swtt'
Debug: Executing 'find /var/opt/lib/pe-puppet/puppet-module/cache/tmp-unpacker20140513-15637-l2swtt -type d -exec chmod 755 {} +'
Debug: Executing 'find /var/opt/lib/pe-puppet/puppet-module/cache/tmp-unpacker20140513-15637-l2swtt -type f -exec chmod a-wst {} +'
Debug: Executing 'chown -R 0:0 /var/opt/lib/pe-puppet/puppet-module/cache/tmp-unpacker20140513-15637-l2swtt'
Notice: Installing -- do not interrupt ...
Error: No such file or directory - /etc/puppetlabs/puppet/modules/r10k/spec/fixtures/modules/r10k/files

There is a broken sym link from a test directory.. at this point the ./r10k/files directory doesn't exist.. and

$ ll /etc/puppetlabs/puppet/modules/r10k/spec/fixtures/modules/r10k/files
/etc/puppetlabs/puppet/modules/r10k/spec/fixtures/modules/r10k/files -> ../../../../files

 ll /etc/puppetlabs/puppet/modules/r10k/
drwxr-xr-x 4 root pe-puppet 4096 May 13 08:31 spec

[brettswift]

I'm not really sure what this could be. When extracting the artifact manually with tar, it preserves the sym links. This may be a bug with the puppet module tool instead?

I can direct the puppet module tool to install zack-r10k to a different directory and I get the same result.

[brettswift]

One more thing - in vagrant, this works on CentOS 6.4.

[acidprime]

Can't seem to replicate

puppet module install zack/r10k --debug --version 1.0.2

Notice: Preparing to install into /etc/puppetlabs/puppet/modules ...
Notice: Downloading from https://forgeapi.puppetlabs.com ...
Debug: HTTP GET https://forgeapi.puppetlabs.com/v3/releases?module=zack-r10k
Debug: HTTP GET https://forgeapi.puppetlabs.com/v3/releases?module=puppetlabs-stdlib
Debug: HTTP GET https://forgeapi.puppetlabs.com/v3/releases?module=puppetlabs-stdlib&limit=20&offset=20
Notice: Found at least one version of puppetlabs-stdlib compatible with PE (3.2.2);
Notice: Skipping versions which don't express PE compatibility. To install
the most recent version of the module regardless of compatibility
with PE, use the '--ignore-requirements' flag.
Debug: HTTP GET https://forgeapi.puppetlabs.com/v3/releases?module=puppetlabs-ruby
Debug: HTTP GET https://forgeapi.puppetlabs.com/v3/releases?module=puppetlabs-gcc
Debug: HTTP GET https://forgeapi.puppetlabs.com/v3/releases?module=puppetlabs-pe_gem
Debug: HTTP GET https://forgeapi.puppetlabs.com/v3/releases?module=mhuffnagle-make
Debug: HTTP GET https://forgeapi.puppetlabs.com/v3/releases?module=puppetlabs-inifile
Notice: Found at least one version of puppetlabs-inifile compatible with PE (3.2.2);
Notice: Skipping versions which don't express PE compatibility. To install
the most recent version of the module regardless of compatibility
with PE, use the '--ignore-requirements' flag.
Debug: HTTP GET https://forgeapi.puppetlabs.com/v3/releases?module=puppetlabs-vcsrepo
Debug: HTTP GET https://forgeapi.puppetlabs.com/v3/releases?module=puppetlabs-git
Debug: HTTP GET https://forgeapi.puppetlabs.com/v3/releases?module=gentoo-portage
Debug: HTTP GET https://forgeapi.puppetlabs.com/v3/releases?module=puppetlabs-concat
Notice: Found at least one version of puppetlabs-concat compatible with PE (3.2.2);
Notice: Skipping versions which don't express PE compatibility. To install
the most recent version of the module regardless of compatibility
with PE, use the '--ignore-requirements' flag.
Debug: HTTP GET https://forgeapi.puppetlabs.com/v3/releases?module=ripienaar-concat
Info: Resolving dependencies ...
Info: Preparing to install ...
Debug: HTTP GET https://forgeapi.puppetlabs.com/v3/files/zack-r10k-1.0.2.tar.gz
Debug: Executing 'tar xzf /var/opt/lib/pe-puppet/puppet-module/cache/zack-r10k20140603-10400-gijt7s --no-same-owner -C /var/opt/lib/pe-puppet/puppet-module/cache/tmp-unpacker20140603-10400-nk0ww'
Debug: Executing 'find /var/opt/lib/pe-puppet/puppet-module/cache/tmp-unpacker20140603-10400-nk0ww -type d -exec chmod 755 {} +'
Debug: Executing 'find /var/opt/lib/pe-puppet/puppet-module/cache/tmp-unpacker20140603-10400-nk0ww -type f -exec chmod a-wst {} +'
Debug: Executing 'chown -R 0:0 /var/opt/lib/pe-puppet/puppet-module/cache/tmp-unpacker20140603-10400-nk0ww'
Notice: Installing -- do not interrupt ...
/etc/puppetlabs/puppet/modules
└─┬ zack-r10k (v1.0.2)
  ├── gentoo-portage (v2.1.0)
  ├── mhuffnagle-make (v0.0.2)
  ├── puppetlabs-gcc (v0.1.0)
  ├── puppetlabs-git (v0.0.3)
  ├── puppetlabs-inifile (v1.0.0) [/opt/puppet/share/puppet/modules]
  ├── puppetlabs-pe_gem (v0.0.1)
  ├── puppetlabs-ruby (v0.1.0)
  ├── puppetlabs-stdlib (v3.2.0) [/opt/puppet/share/puppet/modules]
  └── puppetlabs-vcsrepo (v0.2.0)

3.4.3 (Puppet Enterprise 3.2.2)

[acidprime]

if you are still having issues I would probobly just clone the module from github and file a bug with the puppet module tool with the PE support team

[brettswift]

What OS are you running this on? If you want I could send you a
Vagrantfile to reproduce this. It won't work on CentOS or RHEL 6.4 for
me.

Brett

On Tue, Jun 3, 2014 at 1:52 PM, Zack Smith notifications@github.com wrote:

Closed #43 https://github.com/acidprime/r10k/issues/43.

—
Reply to this email directly or view it on GitHub
https://github.com/acidprime/r10k/issues/43#event-127639608.

[acidprime]

@brettswift
've been speaking with the puppet forge team about this, and it looks like it may in fact be a bug introduced in PE 3.2.1. There's an issue for it on our internal tracker as well, created by the forge team.

Essentially, it occurs when a module takes "too long" to download. Either due to a slow network connection, or a large module, or forge performance degradation (although hopefully it's rarely that last one!).

Currently, the only known "workaround" is to download the module tarball and install it directly. E.g. something like:

wget https://forgeapi.puppetlabs.com/v3/files/zack-r10k-1.0.2.tar.gz
puppet module install zack-r10k-1.0.2.tar.gz

It will still reach out to the network to download the dependencies, and that should succeed.

If even that fails you can install the module with --ignore-dependencies, and it won't need to reach out to the forge at all during the command's execution, but you'll then have to manually install the module's dependencies.

[brettswift]

Thanks for the update. Right now I did a manual install of dependencies and then just an untar of the module. I didn't realize that you could use the module tool on the tar file.

As a complete side note - the forge seemed to have forwarded their http API to an https API last night - which actually broke my server bootstrap script as I didn't have the no-check-certificate option on my wget call. Not a big problem though.

Thanks again! I'll monitor this issue.

Brett
Sent from my iPhone

On Jun 12, 2014, at 19:45, Zack Smith notifications@github.com wrote:

@brettswift
've been speaking with the puppet forge team about this, and it looks like it may in fact be a bug introduced in PE 3.2.1. There's an issue for it on our internal tracker as well, created by the forge team.

Essentially, it occurs when a module takes "too long" to download. Either due to a slow network connection, or a large module, or forge performance degradation (although hopefully it's rarely that last one!).

Currently, the only known "workaround" is to download the module tarball and install it directly. E.g. something like:

wget https://forgeapi.puppetlabs.com/v3/files/zack-r10k-1.0.2.tar.gz
puppet module install zack-r10k-1.0.2.tar.gz
It will still reach out to the network to download the dependencies, and that should succeed.

If even that fails you can install the module with --ignore-dependencies, and it won't need to reach out to the forge at all during the command's execution, but you'll then have to manually install the module's dependencies.

Sorry you were bit by this! I'm making a note to engineering that you were affected by this, and would like to see the bug resolved.

—
Reply to this email directly or view it on GitHub.

[dreilly1982]

I am seeing this as well with PE 3.2.3, I have also downloaded and attempted to install from tar, both with and without --ignore-dependencies flag. The error I am getting is SLIGHTLY different.

Error: No such file or directory - /etc/puppetlabs/puppet/modules/r10k/spec/fixtures/modules/r10k/templates