Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

autoload puppet/provider/archive/curl #105

Closed
eperdeme opened this issue Dec 23, 2015 · 36 comments
Closed

autoload puppet/provider/archive/curl #105

eperdeme opened this issue Dec 23, 2015 · 36 comments

Comments

@eperdeme
Copy link
Contributor

Hello,

I get this error randomly on my clients and if I re-run puppet sometimes it goes away and works without any problems. Any idea why it randomly occurs ?

Puppet: 3.8.2
Module; 0.4.4

Error: Could not retrieve catalog from remote server: Error 400 on SERVER: Evaluation Error: Error while evaluating a Resource Statement, Evaluation Error: Error while evaluating a Resource Statement, Could not autoload puppet/type/archive: Could not autoload puppet/provider/archive/curl: Could not find parent provider ruby of curl at /etc/puppetlabs/code/environments/develop/modules/archive/manifests/nexus.pp:53:3 at /etc/puppetlabs/code/environments/develop/modules/ui_broker/manifests/init.pp:33 on node rnode01.test.com
Warning: Not using cache on failed catalog
Error: Could not retrieve catalog; skipping run

@igalic
Copy link
Contributor

igalic commented Jan 4, 2016

that's interesting / bizarre.

@daenney
Copy link
Member

daenney commented Jan 4, 2016

Alright, breaking down the issue:

It's complaining that:

Could not autoload puppet/provider/archive/curl: Could not find parent provider ruby of curl

This relates to L1 of lib/provider/puppet/archive/curl.rb which does:

Puppet::Type.type(:archive).provide(:curl, :parent => :ruby) do

The ruby parent provider we can see here.

As long as those two files are in the right location on your Puppet master and plugin sync is turned on everywhere this Should Work ™️.

@project0
Copy link

project0 commented Feb 2, 2016

i have exactly the same issue with wget, but it occures every time and i dont know how to fix. Pluginsync works fine and all files are present on destination node.

Error: Could not retrieve catalog from remote server: Error 400 on SERVER: Could not autoload puppet/type/archive: Could not autoload puppet/provider/archive/wget: Could not find parent provider ruby of wget

Puppet 3.6.1
Module 0.4.4

@project0
Copy link

project0 commented Feb 2, 2016

I solved the problem right now, it seems to be a problem on the server side when using environments.
We have different environments and put puppet modules in each environment, the puppetmaster itself has no modules in /etc/puppet/modules. The puppetmaster doesnt have the module in its cache, so it fails to autoload (probably related to $LOAD_PATH), but i dont know why the provider is applied on server side?

@daenney
Copy link
Member

daenney commented Feb 2, 2016

Oh, this is a typical problem with the old Puppet server (the ruby/passenger one). For things to be loadable they need to exist in the production environment and that's the environment it will be loaded from. At least that's how it used to work for functions, unsure about types/providers but I seem to remember a similar issue. Switching to the new puppetserver will fix that.

@daenney daenney closed this as completed Feb 2, 2016
@gdubicki
Copy link
Member

gdubicki commented Apr 7, 2016

I am having this problem too with Puppet 3.8.4, using r10k environments (a custom-named one, not production). What should I do to resolve this? Puppet upgrade is not an option.

@bastelfreak
Copy link
Member

I don't think that there is another solution, Update your puppetserver or switch to a env named production.

@bastelfreak bastelfreak reopened this Apr 7, 2016
@rnelson0
Copy link
Sponsor Member

rnelson0 commented Apr 7, 2016

This is not a module issue but a puppet environments issue. The auto load failures are caused because the master service is not loading the required files into its environment, for a variety of possible causes. There are multiple puppet labs tickets to resolve this. It affects tons of modules with types and providers.

In addition to bastelfreak's suggestion to use production, you also need to make sure the production environment is loaded at least once. For example, restarting the master service then immediately testing against an environment called foo will generate errors, until at least one agent (including the master) checks in against production.

@gdubicki
Copy link
Member

gdubicki commented Apr 8, 2016

I can't use production (yet) as I am still developing my solution. Our process require me to use my own environment during that.

I have tried loading the production environment at least once and it didn't help. Any other ideas?

@daenney
Copy link
Member

daenney commented Apr 9, 2016

Spin up a new Puppet master with its own 'production' environment and point your node at that. Since this is not a problem with the module but how Ruby code loading works when running on Passenger I'm closing the issue.

@daenney daenney closed this as completed Apr 9, 2016
@flozano
Copy link

flozano commented May 13, 2016

We're running in puppetserver:

puppet-server-3.8.6-1.el6.noarch
puppetlabs-release-6-11.noarch
puppet-3.8.6-1.el6.noarch
puppetserver-1.1.3-1.el6.noarch
$ sudo service puppetserver status
puppetserver (pid  6472) is running...
$ sudo service puppetmaster status
puppet is stopped

and hitting this issue too... we don't have an environment named "production", nor we can have it.

EDIT: After a retry, the issue was gone... Quite strange.

@igalic
Copy link
Contributor

igalic commented May 17, 2016

yeah, i've experienced similar issues on puppetserver with the file_concat type/provider…

@nanliu
Copy link
Contributor

nanliu commented May 17, 2016

This issue can occur for multiple environments with different module version and it's still an open issue for puppetserver: https://tickets.puppetlabs.com/browse/SERVER-94

@chilcano
Copy link

chilcano commented Aug 4, 2016

I've got same issue on Puppet 4.5 with :

Error: Could not retrieve catalog from remote server: Error 400 on SERVER: Evaluation Error: Error while evaluating a Resource Statement, Evaluation Error: Error while evaluating a Type-Name, Could not autoload puppet/type/archive: Could not autoload puppet/provider/archive/wget: Could not find parent provider ruby of wget at /etc/puppetlabs/code/environments/development/modules/kafka/manifests/init.pp:108:16 at /etc/puppetlabs/code/environments/development/modules/role/manifests/kafka_standalone.pp:15 on node slzcsr000000132-013.fusion
Warning: Not using cache on failed catalog
Error: Could not retrieve catalog; skipping run

... and solved it copying/installing puppet-archive module to /etc/puppetlabs/code/environments/production.

Regards.

@ryayon
Copy link

ryayon commented Dec 5, 2016

Chicalno found a workaround (although it is not a solution)

@markfaine
Copy link

@chilcano Is that literally all that you did? We don't have a production environment but I created an empty one with just two modules archive and stdlib. I then tried agent -t --environment production which completed successfully and then tried agent -t on my actual environment and that failed with the same error message.

@chilcano
Copy link

@markfaine Yes, I did just that.

@gdubicki
Copy link
Member

Sorry for reporting that so late :P... I use puppet-master 3.8.7 installed from RPM on Centos 7 and when this problem occured again today, restarting the puppetmaster and then re-running the agent helped.

@stefanandres
Copy link

It's still the same issue for me with Puppet-4.10.1 and puppetserver-2.7.2 with r10k environments.

@haani-niyaz
Copy link

I ran into the same problem.

Based on a comment made by @project0, I added the module to my basemodulepath and re-ran the agent which worked. I then removed the module from the basemodulepath and it continues to work.

You can run the following command to find your basemodulepath:
$ sudo puppet config print basemodulepath

My feature branch environment.conf file:
modulepath = site:modules:$basemodulepath

@kwisatz
Copy link

kwisatz commented Oct 23, 2017

I can confirm @haani-niyaz 's work-around works for me as well. I'm still on puppet 3.8.7 with r10k, but I'm confused by the fact that this seems to be a problem on puppet 4.x with r10k too. Can anyone confirm this is an issue upstream (e.g. with puppetserver), or is it the combination of a (mis-)conception on the puppetserver, of how r10k handles environments, and a specific way of doing things in the module?

@akomakom
Copy link

I had a similar issue that was caused by rvm with a different version of ruby set as default while running puppet-agent 5.x (an environment issue).

Could not autoload puppet/type/a2mod: Could not autoload puppet/provider/a2mod/redhat: no such file to load -- puppet/provider/a2mod

"puppet agent -t" worked fine after "rvm system"

@karelyatin
Copy link

I too faced this issue on puppet standalone:-
CentOS Linux release 7.4.1708 (Core)
Runtime environment: puppet_version=4.8.2, ruby_version=2.0.0, run_mode=user, default_encoding=UTF-8
Following ways made it to work but can't find the actual fix for it:-
puppet-archive is located at /usr/share/openstack-puppet/modules/archive

  1. If i keep puppet-archive to /etc/puppet/modules it works
    sudo cp -a /usr/share/openstack-puppet/modules/archive /etc/puppet/modules/
    sudo puppet apply --modulepath /etc/puppet/modules -t -e 'archive::download {"apache-tomcat-6.0.26.tar.gz": ensure => present, url => "http://archive.apache.org/dist/tomcat/tomcat-6/v6.0.26/bin/apache-tomcat-6.0.26.tar.gz",}'
  2. Create a directory /etc/puppet/environments and run puppet apply works:-
    sudo puppet apply --modulepath /usr/share/openstack-puppet/modules/ -t -e 'archive::download {"apache-tomcat-6.0.26.tar.gz": ensure => present, url => "http://archive.apache.org/dist/tomcat/tomcat-6/v6.0.26/bin/apache-tomcat-6.0.26.tar.gz",}'

Any suggestions what's the cause for it?

@alexjfisher
Copy link
Member

@karelyatin Have you tried using puppet generate types?

@karelyatin
Copy link

@alexjfisher Thanks for the reply, I am getting below error while running it. Am i running it correctly?
[centos@poitest6 weirdo]$ sudo puppet generate types --environment production
Error: undefined method path_to_env' for #<Puppet::Settings::EnvironmentConf::Static:0x000000038e23c8> Error: Try 'puppet help generate types' for usage [centos@poitest6 weirdo]$ sudo puppet generate types Error: undefined method path_to_env' for #Puppet::Settings::EnvironmentConf::Static:0x000000022025e0
Error: Try 'puppet help generate types' for usage

NOTE: i am running puppet standalone, no puppet master/agent

@alexjfisher
Copy link
Member

You should still be able to run generate types, but actually in a masterless setup, I doubt it will change much.

To trick the tool into running, maybe...

puppet generate types --environmentpath=/usr/share --environment=openstack-puppet

?

@karelyatin
Copy link

@alexjfisher Still failing,
can you suggest on 2nd point of mine i.e:-
2. Create a directory /etc/puppet/environments and run puppet apply works:-
sudo puppet apply --modulepath /usr/share/openstack-puppet/modules/ -t -e 'archive::download {"apache-tomcat-6.0.26.tar.gz": ensure => present, url => "http://archive.apache.org/dist/tomcat/tomcat-6/v6.0.26/bin/apache-tomcat-6.0.26.tar.gz",}'

Why above is working after creating /etc/puppet/environments directory.

@project0
Copy link

project0 commented Jan 3, 2018

I faced an similar problem in puppet-elasticsearch which has been fixed by setting the correct load path (https://github.com/elastic/puppet-elasticsearch/blob/master/lib/puppet/type/elasticsearch_index.rb#L1). In my opinion this should also work in this module.

rdoproject pushed a commit to rdo-infra/ansible-role-weirdo-puppet-openstack that referenced this issue Jan 4, 2018
When puppet modules are installed from rpm
ensure /etc/puppet/environments directory exists.
This is a workaround to avoid autoload issue [1].

[1] voxpupuli/puppet-archive#105

Change-Id: Ib8cfefb774a7eac0a0ad584d3a4e5d6779d5de4f
@ljkimmel
Copy link

ljkimmel commented Feb 1, 2018

I feel like this is related to something in the way the module is written. I have a ton of modules in use and this is the only one that has this type of issue.

I can kick off a handful of server builds and maybe half of them, on average, will fail with this error while the others will work fine. Typically, at some point a Puppet run will complete.

@alexjfisher
Copy link
Member

@ljkimmel Have you tried using puppet generate types?

@ljkimmel
Copy link

ljkimmel commented Feb 2, 2018

'Generate' does not appear to be a valid subcommand to the opensource version of Puppet as used with Foreman.

@jsnod
Copy link

jsnod commented Apr 3, 2018

I came across this with Puppet v4.10.10 with r10k and running puppet generate types on the client fixed it for me.

@schlitzered
Copy link

schlitzered commented Apr 20, 2018

i am having the same issue with puppet 5.5.0, and i am also not using the "production" environment

@khaefeli
Copy link

same issue here with 5.5.
puppet generate types helped

@negast
Copy link

negast commented Jan 10, 2019

Had the same issue resolved it by copying the archive module in my production environment. I could still use it in my development environment
cp -r <environment_dir>/modules/archive ---> <production_dir>/modules/

@EmmaTinten
Copy link

This problem still exists with puppet-agent 6.11.1. and server 6.7.2. Adding the module to the Puppetfile for the production environment will solve the problem.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests