manifestdir is deprecated in Puppet 3.6.0 #3740

Closed
ferventcoder opened this Issue May 9, 2014 · 73 comments

Projects

None yet
@ferventcoder

manifest now has the ability to take a directory or fully qualified path to a manifest file. If it is a directory it will parse all files in there and run the result.

Heads up what you are going to start seeing issues on:

==> default: Running provisioner: puppet...
Running Puppet with /tmp/vagrant-puppet-4/manifests/site.pp...
Warning: Setting manifestdir is deprecated. See http://links.puppetlabs.com/env-settings-deprecations
(at c:/Program Files/Puppet Labs/Puppet/puppet/lib/puppet/settings.rb:260:in block (2 levels) in parse_global_options'; c:/Program Files/Puppet Labs/Puppet/puppet/lib/puppet/util/command_line/puppet_option_parser.rb:81:incall'; c:/Program Files/Puppet Labs/Puppet/puppet/lib/puppet/util/command_line/puppet_option_parser.rb:81:in block in pass_only_last_value_on_to'; c:/Program Files/Puppet Labs/Puppet/puppet/lib/puppet/util/command_line/trollop.rb:432:incall'; c:/Program Files/Puppet Labs/Puppet/puppet/lib/puppet/util/command_line/trollop.rb:432:in block in parse'; c:/Program Files/Puppet Labs/Puppet/puppet/lib/puppet/util/command_line/trollop.rb:393:ineach'; c:/Program Files/Puppet Labs/Puppet/puppet/lib/puppet/util/command_line/trollop.rb:393:in parse'; c:/Program Files/Puppet Labs/Puppet/puppet/lib/puppet/util/command_line/puppet_option_parser.rb:74:inparse'; c:/Program Files/Puppet Labs/Puppet/puppet/lib/puppet/settings.rb:270:in parse_global_options'; c:/Program Files/Puppet Labs/Puppet/puppet/lib/puppet/settings.rb:239:ininitialize_global_settings'; c:/Program Files/Puppet Labs/Puppet/puppet/lib/puppet.rb:150:in do_initialize_settings_for_run_mode'; c:/Program Files/Puppet Labs/Puppet/puppet/lib/puppet.rb:136:ininitialize_settings'; c:/Program Files/Puppet Labs/Puppet/puppet/lib/puppet/util/command_line.rb:86:in block in execute'; c:/Program Files/Puppet Labs/Puppet/puppet/lib/puppet/util.rb:479:inexit_on_fail'; c:/Program Files/Puppet Labs/Puppet/puppet/lib/puppet/util/command_line.rb:85:in execute'; c:/Program Files/Puppet Labs/Puppet/puppet/bin/puppet:4:in

')

I'm thinking we can change the provisioner from this:

 config.vm.provision :puppet, :options => ["--debug --trace --verbose"] do |puppet|
    puppet.manifests_path = "puppet/manifests"
    puppet.manifest_file  = "site.pp"
    puppet.module_path = "puppet/modules"
  end

to this:

 config.vm.provision :puppet, :options => ["--debug --trace --verbose"] do |puppet|
    puppet.manifest_file  = "puppet/manifests/site.pp"
    puppet.module_path = "puppet/modules"
  end

or try something else? Thoughts?

From my comments below, leave the Ux the same and merge the options together and provide them to puppet as one setting.

@ferventcoder

@zaphod42 this sum it up?

@ferventcoder ferventcoder changed the title from [Enhancement] Puppet Shell provisioner - manifestsdir is deprecated in Puppet 3.6.0 to [Enhancement] Puppet Shell provisioner - manifestdir is deprecated in Puppet 3.6.0 May 9, 2014
@ferventcoder ferventcoder changed the title from [Enhancement] Puppet Shell provisioner - manifestdir is deprecated in Puppet 3.6.0 to [puppet-provisioner] manifestdir is deprecated in Puppet 3.6.0 - may require changes to provisioner interface May 9, 2014
@ferventcoder

Alternatively, it could stay the same and the options merge to provide manifest as one setting.

@mitchellh
Owner

Yeah I think the options staying the same works. I think internally we can just continue to share that folder and then just change the way that we are calling Puppet.

@zaphod42
zaphod42 commented May 9, 2014

πŸ‘ the --manifestdir option (and setting) is going away as part of the directory environment work.

@narkisr
narkisr commented May 26, 2014

A fix for this will be great

@ppp0 ppp0 referenced this issue in cargomedia/puppet-packages May 27, 2014
Merged

Setting modulepath is deprecated in puppet.conf #603

@gustavoca

+1

@feresr
feresr commented Jun 4, 2014

Same problem here, and I'm only using

config.vm.provision "puppet"

@sitle
sitle commented Jun 14, 2014

I'm stuck with this problem. Can you submit a patch for this please ?

++1

@ttsuchi
ttsuchi commented Jun 17, 2014

+1

@salimane
Contributor

+1

@KerryJones

+1

@Stoosh
Stoosh commented Jul 14, 2014

+1

@kensykora

πŸ‘

@benh57
Contributor
benh57 commented Aug 5, 2014

I can take a look at patching this.

We can add a new setting to simply pass in environmentpath directly from vagrant, which will override the manifest and modulepath settings. We could also allow folks to pass in an environment.

If environmentpath is not passed in was thinking we could create a temp environment dir with an environment.conf in it which is generated using the passed-in from Vagrantfile modulepath and manifestfile settings.

@rasschaert
Contributor

Can't use the puppet provisioner anymore so I've been using this workaround until this issue is addressed:

    config.vm.provision :shell, inline: 'sudo puppet apply --environment dev \
      --pluginsync --hiera_config /etc/puppet/hiera.yaml \
      /etc/puppet/environments/dev/manifests \
      --environmentpath /etc/puppet/environments'

...along with some synced_folders to put the manifests and hiera files in the right folders.

@ferventcoder

What? Why can't you use the puppet provisioner anymore? A deprecation
warning is what this issue is about.

On Wednesday, August 6, 2014, Kenny Rasschaert notifications@github.com
wrote:

Can't use the puppet provisioner anymore so I've been using this
workaround until this issue is addressed:

config.vm.provision :shell, inline: 'sudo puppet apply --environment dev \      --pluginsync --hiera_config /etc/puppet/hiera.yaml \      /etc/puppet/environments/dev/manifests \      --environmentpath /etc/puppet/environments'

...along with some synced_folders to put the manifests and hiera files in
the right folders.

β€”
Reply to this email directly or view it on GitHub
#3740 (comment).

Rob
"Be passionate in all you do"

http://devlicio.us/blogs/rob_reynolds
http://ferventcoder.com
http://twitter.com/ferventcoder

@rasschaert
Contributor

Oh, I think I wanted to post that comment under issue 3402. I had a couple of tabs open with open puppet privisoner issues. My bad.

@lmayorga1980

Is this already merged? to 1.6.3?

@ferventcoder

@rasschaert no worries. :)

@benh57
Contributor
benh57 commented Aug 11, 2014

@lmayorga1980 as far as i know no one's implemented this yet. I planned to take a look at it soon if no one else does.

@mdwheele mdwheele referenced this issue in mdwheele/vm Aug 27, 2014
Closed

Puppet Deprecation Notices #45

@vpassapera

+1

@fe-lix-
fe-lix- commented Sep 8, 2014

+1

@rbu
rbu commented Sep 10, 2014

Deprecation warnings should be fixed so we can upgrade before it breaks.

@lmayorga1980

πŸ‘ +1

@larstobi

+1

@benh57
Contributor
benh57 commented Sep 15, 2014

Ok, I started implementing this locally. Will be a significant refactor of the puppet provisioner to fully support directory environments.

@benh57
Contributor
benh57 commented Sep 15, 2014

I see various folks commenting that the options should remain the same for now. IMO, it would be cleanest if Vagrant had full environment support, ie, i could just specify:

puppet.environment = 'production'
puppet.environmentpath = '/path/to/my/environments'

and it would 'just work'.

On versions of puppet where its deprecated, but folks haven't moved over to environments yet (ie still using the old options in the vagrantfile), we could make a 'fake' environment to make the warning go away. (the 'change how we're calling puppet' suggested above)

Let me know if anyone objects. ;)

@cgeisel
cgeisel commented Sep 15, 2014

@benh57 great! The warnings don't bother me, but I am looking forward to support for puppet environments, so thank you!

@jtomaszon

Same here!

==> default: Warning: Setting templatedir is deprecated. See http://links.puppetlabs.com/env-settings-deprecations
==> default:    (at /usr/lib/ruby/vendor_ruby/puppet/settings.rb:1117:in `issue_deprecation_warning')

+1 to environment support!

@vpassapera

Any updates on this? Very much looking forward to trying it out!

Ty!!

@mcandre
mcandre commented Dec 3, 2014

+1

@flagster76 flagster76 referenced this issue in HBRGTech/daisy Dec 8, 2014
Open

"Warnings" while provisioning #6

@sethvargo sethvargo changed the title from [puppet-provisioner] manifestdir is deprecated in Puppet 3.6.0 - may require changes to provisioner interface to manifestdir is deprecated in Puppet 3.6.0 Dec 13, 2014
@cisco87
Contributor
cisco87 commented Dec 17, 2014

If I can make a suggestion for those who stopped using the vagrants puppet provisioner due to this problem: you can still uninstall your existing puppet and reinstall the compatible version using

gem install puppet -v "3.5.1"
while this gets fixed

@adamcstephens

This doesn't break the provisioner, the deprecation warning is just that, a warning.

@ferventcoder

@cisco87 This is deprecated, not broken as @adamcstephens pointed out.

@netmikey
netmikey commented Feb 5, 2015

+1

@zipkid
zipkid commented Mar 13, 2015

Hello....
Puppet 4 is about to be released quite soon.
I am in the process of testing it and because of this issue i can not use the puppet provisioner.

==> puppetserver: Running provisioner: puppet...
==> puppetserver: Running Puppet with default.pp...
==> puppetserver: Error: Could not parse application options: invalid option: --manifestdir

It seems like the discussion of how to fix/implement this is done, so it would be extremely useful to have this fixed real soon...

Stefan - Zipkid - Goethals.

@zipkid
zipkid commented Mar 20, 2015

@benh57 , did you manage to get some code ready to submit for this?

@laapsaap

+1

We cant test puppet 4 rc2 because of this with vagrant. Its an error not warning now.

==> default: Error: Could not parse application options: invalid option: --manifestdir
The SSH command responded with a non-zero exit status. Vagrant
assumes that this means the command failed. The output for this command
should be in the log above. Please read the output to determine what
went wrong.

@benh57
Contributor
benh57 commented Apr 11, 2015

Will continue work on this this weekend.

@benh57
Contributor
benh57 commented Apr 12, 2015

If anyone would like to test it out, code currently is on my fork:

https://github.com/benh57/vagrant/tree/environments_wip
You can specify:
puppet.environment_path = "../puppet/environments"
puppet.environment = "testenv"

-- If only environment and environments_path are specified, it will parse and use the manifest specified in the environment.conf file.
-- If manifests_path and manifest_file is specified along with the environment options, the manifest from the environment will be overridden by the specified manifest_file.
-- If manifests_path and manifest_file are specified without environments, the old non-environment mode will be used (which will fail on Puppet 4+)

Will submit PR soon.

@laapsaap

^^ Confirmed working https://github.com/benh57/vagrant/tree/environments_wip

Its working great. Thank you!

@roman-mueller

Puppet 4 has been released and now Vagrant won't work anymore without shell provisioner workarounds.
Would be great to get this solved soon.

@sethvargo sethvargo closed this May 31, 2015
@l0b0 l0b0 referenced this issue in gini/puppet-idea Jun 20, 2015
Closed

Not compatible with Puppet 4 #11

@l0b0 l0b0 added a commit to l0b0/root that referenced this issue Jun 20, 2015
@l0b0 l0b0 Install IntelliJ IDEA Ultimate 14.1.3
This is currently untestable on my system because of
<mitchellh/vagrant#3740>.

Had to include a workaround dependency; see
<gini/puppet-idea#9 (comment)>.
7adb0e5
@sknebel sknebel referenced this issue in meet-here/vagrant-env Jun 23, 2015
Closed

templatedir warning #6

@l0b0
l0b0 commented Jun 25, 2015

Is there a configuration workaround for masterless Puppet (that is, without installing a fork or older version)?

$ pacman --query --owns "$(which puppet)"
/usr/bin/puppet is owned by puppet 4.1.0-1
$ vagrant --version
Vagrant 1.7.2
@ball-hayden

I think the only workaround that wouldn't involve forking or using an older version of puppet would be to run a shell provisioner that invokes puppet apply with the correct flags...

@ball-hayden

Although, @sethvargo has closed this issue - has this been fixed on master?

@sethvargo
Collaborator

@ball-hayden Puppet 4 support has landed on master, but is not yet in a released version. Thanks!

@lmayorga1980

@ball-hayden Fixed on master but you will get a PATH issue.

screen shot 2015-06-25 at 11 18 46 am

@benh57
Contributor
benh57 commented Jun 25, 2015

We can fix the puppet agent path autodetection by adding more common paths to check.

But, to work around you can use the new config parameter 'puppet.binary_path' to pass the path to your puppet agent.

@lmayorga1980

πŸ‘ @benh57

@ferventcoder

@lmayorga1980 what was that issue again? I provided what I think is the commit that causes the issue.

@ferventcoder

Found it #5806

@lmayorga1980

@ferventcoder seems like my comment was just noise. I think there is another issue for this.

@alvagante alvagante added a commit to example42/tp-playground that referenced this issue Jul 8, 2015
@alvagante alvagante Preparing for Puppet4
Waiting for mitchellh/vagrant#3740 fix
5304313
@martijn
martijn commented Sep 1, 2015

So Vagrant 1.7.4 still calls puppet with --manifestdir, yet newer boxes like centos-7.0-64-puppet 1.0.2 include a version of Puppet that no longer supports this parameter. Seems like that needs fixing.

@thenexxuz

This helped me at least get up and running without needing to do much work on updating a lot of things.

I added this to my Vagrantfile after the config.vm.box line:
config.vm.box_version = "1.0.1"

@codesplicer

+1
Still facing this issue with Error: Could not parse application options: invalid option: --manifestdir using the puppetlabs/ubuntu-12.04-64-puppet box at v1.0.2, which ships with puppet v4.2.1. As @thenexxuz suggested, using v1.0.1 of this box is compatible as it ships with puppet v3.7.4

@jbeard6
jbeard6 commented Sep 8, 2015

For what it's worth, I have worked around this issue by using a script provisioner to invoke puppet apply directly. This not only has the benefit of working, but also more closely replicates how puppet will actually be used in a production environment to apply my config.

Still, I'm looking forward to having this issue resolved. This has been a long time coming.

@ball-hayden

I have been working around this by placing the contents of https://github.com/benh57/vagrant/tree/environments_wip/plugins/provisioners/puppet into lib/puppet in my project directory, adjusting the plugin name to puppet4, then adding require_relative 'lib/puppet/plugin.rb' at the top of my Vagrantfile. This allows me to use a 'puppet4' provisioner, which works correctly.

@benh57
Contributor
benh57 commented Sep 9, 2015

With puppet 4 you need to switch your Vagrantfile to use puppet environments, instead of the manifest file setting. This is documented in the vagrant docs. I'm not sure how that premade box is set up, but i can try it out.

@benh57
Contributor
benh57 commented Sep 9, 2015
  • One would assume if you pass a path to manifest file, that you want to use that manifest file (and the old option).

I suppose i could, if an environment file is not specified, make a 'fake' environment file, pointing at the specified manifest file. However, that would only work going back a couple versions of puppet, i think it would work on 3 or later? We have no autodetection of puppet version, but that could potentially be added.

@benh57
Contributor
benh57 commented Sep 9, 2015

@ball-hayden I think all of that branch ('environments_wip' ) made it into Vagrant 1.7.4, didn't it? Maybe it was broken later by another change?

@benh57
Contributor
benh57 commented Sep 9, 2015

@jbeard6 It was fixed. If you're using puppet 4, pass your environment. This will put it into environments mode.

puppet.environment = "production"
puppet.environment_path = "path/to/environments/on/host"

(and optionally, the path to a manifest - which will override the environment's manifest if you set it!)

puppet.manifests_path = "puppet/manifests"
puppet.manifest_file = "site.pp"

If you JUST to the last 2 settings, we assume that you are on puppet <4, and pass --manifestdir.

In other words -- this bug is not a bug anymore.

  • I am open to a solution like adding autodetection of the puppet version so that vagrant can be more smart, if folks desire this.
@codesplicer

@benh57 This solution assumes that you are in fact using puppet environments with a puppet master, which isn't the case for those of us operating puppet masterless.

@codesplicer

For the record I managed to workaround this issue by using symlinks. Given the following folder tree:

project_foo
    \ puppet
        | puppet.conf
        \ modules
        \ manifests

I created a new folder environmentswhich contained a symlink to the puppet folder in the project_foo folder

project_foo
    \ puppet
    | puppet.conf
        \ modules
        \ manifests
        \ environments
            \ production    <-- Symlink to ../../

Then in my Vagrantfile, as suggested by @benh57 I added the following:

config.vm.provision "puppet" do |puppet|
    puppet.environment = "production"
    puppet.environment_path = "puppet/environments"
    puppet.manifests_path = "puppet/manifests"
    puppet.manifest_file = "vagrant.pp"
    ...

Its a somewhat dirty hack but it works for now.

@adamcstephens

As can be seen by the removal of manifestdir (following the original deprecation notice), environments are intended for both puppet apply and agent. vagrant has support for this now, so I agree with @benh57 that this is no longer a bug.

See BREAK: Directory Environments Replace Config File Environments here: http://docs.puppetlabs.com/puppet/4.0/reference/release_notes.html

@l0b0
l0b0 commented Sep 9, 2015

@adamcstephens From reading Directory Environments Replace Config File Environments and the linked documentation I don't see how it's possible to use environments with masterless Puppet. Some relevant quotes:

Creating Environments:

[The environment] must be located on the Puppet master server(s)

What if I don't have a master at all, even less a separate server? There doesn't seem to be any mention of masterless Puppet in this article.

About Environments:

A Puppet master server can serve each environment with completely different main manifests and modulepaths.
[...]
In Puppet manifests, you can get the name of the current environment by using the $environment variable, which is set by the Puppet master.

Again, what about masterless Puppet?

$environment configuration reference:

For clients (e.g., puppet agent) this determines the environment itself, which is used to find modules and much more. For servers (i.e., puppet master) this provides the default environment for nodes we know nothing about.

No mention of puppet apply. In fact all the documentation around environments seems to assume a master/agent setup, and doesn't mention (in any way I have been able to discover) how it applies to masterless Puppet.

Can you please reopen this bug, or else direct the 30+ people who are interested in this to some documentation on how to do the transition to masterless Puppet environments, if possible?

@benh57
Contributor
benh57 commented Sep 12, 2015

I use vagrant with my environments all day with no puppet master, to test
my configurations for (later) use with the puppet master server.

It's just a named folder, with an environment.conf file in it. I'm not sure
how that's a hack, since it's the intended puppet (4) way.

Maybe it would be 'cleaner' if vagrant would detect your puppet version and
'just work' and help you out by 'faking' an environment ? That seems almost
more of a hack. Though it would have the benefit that existing Vagrantfiles
would continue to work. Perhaps vagrant could warn..

On Wed, Sep 9, 2015 at 6:36 AM, Victor Engmark notifications@github.com
wrote:

@adamcstephens https://github.com/adamcstephens From reading Directory
Environments Replace Config File Environments
https://docs.puppetlabs.com/puppet/4.0/reference/release_notes.html#break-directory-environments-replace-config-file-environments
and the linked documentation I don't see how it's possible to use
environments with masterless Puppet. Some relevant quotes:

Creating Environments
https://docs.puppetlabs.com/puppet/latest/reference/environments_creating.html
:

[The environment] must be located on the Puppet master server(s)

What if I don't have a master at all, even less a separate server? There
doesn't seem to be any mention of masterless Puppet in this article.

About Environments
https://docs.puppetlabs.com/puppet/latest/reference/environments.html:

A Puppet master server can serve each environment with completely
different main manifests and modulepaths.
[...]
In Puppet manifests, you can get the name of the current environment by
using the $environment variable, which is set by the Puppet master.

Again, what about masterless Puppet?

$environment configuration reference
https://docs.puppetlabs.com/references/4.2.latest/configuration.html#environment
:

For clients (e.g., puppet agent) this determines the environment itself,
which is used to find modules and much more. For servers (i.e., puppet
master) this provides the default environment for nodes we know nothing
about.

No mention of puppet apply. In fact all the documentation around
environments seems to assume a master/agent setup, and doesn't mention (in
any way I have been able to discover) how it applies to masterless Puppet.

β€”
Reply to this email directly or view it on GitHub
#3740 (comment).

@l0b0 l0b0 added a commit to l0b0/root that referenced this issue Sep 14, 2015
@l0b0 l0b0 Specify an environment for the Puppet run
Necessary to work around Vagrant issues with Puppet 4
<mitchellh/vagrant#3740 (comment)>.
869cec4
@sultano
sultano commented Oct 14, 2015

@adamcstephens @benh57 It would seem @l0b0 may have a valid point. I agree the intended Puppet 4 way would be to use directory environments instead of dynamic environments.

Starting with Puppet 3.6, Directory Environments started taking over from Dynamic Environments as Puppet’s mechanism for serving different versions of modules and code. In Puppet 4, they’re the default and other environment support is gone.

Source: Directory Environments Replace Config File Environments

I just fail to see why the environment needs to be specified when using puppet apply

Puppet apply does not use the manifest from an environment; it always uses the manifest given on the CLI.

Source: Directories: The Main Manifest(s)

So Vagrantfile's without the environment specified in a masterless setup should still work as expected?

@sultano
sultano commented Oct 14, 2015

As a workaround, I've set the following in my Vagrantfile to force environments mode without having to change the folder structure.

puppet.environment = "production"
puppet.environment_path = "../../"
@benh57
Contributor
benh57 commented Oct 14, 2015

Suleman -- I chose to have it default to using the old system, so that pre
existing vagrantfiles + machines would continue to work.

Vagrant currently has no mechanism to detect your puppet version.

If you're on the older puppet versions, no changes need be made. Changes to
vagrantfiles are only required if you upgrade puppet. I don't consider
specifying the environment as you are doing a workaround, rather it's the
intended behavior.

Ideal would probably be if the mode flipped automatically depending on your
installed puppet version -- anyone is free to implement that.. I just
implemented this based on my needs, well after posting the proposal here.
I'm not a core vagrant developer.

-Ben

On Wed, Oct 14, 2015 at 2:19 AM, Suleman Chikhalia <notifications@github.com

wrote:

As a workaround, I've set the following in my Vagrantfile to force
environments mode without having to change the folder structure.

puppet.environment = "production"
puppet.environment_path = "../../"

β€”
Reply to this email directly or view it on GitHub
#3740 (comment).

@ferventcoder

Vagrant prolly could query Puppet to see what version it is and adjust accordingly.

@martijn
martijn commented Nov 3, 2015

Ok I missed the whole environments / Puppet 4 thing (should've read the changelog). I'm going to try it out but I assume that fixes the issue. Thanks for clarifying @benh57

@iainhallam
iainhallam commented Nov 2, 2016 edited

The documentation currently says (https://www.vagrantup.com/docs/provisioning/puppet_apply.html):

If you are using Puppet 4 or higher, you can provision using Puppet Environments by specifying the name of the environment and the path on the local disk to the environment files

I read this and thought that since I was using only a single manifest, I wouldn't use the environments configuration as it's just extra things to do, when all I want is to apply a single file. If Vagrant is actually using environment configuration as a flag to decide whether to use Puppet 4-style calling, then this needs to be changed to "must" or at least "should".

An alternative would be to add a "manifest" configuration that also makes Vagrant use v4-style calls to Puppet but can take a path and filename together.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment