RFC: Support Puppet 4 only #21
Comments
|
I've been asking my self this question as well for a while. |
|
It will be time when there will be a decent linting/style guide for the new puppet 4 features, otherwise that will be horrible. |
That's a very good point. Having a functional style guide will be important to avoid all kinds of unnecessary discussion on that topic. |
|
Should we make a Puppet-4-Style-Guide repo ? |
|
It's a thought though I would suggest we post the question to puppet-dev@ first. There might already be things in the work and it would be a shame to duplicate such efforts. |
|
I wish PMT could actually consume the metadata in modules, i.e. installing only compatible versions. Maybe we should contribute this feature? |
That's almost impossible to do. The master can be on a much newer version than the agent and even have modules in its environment that don't work with specific distros that the agents can be running on. The only place you could reliably do it is in a masterless environment because there you only have to care about the local machine. |
grumble grumble reality grumble grumble |
|
Alright, total aside here and stop derailing my RFCs already @danzilio but: If there was some way of declaring API compatibility, like API levels in Android, the agents could do this work. Consider a module which declares that it will only work with Language level 4 and is determined to be part of the catalog that an agent needs to apply which only supports up to Language Level 3. There's nothing to stop us from halting execution right then. Similarly we can probably have the agent check the module constraints, so that it doesn't try to run on a too old version of the agent or a platform that's not supported (potentially with a flag to override that if you're on one of those weird distros that usually works but no one bothers to check for). I think what you're suggesting is actually quite feasible, just not by leveraging the PMT. |
|
@daenney keeps taking the bait ;) |
I suppose that's true, but given an agent/master workflow where you use PMT on the master to download modules, the majority case is that your agents are not going to cross major-version boundaries. One concrete improvement to PMT that might help a lot would be adding the ability to fall back to other versions if the latest/current one expresses incompatibility with the version of puppet. (Or hell, even just display what other versions are available) |
|
One meta comment @daenney: is the scope of this discussion about the contents of this repo? Or more broadly about stuff inside the vox pupuli namespace? |
|
@ahpook The latter, the contents of this repo are irrelevant. We use the "plumbing" repository for things w.r.t to how work/function, the "plumbing" of our community/project. |
|
We had a similar conversation on the dotnet module: voxpupuli/puppet-dotnet#9 |
|
Oh man can fixing the PMT not be part of scope here? |
Completely agree. Part of a different discussion. This is part of "how do we decide when it's time to move and how do we go about it". |
|
@daenney I disagree that we should drop puppet3 support before enabling puppet4 support. I think we need to have branches. I think every module should have a table like this in the readme: If we hard cut modules to puppet4 at some arbitrary date, we'll just grow uncontrolled forks of people patching the last puppet3 version. If we keep a puppet3 branch around for a couple years, then people who haven't upgraded yet have a place to collaborate. Creating a puppet4 branch (or moving master to puppet4) enables people to start using puppet4 and will help drive puppet4 adoption. We should do that sooner rather than later. |
I never proposed just yoloing it on some arbitrary date. I set very specific data sources that I would like to poll to get a feel at which point we can make this switch, where we stop actively supporting Puppet 3, without hurting people and a period in which we should continue to maintain a compatible version:
Moving on.
That's a good idea though "a couple of years" feels like stretching this a bit too far. Especially since at some point all of us will likely have moved on and needing to keep reviewing and maintaining Puppet 3 compatible code will be a drainer. At least setting the expectation that PRs will likely take a long time to be reviewed and merged if you come up with a Puppet 3 fix should be made clear.
Not unreasonable. But then we need to iron out a few things like how do we deal with a potentially diverging feature set and the fact that some of our tools are currently half-functioning on Puppet 4 like puppet-lint and spec-puppet. We also can't just move "sooner rather than later" if the majority of our own contributors are still on Puppet 3, hence the suggested internal survey to figure out how far along we are. |
|
Sure, sorry to derail it @nibalizer. I totally think the stdlib model is a workable one - major versions to express breaking compatibility with older puppet major versions. Keep a branch per major in the main repo. For the Puppet projects, we put new features only in the latest major branch but backport bugfixes as long as it's not super invasive, but YMMV. The tooling ecosystem problem is a big one. Is there a hit-list of things needed for puppet-lint and rspec to fully work on Puppet 4? |
|
Nope, but that's a great idea. I've added a few now. |
|
What should we do for the forge? I think if we use branch we should also publish on the forge under different names? puppet-corosync and puppet-corosync4 ? |
|
😭 😭 😭 😭 😭 😭 😭 |
|
@roidelapluie Why not just have a major version bump which is then puppet 4 ready? like have version 1.x for puppet 3.x and 2.x for puppet 4.x ? |
|
Because users rely on the Forge API "current_release" ? is PMT able to retrieve the latest release that is available for your puppet version? |
|
That's going to cause major confusion. Is puppet-corosync4 version 4 of the module, compatible with version 4 of corosync, compatible with Puppet 4? And when we move to Puppet 5 which won't have a language transition like this, do we stay with 4, does it become 5 or do we decide to drop 3 and go back to the bare name? This should be fixed in the PMT, it should simply be possible to instruct it which version you want fetched just like r10k and librarian-puppet can and/or respect the Puppet version constraint. |
|
ok makes sense |
I think that's what I suggested last week :) |
|
@ahpook I just added two issues for you related to the PMT ;). |
|
So we're going to maintain two versions of puppet for our modules? That's a tall order. I would say we start with more puppet4 testing. For example, adding 4.3.1 to the test matrix in modulesync and making sure it at least runs. We can worry about porting all the syntactic sugar later. |
Where did you get that from? So far, these are proposals. No road has been taken or any decision made as to what we'll support, how or when.
That's already the case: https://github.com/voxpupuli/modulesync_config/blob/master/config_defaults.yml#L8-L18 |
|
We are approaching the EOL date for Puppet 3 now, just 4 months away. It appears that all of the outstanding issues are still outstanding. Are they all actually required to swing to P4 support only? I think the PMT ones specifically can be moved to a "Desired/Optional" section. Of the rest, I would like to address the puppet-lint tasks sometime between now and the end of the community summit/hack at PuppetConf and have put them all against the 2.1.0 milestone. The style guide has been receiving regular updates and I believe is future parser compatible - and the future of puppet-lint is to line its checks up with the guide. That leaves the rspec-puppet issue, which I cannot speak to. I do think that "once we move to Puppet 4 only the code will simply not work anymore on older versions of Puppet" is not fully accurate - we can support only Puppet 4 but much of the code will continue to work unsupported with Puppet 3, until actual changes are made that rely on P4 only constructs. Only the modules that are currently Puppet 4-only would not be backward compatible on Day 0 of the change. While that may (rapidly?) change, it eases the transition. For a timeframe, I suggest that we announce our planned drop of Puppet 3 support by November 1st via the website, twitter, and the mailing list. By November 30, we ensure that all modules have an up to date release supporting P3. Starting on January 1, we update modules to remove the support for P3 (metadata, travis, anything else?) and release new MAJOR versions when the next fix/feature is implemented. We support P3 only for emergency fixes on the older branches through 4/30/2017, one third of the calendar year. By announcing deadlines for ending P3 support and patching ahead of time, we can ensure that it's really difficult for anyone to be surprised by the change, and we do not need to rewrite all the modules top to bottom, either. |
From my experience, December and January are hectic enough with last-minute oh-no-vacation oh-shit-new-legislation god-damn-not-this-thing-too issues that we shouldn't compound that with Puppet module changes since it's so difficult for people to avoid upgrading across major version boundaries with the tools we currently have in our ecosystem (most notably PMT and nothing does any kind of Puppet compatibility checking even though the modules provide the necessary metadata). |
|
When do you think would be a better time to make the cut? |
|
I would suggest we keep your timeline with regards to adding the warnings etc. but only flip the bit in February so that everyone's had some time to catch their breath after new year. |
|
Revised proposal (in the one true date format):
|
|
@rnelson0 Consider moving the end date forward from 20170430 to 20170331 perhaps, otherwise sounds good to me. |
|
I created a couple puppet linters to migrate facts to a newer style for puppet 4
|
|
@mmckinst Nice! If they're not on our site yet, would you mind submitting a PR against voxpupuli/voxpupuli.github.io, w.r.t to https://github.com/voxpupuli/voxpupuli.github.io#puppet-plugins? Essentially updating this file https://github.com/voxpupuli/voxpupuli.github.io/blob/master/_data/tools/puppet-lint.json with your linters. |
useful to take advantage of the structured facts and `$facts` hash in puppet 4 but puppet 3 users can also make use of them if they've configured puppet correctly or are running a new version of facter * top_scope_facts - lints for facts like `$::operatingsystem` and replaces them with `$facts['operatingsystem']` . * legacy_facts - lints for so called legacy facts like `$facts['operatingsystemmajrelease']` or `$::operatingsystemmajrelease` and replaces them with the new fancy structured fact like `$facts['os']['release']['major']` ref voxpupuli/plumbing#21
useful to take advantage of the structured facts and `$facts` hash in puppet 4 but puppet 3 users can also make use of them if they've configured puppet correctly or are running a new version of facter * top_scope_facts - lints for facts like `$::operatingsystem` and replaces them with `$facts['operatingsystem']` . * legacy_facts - lints for so called legacy facts like `$facts['operatingsystemmajrelease']` or `$::operatingsystemmajrelease` and replaces them with the new fancy structured fact like `$facts['os']['release']['major']` ref voxpupuli/plumbing#21
|
I've engaged the PMC now that we have one to get the announcement out! @juniorsysadmin I specified April so there was a 90 day window between deprecation and end of support. Do you think 60 days is sufficient? (sorry I missed the comment back in August) |
Yes |
|
Good evening everybody. The PMC has created a proposal for the Puppet 4 deprecation. We would like to discuss the following with you and make a decision in the next days via lazy consensus. Deprecation of Puppet 3The End of Puppet 3Puppet 3 will be End of Life 2016-12-31. We see Vox Pupuli as a vendor of many software products. As a vendor we are responsible for the security of our users and should not encourage them to use outdated or end-of-life software. Therefor we will start to deprecate Puppet3 at 2017-01-01. Timeline
|
|
I think we previously noted that we don't want to force people to come back from a holiday vacation and have to deal with upgrading the world right away. If we could give people at least 15-30 days before we start the new, puppet3-less releases, it may make for a better user experience. Otherwise I do like the accelerated timelines if we get consensus. |
|
Whoa I don't think adding a warn() function to every module is in any way polite. Someone could easily get a dozen or more warnings every puppet run. |
|
To me, it seems like we only need to do one thing on 2017-01-01: Write a blog post that says "Puppet 3 is dead, the Puppet 4 floodgates are open, write whatever you want!" Then whenever a module gets puppet4 only code, the next release of it bumps the major version. At the same time the puppet4 code is about to merge, a 'puppet3' branch should be created, in case anyone needs the last functional puppet3 code. I don't think we need to release every module just to add warnings. I don't think we need to drop puppet3 from metadata.json unless puppet4 code was added. |
|
@nibalizer - would adding a suppress_puppet3_deprecation_warning parameter that defaults to false on all modules be more polite? That would enable a consistent way via hiera's APL to disable the warnings? |
There is a “warn_once” option for puppet deprecations that uses a symbol/string for the uniquness. Thus, if all vox populi modules uses the same warn_once symbol it is only shown once.
|
|
@nibalizer that's the way I chose to do the puppet3 deprecation for the NTP example module: 4.2.0 was the last release to support puppet 3, with the change to ntp v5.0 we "silently" dropped puppet 3 support and started using puppet 4 features. After puppet 3, the Long-term my goal is that any "last" release of a module in a major series (like the NTP v5.0.0 above) will provide a configuration that a) does not cause deprecation warnings and b) will work unmodified in the major+1 version (like the NTP v6.0.0 release). This should avoid the pain we had with e.g. apt, allow folks to discover the changes they need to make to their config, and update on their own pace. |
|
@Cinderhaze @hlindberg I don't think we need the deprecation warnings. I get warnings from software all the time that I can do nothing about. Who is this deprecation message for? Someone who is running puppet3 but plugged in enough to the outside world to update to latest of a voxpupuli module? That person should already know that puppet3 is EOL and doesn't need us warning them every time they run puppet. It is not a module's job to scold you for running an old puppet, I think. |
|
@DavidS exactly.
That can happen at a different time for every module. Or someone can go wild and do it to all the modules in the span of a couple weeks. But I think doing the other 4 steps without having puppet 4 code to merge in is just busywork. |
|
FWIW I read "We will add a warning" as adding a note to the readme in a published version. I definitely do not like the idea of a warning function adding output to runs. |
|
@DavidS What we do with puppet-corosync is we have a LTS version we will support until Puppet 3 vox pupuli depreciation, and newer versions which use puppet 4 features |
|
@roidelapluie technically the last puppet3 compatible ntp version is commercially supported by us, and I would expect to do hotfix releases for security issues, and other critical breakage, but e.g. no new platforms (puppet 3 wouldn't support them anyways), or new features (backporting is rewriting). |
|
https://voxpupuli.org/blog/2016/12/22/putting-down-puppet-3/ I ended up writing that and @bbriggs merged it. In hindsight, we should have put a nice 'need decision by X' on this so that we'd have our plan in place and people could plan around it. |
RFC: Support Puppet 4 only
This is going to be a bit controversial because I'm raising it this early. This is not a proposal to move to Puppet 4 only now now now. Instead it is meant for us to find a good way on how to move to Puppet 4 and define some metrics at which point we deem it acceptable to move to a Puppet 4 only version of our modules.
Reasons for moving to Puppet 4
Puppet 4 brings a vast number of improvements but besides the runtime speedup it's mostly language features and sanitisation that we, the developers, can benefit from allowing us to write more concise, bug-free, testable and compact manifests. This in turn raises the quality of our modules which is good for everyone that uses them. Some of the features deemed most important:
validate_*andis_*functions) which gets us decent error reporting for freecreate_resourcestoo)$factshash (instead of all the$::facts) which will facilitate migrating to astrict_variablesenabled futureAdditionally a new way to deal with data in modules is starting to emerge in Puppet 4 that in due time we'll be able to take advantage of too (though perhaps not before Puppet 5 depending on when the feature is deemed finalised).
Collecting data
The most important thing to do in order to decide when the time is right is to collect as much data as we can on Puppet 4 deployments and their ratio to Puppet 3.
Data that could help us make this decision:
When to move
The problem with all of this is once we move to Puppet 4 only the code will simply not work anymore on older versions of Puppet, most notably Puppet 3 which is still widely deployed. Though Puppet 4 has been out for a year now and multiple community members are holding talks at PuppetConf, Puppet Camp's and Configuration Management Camp on how to migrate to Puppet 4 and what new features it brings adoption is still slow. Puppet Labs has successfully been pushing Puppet 4 through PE upgrades.
As such we need to define a threshold at which point, based on the data that we decide to collect, we believe the time is right to move to Puppet 4. However, when we do decide to do so we should still provide critical bug fix releases for Puppet 3 compatible versions of our modules for a number of months to not unnecessarily complicate other people's lives.
Therefor I would propose a transition somewhere along these lines:
metadata.jsonThe one question that remains: how do we decide it is time?
List of tasks / bugs that we need to do/fix prior to being able to move
puppet module tool
metadata.jsonrspec-puppet
puppet-lint
The text was updated successfully, but these errors were encountered: