Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

(PUP-7095) Enable portage install and uninstall options for the package provider #5498

Merged

Conversation

@prometheanfire
Copy link
Contributor

@prometheanfire prometheanfire commented Jan 12, 2017

Allow the portage package provider to accept install and uninstall
options. This is useful if you do not want to use the default global
EMERGE_DEFAULT_OPTS paramaters setable in make.conf.

Allow the portage package provider to purge package if requested.

Allow package sets (@world @System and the like)

@prometheanfire
Copy link
Contributor Author

@prometheanfire prometheanfire commented Jan 12, 2017

my ruby is a bit rusty, but I believe I generate the correct install and uninstall commands, not sure if anything needed to be passed differently.

Loading

@prometheanfire prometheanfire force-pushed the enable_portage_options branch 2 times, most recently from f91fc80 to 1b9264a Jan 12, 2017
@puppetcla
Copy link

@puppetcla puppetcla commented Jan 12, 2017

CLA signed by all contributors.

Loading

@prometheanfire prometheanfire force-pushed the enable_portage_options branch 10 times, most recently from d23b342 to d5dd81c Jan 13, 2017
@prometheanfire prometheanfire force-pushed the enable_portage_options branch 5 times, most recently from d1b2c89 to 43272a8 Mar 19, 2017
@prometheanfire
Copy link
Contributor Author

@prometheanfire prometheanfire commented Mar 19, 2017

Ok, this should be done and good

Loading

@prometheanfire prometheanfire force-pushed the enable_portage_options branch 10 times, most recently from 7527752 to 498d4dc Mar 19, 2017
@prometheanfire prometheanfire force-pushed the enable_portage_options branch 2 times, most recently from 1463aa0 to 677f871 Mar 19, 2017
@MikaelSmith MikaelSmith changed the title Enable portage install and uninstall options for the package provider (PUP-7095) Enable portage install and uninstall options for the package provider Mar 20, 2017
has_features :versionable, :reinstallable
This provider supports the `install_options` and `uninstall_options` attributes, which allows command-line
flags to be passed to apt-get. These options should be specified as a string (e.g. '--flag'), a hash
Copy link
Contributor

@adrienthebo adrienthebo Mar 20, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/apt-get/emerge/

Loading

Copy link
Contributor Author

@prometheanfire prometheanfire Mar 20, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

Loading

@prometheanfire prometheanfire force-pushed the enable_portage_options branch from 677f871 to 3462fd4 Mar 20, 2017
@prometheanfire
Copy link
Contributor Author

@prometheanfire prometheanfire commented Mar 20, 2017

I just tested this on a test server and now ensure => latest works, before it didn't, no bugs so far.

I've also one-off tested uninstalling and installing based on package version and slot. Install options work (I passed it --update on a ensure => latest package and it called emerge --update --update as it should have. Not sure what else to test atm.

Loading

@prometheanfire
Copy link
Contributor Author

@prometheanfire prometheanfire commented Mar 20, 2017

My puppet runs went to ~54 seconds from ~8 seconds though. Looking into it now. My guess is all the qatom calls (which call out to binaries on the system) really added up more than I thought.

Loading

@prometheanfire
Copy link
Contributor Author

@prometheanfire prometheanfire commented Mar 21, 2017

it's the emerge --list-sets it's running every package run.

setting that statically got my run back down to 9 seconds

Loading

@prometheanfire
Copy link
Contributor Author

@prometheanfire prometheanfire commented Mar 21, 2017

so, how do I store the output of emerge --list-sets between package runs?

Loading

@prometheanfire prometheanfire force-pushed the enable_portage_options branch 2 times, most recently from 3150728 to 6dd62f3 Mar 21, 2017
@prometheanfire
Copy link
Contributor Author

@prometheanfire prometheanfire commented Mar 21, 2017

fixed the runtime with dominic's help on irc. Old runtime was ~8 seconds, new runtime is ~5.4 seconds based on installing ~32 packages and a bunch of other services / files.

Loading

@prometheanfire prometheanfire force-pushed the enable_portage_options branch from 6dd62f3 to b03bd72 Mar 21, 2017
@@ -119,7 +205,7 @@

result = @slotted_provider.query
expect(result[:name]).to eq("ruby")
expect(result[:ensure]).to eq("1.9.2")
expect(result[:version_available]).to eq("1.9.3")
expect(result[:ensure]).to eq(:absent)
Copy link
Member

@MikaelSmith MikaelSmith Mar 21, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this output different now?

Loading

Copy link
Contributor Author

@prometheanfire prometheanfire Mar 22, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixing soon, just verifying

version_for_slot = versions_and_slots.find { |version_and_slot| version_and_slot.last == slot}
becomes
version_for_slot = versions_and_slots.find { |version_and_slot| version_and_slot.last == slot[1..-1] }

because slot has a colon in front of it :2.1 and only in this function is that not expected. should be 2.1

Loading

@@ -119,7 +205,7 @@

result = @slotted_provider.query
expect(result[:name]).to eq("ruby")
expect(result[:ensure]).to eq("1.9.2")
expect(result[:version_available]).to eq("1.9.3")
expect(result[:ensure]).to eq(:absent)
Copy link
Contributor Author

@prometheanfire prometheanfire Mar 22, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixing soon, just verifying

version_for_slot = versions_and_slots.find { |version_and_slot| version_and_slot.last == slot}
becomes
version_for_slot = versions_and_slots.find { |version_and_slot| version_and_slot.last == slot[1..-1] }

because slot has a colon in front of it :2.1 and only in this function is that not expected. should be 2.1

Loading

versions_and_slots.map! { |version_and_slot| version_and_slot.split(':') }
# [2.7.12: 2.7
# 3.4.5: 3.4
# 3.5.2: 3.5]
version_for_slot = versions_and_slots.find { |version_and_slot| version_and_slot.last == slot }
Copy link
Contributor Author

@prometheanfire prometheanfire Mar 22, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This needed the slot fixed to not be :2.1 or whatever

Loading

package[:ensure] = eix_get_version_for_versions(package[:installed_versions], qatom[:pv])
elsif qatom[:slot]
package[:version_available] = eix_get_version_for_slot(package[:slot_versions_available], qatom[:slot])
package[:ensure] = eix_get_version_for_slot(package[:installed_slots], qatom[:slot])
Copy link
Contributor Author

@prometheanfire prometheanfire Mar 22, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ensure and version_available should be reversed. version_available means 'what's currently installed'. ensure is what version to try and install.

Loading

@match_result = "app-misc sl [] [] [] [] http://www.tkl.iis.u-tokyo.ac.jp/~toyoda/index_e.html http://www.izumix.org.uk/sl/ sophisticated graphical program which corrects your miss typing\n"
@slot_match_result = "dev-lang ruby [2.0.0] [2.1.0] [1.8.7:1.8,1.9.2:1.9,2.0.0:2.0] [1.9.3:1.9,2.0.1:2.0,2.1.0:2.1] http://www.ruby-lang.org/ An object-oriented scripting language\n"
@match_result = "app-misc sl [] [5.02] [] [] [5.02] [5.02:0] http://www.tkl.iis.u-tokyo.ac.jp/~toyoda/index_e.html https://github.com/mtoyoda/sl/ sophisticated graphical program which corrects your miss typing\n"
@slot_match_result = "dev-lang ruby [2.1.9] [2.1.9] [2.1.9:2.1] [2.1.9] [2.1.9,,,,,,,] [2.1.9:2.1] http://www.ruby-lang.org/ An object-oriented scripting language\n"
Copy link
Contributor Author

@prometheanfire prometheanfire Mar 22, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

needed to alter the slot_match result so that it's upgradable, like before.

Loading

end

def self.eix_result_fields
[:category, :name, :ensure, :version_available, :installed_slots, :slot_versions_available, :vendor, :description]
# ensure:[3.4.5], version_available:[3.5.2], installed_slots:[2.7.12:2.7,3.4.5:3.4], installable_versions:[2.7.10-r1,2.7.12,3.4.3-r1,3.4.5,3.5.2] slot_versions_available:[2.7.12:2.7,3.4.5:3.4,3.5.2:3.5]
[:category, :name, :ensure, :version_available, :installed_slots, :installed_versions, :installable_versions, :slot_versions_available, :vendor, :description]
Copy link
Contributor Author

@prometheanfire prometheanfire Mar 22, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

bestversion should be what we want to ensure, not version_available (just need to switch them here too)

Loading

@prometheanfire prometheanfire force-pushed the enable_portage_options branch from b03bd72 to 018afba Mar 22, 2017
@prometheanfire
Copy link
Contributor Author

@prometheanfire prometheanfire commented Mar 22, 2017

@MikaelSmith thanks for pointing it out, fixed :D

Loading

@prometheanfire prometheanfire force-pushed the enable_portage_options branch from 018afba to 46f21c7 Mar 22, 2017
Allow the portage package provider to accept install and uninstall
options.  This is useful if you do not want to use the default global
EMERGE_DEFAULT_OPTS paramaters setable in make.conf.

Allow the portage package provider to purge package if requested.

Allow package sets (@world @System and the like)

Use ||= where possible and class variables when appropriate to reduce
runtime.  Old runtime was ~8.1 seconds, new runtime is ~5.4 seconds for
32 packages (and a bunch of other things).
@prometheanfire prometheanfire force-pushed the enable_portage_options branch from 46f21c7 to 26b064b Mar 22, 2017
@MikaelSmith MikaelSmith merged commit 43e2c93 into puppetlabs:master Mar 22, 2017
2 checks passed
Loading
@prometheanfire prometheanfire deleted the enable_portage_options branch Mar 22, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
4 participants