Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

(BOLT-702) Install puppet agent on osx #307

Merged
merged 3 commits into from Aug 7, 2018
Merged

Conversation

donoghuc
Copy link
Member

@donoghuc donoghuc commented Aug 1, 2018

Support puppet-agent install on osx 11,12,13. Logic is based on https://github.com/puppetlabs/puppetlabs-pe_repo/blob/hoyt/templates/osx.bash.erb#L5

tasks/shell.sh Outdated
fi

# Random function since not all shells have $RANDOM
if exists hexdump; then
random_number=random_hexdump
random_number=$(random_hexdump)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was adding the string "random_hexdump" to the path instead of a random number.

tasks/shell.sh Outdated
@@ -250,12 +249,17 @@ random_hexdump () {
if test "x$TMPDIR" = "x"; then
tmp="/tmp"
else
tmp=$TMPDIR
tmp=${TMPDIR}
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For Vmpooler the TMPDIR environment variable had a trailing /.

@donoghuc
Copy link
Member Author

donoghuc commented Aug 1, 2018

I have tested on Vmpooler osx-1011-x86_64, osx-1012-x86_64, osx-1013-x86_64 with both latest and manually specified version. Added WIP label until I can figure out automated tests.

Also, it appears do_checksum is never used? Are we leaving that in for an improvement later?

do_checksum() {

@puppetcla
Copy link

CLA signed by all contributors.

@MikaelSmith

This comment has been minimized.

@donoghuc donoghuc force-pushed the BOLT-702 branch 4 times, most recently from 50886c8 to f27eafd Compare August 2, 2018 19:58
@donoghuc
Copy link
Member Author

donoghuc commented Aug 2, 2018

Added acceptance tests targets for vmpooler. They can be run "manually" with proper beaker credentials. Setup
bundle install
Point beaker to ssh key nodeset yaml

#osx1-12-64.yml
 ---
HOSTS:
  osx1013-64-1:
    hypervisor: vmpooler
    platform: osx-10.13-x86_64
    packaging_platform: osx-10.13-x86_64
    template: osx-1013-x86_64
    roles:
    - target
CONFIG:
  nfs_server: none
  consoleport: 443
  pooling_api: http://vmpooler.delivery.puppetlabs.net/
  keyfile: ~/.ssh/id_rsa-acceptance

Example invocation

puppetlabs-puppet_agent/task_spec$ GEM_BOLT=true BEAKER_debug=true BEAKER_set=osx1012-64 bundle exec rake task_acceptance

@@ -147,9 +159,10 @@ def bolt_on(target, type, action, object,
File.open(inventory_path, 'w') {|fh| fh.write(inventory.to_json)}
File.open(config_path, 'w') {|fh| fh.write("---\n")}
command += ['--inventoryfile', inventory_path]
command += ['--configfile', inventory_path]
command += ['--configfile', config_path]
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Was this set to inventory_path on purpose?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nope

keys = host.connection.instance_variable_get(:@ssh).options[:keys]
key = keys.first if keys
config[:ssh][:"private-key"] = key if key
node_name = host.hostname
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should probably still build node_name = "ssh://#{host.host_hash[:ip]}?n=#{host.hostname}"

@@ -50,8 +51,20 @@ def add_node(node, group_name, groups)
config: config
}
else
# This is very unlikely to work
node_name, node = host.hostname
# host_hash is differnet depending on hypervisor (docker vs vmpooler)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we decide how to process the host_hash with a conditional on the hypervisor directly

@@ -147,9 +159,10 @@ def bolt_on(target, type, action, object,
File.open(inventory_path, 'w') {|fh| fh.write(inventory.to_json)}
File.open(config_path, 'w') {|fh| fh.write("---\n")}
command += ['--inventoryfile', inventory_path]
command += ['--configfile', inventory_path]
command += ['--configfile', config_path]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nope

"10.6") platform_version="10.6" ;;
"10.7"|"10.8"|"10.9") platform_version="10.7" ;;
*) echo "No builds for platform: $major_version"
"10.6"|"10.7"|"10.8"|"10.9"|"10.10") echo "No builds for platform: $major_version"
report_bug
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can probably leave this out

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Leave out report_bug? Or the version check in general?

@@ -409,8 +413,16 @@ do_download() {
unable_to_retrieve_package
}

latest_version() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we name this function to make it clear it's for osx

@MikaelSmith
Copy link
Contributor

Updated.

@puppetcla
Copy link

CLA signed by all contributors.

Support puppet-agent install on macOS 11, 12, and 13. Logic is based on
the PE install script.
@MikaelSmith
Copy link
Contributor

Fixed.

@donoghuc
Copy link
Member Author

donoghuc commented Aug 6, 2018

Are we planning on utilizing the do_checksum function in install_shell in the future? I do not see it currently being used.

@nicklewis
Copy link
Contributor

I get a bunch of extraneous non-JSON output. Is that expected?

  11:25:15 -0700 INFO: Version parameter not defined, assuming latest
  11:25:15 -0700 INFO: Downloading Puppet latest for mac_os_x...
  11:25:15 -0700 INFO: OSX platform! Lets get you a DMG...
  11:25:15 -0700 INFO: Downloading http://downloads.puppetlabs.com/mac/puppet5/10.13/x86_64/puppet-agent-latest.dmg
  11:25:15 -0700 INFO:   to file /var/folders/zz/zyxvpxvq6csfxvn_n0000000000000/T/install.sh.4417.10014/puppet-agent-latest.dmg
  11:25:15 -0700 INFO: Trying curl...
  11:25:19 -0700 INFO: installing puppetlabs dmg with hdiutil and installer
  Checksumming Protective Master Boot Record (MBR : 0)…
  Protective Master Boot Record (MBR :: verified   CRC32 $51761946
  Checksumming GPT Header (Primary GPT Header : 1)…
   GPT Header (Primary GPT Header : 1): verified   CRC32 $70C93C6C
  Checksumming GPT Partition Data (Primary GPT Table : 2)…
  GPT Partition Data (Primary GPT Tabl: verified   CRC32 $907C9421
  Checksumming  (Apple_Free : 3)…
                      (Apple_Free : 3): verified   CRC32 $00000000
  Checksumming disk image (Apple_HFS : 4)…
            disk image (Apple_HFS : 4): verified   CRC32 $774733E6
  Checksumming  (Apple_Free : 5)…
                      (Apple_Free : 5): verified   CRC32 $00000000
  Checksumming GPT Partition Data (Backup GPT Table : 6)…
  GPT Partition Data (Backup GPT Table: verified   CRC32 $907C9421
  Checksumming GPT Header (Backup GPT Header : 7)…
    GPT Header (Backup GPT Header : 7): verified   CRC32 $84262376
  verified   CRC32 $8EB0713A
  /dev/disk1          	GUID_partition_scheme          	
  /dev/disk1s1        	Apple_HFS                      	/private/var/folders/zz/zyxvpxvq6csfxvn_n0000000000000/T/40998.F5jP8S0H
  installer: Package name is puppet-agent
  installer: Installing at base path /
  installer: The install was successful.
  "disk1" unmounted.
  "disk1" ejected.
  {
  }

@donoghuc
Copy link
Member Author

donoghuc commented Aug 6, 2018

@nicklewis I think showing the output is expected. This mimics the behavior of the task output when run against a linux target.

Started on gnk7ytzajqnc458.delivery.puppetlabs.net...
Finished on gnk7ytzajqnc458.delivery.puppetlabs.net:
  11:57:11 -0700 INFO: Version parameter not defined, assuming latest
  11:57:11 -0700 INFO: Downloading Puppet latest for ubuntu...
  11:57:11 -0700 INFO: Ubuntu platform! Lets get you a DEB...
  11:57:11 -0700 INFO: Downloading http://apt.puppetlabs.com/puppet5-release-xenial.deb
  11:57:11 -0700 INFO:   to file /tmp/install.sh.1346.random_hexdump/puppet5-release-xenial.deb
  11:57:11 -0700 INFO: Trying wget...
  11:57:11 -0700 INFO: installing puppetlabs apt repo with dpkg...
  Selecting previously unselected package puppet5-release.
  (Reading database ... 63617 files and directories currently installed.)
  Preparing to unpack .../puppet5-release-xenial.deb ...
  Unpacking puppet5-release (5.0.0-1xenial) ...
  Setting up puppet5-release (5.0.0-1xenial) ...
  Ign:1 http://apt.puppetlabs.com xenial InRelease
  Get:2 http://apt.puppetlabs.com xenial Release [57.5 kB]
  Get:3 http://apt.puppetlabs.com xenial Release.gpg [819 B]
  Get:4 http://apt.puppetlabs.com xenial/puppet5 amd64 Packages [21.1 kB]
  Get:5 http://apt.puppetlabs.com xenial/puppet5 i386 Packages [18.4 kB]
  Get:6 http://apt.puppetlabs.com xenial/puppet5 all Packages [8,439 B]
  Get:7 https://artifactory.delivery.puppetlabs.net/artifactory/ubuntu__remote xenial-security InRelease [107 kB]
  Get:8 https://artifactory.delivery.puppetlabs.net/artifactory/ubuntu__remote xenial-updates InRelease [109 kB]
  Hit:9 https://artifactory.delivery.puppetlabs.net/artifactory/ubuntu__remote xenial InRelease
  Get:10 https://artifactory.delivery.puppetlabs.net/artifactory/ubuntu__remote xenial-security/main Sources [131 kB]
  Get:11 https://artifactory.delivery.puppetlabs.net/artifactory/ubuntu__remote xenial-security/restricted Sources [2,116 B]
  Get:12 https://artifactory.delivery.puppetlabs.net/artifactory/ubuntu__remote xenial-security/universe Sources [69.1 kB]
  Get:13 https://artifactory.delivery.puppetlabs.net/artifactory/ubuntu__remote xenial-security/multiverse Sources [2,088 B]
  Get:14 https://artifactory.delivery.puppetlabs.net/artifactory/ubuntu__remote xenial-security/main amd64 Packages [531 kB]
  Get:15 https://artifactory.delivery.puppetlabs.net/artifactory/ubuntu__remote xenial-security/main i386 Packages [467 kB]
  Get:16 https://artifactory.delivery.puppetlabs.net/artifactory/ubuntu__remote xenial-security/main Translation-en [227 kB]
  Get:17 https://artifactory.delivery.puppetlabs.net/artifactory/ubuntu__remote xenial-security/restricted amd64 Packages [7,204 B]
  Get:18 https://artifactory.delivery.puppetlabs.net/artifactory/ubuntu__remote xenial-security/restricted i386 Packages [7,224 B]
  Get:19 https://artifactory.delivery.puppetlabs.net/artifactory/ubuntu__remote xenial-security/restricted Translation-en [2,152 B]
  Get:20 https://artifactory.delivery.puppetlabs.net/artifactory/ubuntu__remote xenial-security/universe amd64 Packages [362 kB]
  Get:21 https://artifactory.delivery.puppetlabs.net/artifactory/ubuntu__remote xenial-security/universe i386 Packages [308 kB]
  Get:22 https://artifactory.delivery.puppetlabs.net/artifactory/ubuntu__remote xenial-security/universe Translation-en [136 kB]
  Get:23 https://artifactory.delivery.puppetlabs.net/artifactory/ubuntu__remote xenial-security/multiverse amd64 Packages [3,456 B]
  Get:24 https://artifactory.delivery.puppetlabs.net/artifactory/ubuntu__remote xenial-security/multiverse i386 Packages [3,628 B]
  Get:25 https://artifactory.delivery.puppetlabs.net/artifactory/ubuntu__remote xenial-security/multiverse Translation-en [1,744 B]
  Get:26 https://artifactory.delivery.puppetlabs.net/artifactory/ubuntu__remote xenial-updates/main Sources [317 kB]
  Get:27 https://artifactory.delivery.puppetlabs.net/artifactory/ubuntu__remote xenial-updates/restricted Sources [2,528 B]
  Get:28 https://artifactory.delivery.puppetlabs.net/artifactory/ubuntu__remote xenial-updates/universe Sources [217 kB]
  Get:29 https://artifactory.delivery.puppetlabs.net/artifactory/ubuntu__remote xenial-updates/multiverse Sources [8,408 B]
  Get:30 https://artifactory.delivery.puppetlabs.net/artifactory/ubuntu__remote xenial-updates/main amd64 Packages [821 kB]
  Get:31 https://artifactory.delivery.puppetlabs.net/artifactory/ubuntu__remote xenial-updates/main i386 Packages [749 kB]
  Get:32 https://artifactory.delivery.puppetlabs.net/artifactory/ubuntu__remote xenial-updates/main Translation-en [338 kB]
  Get:33 https://artifactory.delivery.puppetlabs.net/artifactory/ubuntu__remote xenial-updates/restricted amd64 Packages [7,556 B]
  Get:34 https://artifactory.delivery.puppetlabs.net/artifactory/ubuntu__remote xenial-updates/restricted i386 Packages [7,524 B]
  Get:35 https://artifactory.delivery.puppetlabs.net/artifactory/ubuntu__remote xenial-updates/restricted Translation-en [2,272 B]
  Get:36 https://artifactory.delivery.puppetlabs.net/artifactory/ubuntu__remote xenial-updates/universe amd64 Packages [676 kB]
  Get:37 https://artifactory.delivery.puppetlabs.net/artifactory/ubuntu__remote xenial-updates/universe i386 Packages [617 kB]
  Get:38 https://artifactory.delivery.puppetlabs.net/artifactory/ubuntu__remote xenial-updates/universe Translation-en [272 kB]
  Get:39 https://artifactory.delivery.puppetlabs.net/artifactory/ubuntu__remote xenial-updates/multiverse amd64 Packages [16.4 kB]
  Get:40 https://artifactory.delivery.puppetlabs.net/artifactory/ubuntu__remote xenial-updates/multiverse i386 Packages [15.5 kB]
  Get:41 https://artifactory.delivery.puppetlabs.net/artifactory/ubuntu__remote xenial-updates/multiverse Translation-en [8,344 B]
  Fetched 6,657 kB in 2s (3,017 kB/s)
  Reading package lists...
  Reading package lists...
  Building dependency tree...
  Reading state information...
  The following NEW packages will be installed:
    puppet-agent
  0 upgraded, 1 newly installed, 0 to remove and 150 not upgraded.
  Need to get 16.9 MB of archives.
  After this operation, 95.9 MB of additional disk space will be used.
  Get:1 http://apt.puppetlabs.com xenial/puppet5 amd64 puppet-agent amd64 5.5.4-1xenial [16.9 MB]
  Fetched 16.9 MB in 0s (43.0 MB/s)
  Selecting previously unselected package puppet-agent.
(Reading database ... 63622 files and directories currently installed.)
  Preparing to unpack .../puppet-agent_5.5.4-1xenial_amd64.deb ...
  Unpacking puppet-agent (5.5.4-1xenial) ...
  Processing triggers for libc-bin (2.23-0ubuntu9) ...
  Setting up puppet-agent (5.5.4-1xenial) ...
  Created symlink from /etc/systemd/system/multi-user.target.wants/puppet.service to /lib/systemd/system/puppet.service.
  Created symlink from /etc/systemd/system/multi-user.target.wants/mcollective.service to /lib/systemd/system/mcollective.service.
  Created symlink from /etc/systemd/system/multi-user.target.wants/pxp-agent.service to /lib/systemd/system/pxp-agent.service.
  Removed symlink /etc/systemd/system/multi-user.target.wants/pxp-agent.service.
  Processing triggers for libc-bin (2.23-0ubuntu9) ...

@MikaelSmith
Copy link
Contributor

I'll go ahead and remove do_checksum.

@adreyer
Copy link
Contributor

adreyer commented Aug 6, 2018

@nicklewis it's not ideal but I think it will take a new ticket to make this capture output and generate json robustly.

if exists sha256sum; then
checksum=`sha256sum $1 | awk '{ print $1 }'`
if test "x$checksum" != "x$2"; then
checksum_mismatch
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove checksum_mismatch as well.

checksum_mismatch() {
critical "Package checksum mismatch!"
report_bug
exit 1
}

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

Previously any Beaker version 3+ was supported. However Beaker 4 was
just released and removes many previously integrated gems. Stick to
Beaker 3 until we can figure out which ones explicitly need to be pulled
in.
Remove unused shell functions do_checksum and checksum_mismatch from the
install task.
@adreyer adreyer merged commit d45d390 into puppetlabs:master Aug 7, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants