Fixes #19227 - katello-change-hostname health check

[johnpmitsch]

This will check services before running the hostname change.
A hammer ping doesn't return a failing exit code if a
service is failing, so this makes use of grepping FAIL

[theforeman-bot]

@johnpmitsch, the Redmine ticket used is for a different project than the one associated with this GitHub repository. Please either:

• check #19227: katello-change-hostname should check services before running is the correct one
• move ticket #19227 from Packaging to the Katello project
• or file a new ticket in the Katello project

If changing the ticket number used, remember to update the PR title and the commit message (using git commit --amend).

---

This message was auto-generated by Foreman's prprocessor

[johnpmitsch]

Failing hammer ping status - script terminates

[root@john ~]# ./katello-change-hostname -u admin -p changeme -d hey.example.com
This script will modify your system. You will need to re-register any Foreman Proxies and Katello clients after script completion. Foreman Proxies will have to be reinstalled. If you are using custom certificates, you will have to run the foreman-installer again with custom certificate options after this script completes. Have you taken the necessary precautions (backups, snapshots, etc...) and want to proceed with changing your hostname? 
 [y/n]:
true
All services are not currently running, please check 'hammer ping'
[root@john ~]# hammer ping
candlepin:      
    Status:          ok
    Server Response: Duration: 23ms
candlepin_auth: 
    Status:          ok
    Server Response: Duration: 23ms
pulp:           
    Status:          FAIL
    Server Response:
foreman_tasks:  
    Status:          ok
    Server Response: Duration: 488ms

Successful hammer ping status - script completes successfully

[root@john ~]# ./katello-change-hostname -u admin -p changeme -d hey.example.com
...
...
...
  Success!
  * Katello is running at https://hey.example.com
  * To install an additional Foreman proxy on separate machine continue by running:

      foreman-proxy-certs-generate --foreman-proxy-fqdn "$FOREMAN_PROXY" --certs-tar "~/$FOREMAN_PROXY-certs.tar"

  The full log is at /var/log/foreman-installer/katello.log
[ INFO 2017-04-12 20:32:57 verbose] pulp.conf is already present, skipping
[ INFO 2017-04-12 20:32:57 verbose] All hooks in group post finished
Regenerating ueber certs
rake aborted!
Don't know how to build task 'katello:regenerate_ueber_certs' (error from running on 3.3 which doesn't have this task)

(See full trace by running task with --trace)
Restarting puppet services
Redirecting to /bin/systemctl restart  puppet.service
Redirecting to /bin/systemctl restart  puppetserver.service

  Hostname change complete!
  
  You will need to re-register any Foreman Proxies or clients with the server. 
  
  When re-registering clients, you will have to reinstall the bootstrap RPM.
  Any mention of the hey.example.com on your clients will have to change to hey.example.com (i.e. /etc/hosts)
  
  For Foreman Proxies, you will need to regenerate the certs on the Katello server and reinstall the Foreman Proxy
  
  If you want to use custom certificates, re-run the foreman-installer with custom certificate options

  Short hostnames have not been updated, please update those manually.
  
[root@john ~]# 
[root@john ~]# hammer ping
candlepin:      
    Status:          ok
    Server Response: Duration: 238ms
candlepin_auth: 
    Status:          ok
    Server Response: Duration: 42ms
pulp:           
    Status:          ok
    Server Response: Duration: 70ms
foreman_tasks:  
    Status:          ok
    Server Response: Duration: 42ms

[akofink]

Looks good, just one small comment.

[akofink]

what about the case when hammer ping returns overall failure? To test this, stop your katello server. hammer_ping_failure is false in that case.

[johnpmitsch]

@akofink I updated to check the return code as well, this will be != 0 if the "status" of hammer ping is failing.

Side note -- shouldn't hammer ping display the overall status as well?

[akofink]

It does display an overall status in some cases, but if the server is down, no response is sent.

$ hammer ping
Error: 503 Service Unavailable

[akofink]

LGTM! ACK!