Installer review (allow to run katello-configure multiple times)

[lzap]

We have been working hard on katello-configure to improve the way it works. Since the installer is Puppet based, our main goal was to allow users to run it multiple times. Like Puppet does, installer do check everything and correct all deviations from what you configured using parameters. This was not possible since some dependency issues and usage of idempotent scripts like cpsetup.

Lots of changes were incorporated - various workarounds were removed and installer was tested with various scenarios. From now on, if you provision a clean installation or if you have existing one, you can always run katello-configure to make sure everything is okay. Please note all the katello-configure options are stored in the /etc/katello/katello-configure.conf so you can use it as an answer file:

katello-configure --answer-file=/etc/katello/katello-configure.conf

You can also provide different parameters and installer will re-configure everything. Please check logs carefully when changing options and report errors if you encouter any. Also note that changing org-name, user-name, user-pass and user-email will require resetting database.

And this also means you can easily switch between katello and headpin. Yes, this is possible and as easy as installing thumbslug plus:

katello-configure --deployment=headpin
katello ping
katello-configure --deployment=katello
katello ping

There is a new option --reset-data which drops all databases: katello, candlepin, pulp (also repos which were generated), elasticsearch and then starts initial configuration again. Since this is dangerous option, you need to provide case-sensitive "YES" to get the desired behavior.

Additional second option --reset-cache just removes all pulp downloaded RPMs from /var/lib/pulp/packages. If you provide them both, katello instance should be completly "clean". Please note all --reset-* options are not stored in the answer file for obvious reasons.

Example of full reset of a katello installation:

katello-configure --answer-file=/etc/katello/katello-configure.conf --reset-data=YES --reset-cache=YES

Basically this does the very same as katello-reset-dbs, but also makes sure all configuration values are correct. We will keep katello-reset-dbs script in git since it can be easily used for development setups.

Katello-configure will also work for upgrades. To upgrade existing installation, use katello-upgrade script first which will do all data migrations which are needed and THEN you can run katello-configure as many times as you want to re-configure stuff, because it always should describe correct state of an installation. It is not recommended to run katello-configure BEFORE katello-upgrade.

IMPORTANT: Since there are big changes in the installer, you need to do clean installation for the first time. If you try to upgrade to the new version of the installer, it will likely fail with a puppet error.

Please report all katello-configure failures along with katello-debug output. It creates nice tarball with all the important logs there (passwords are removed). We recommend to attach it as private comments into bugzilla. Before you report, please try to restart all backend engines and then katello to see if it helps and provide this information.

There are also two more options -b and -d. The first one turns off progress bars logging puppet also on the standard output and the second enables debug messages to be avaiable also on stdout (hidden by default). New version also strips out annoying color codes in the katello-configure log files.

As a side effect, I created a page describing installation of katello via puppet itself (without katello-configure script which only collects params and runs puppet): https://fedorahosted.org/katello/wiki/InstallViaPuppet

ACKS: @mbacovsky @mmccune

Do not merge this request as I would like to retab some files before merging. Thanks.

[xsuchy]

looks very good to me

[lzap]

@mccun934 If you could take a look on the concept - see the long description. Feel free to review puppet as well ;-)

[mccun934]

I'll try it out and let you know how it goes ..

[lzap]

Added one RHBZ fix because I would get merging issues otherwise.

[iNecas]

/me is checking with puppet master

[lzap]

I finished with testing katello-configure on Fedoras. I have implemented
new script called service-wait which corrects some bad behavior of
tomcat and postgresql services. Now it also works with systemd.

To test it, you need to build katello-* packages (src/ and puppet/ spec
files) and install it on a clean machine. Then you can test various
options, re-running k-c, changing options and resetting databases.
Please comment: #345

[lzap]

Few more fixes, PLEASE TEST so I can merge :-)

[mccun934]

first round of testing:

$ katello-configure
Starting Katello configuration
The top-level log file is [/var/log/katello/katello-configure-20120726-141414/main.log]
Populating Katello database schema

Failed, please check [/var/log/katello/katello-configure/db_migrate.log]
Report errors using # katello-debug tool.
$ cat /var/log/katello/katello-configure/db_migrate.log
Could not find gem 'thin (>= 1.2.11, runtime)' in any of the gem sources listed in your Gemfile.
Try running bundle install.
$ rpm -q rubygem-thin
rubygem-thin-1.2.8-4.el6.x86_64

more of a repo issue, than an issue with katello-configure. will continue with an updated rpm

[mccun934]

followup error

$ katello-configure
Starting Katello configuration
The top-level log file is [/var/log/katello/katello-configure-20120726-144653/main.log]
Populating Katello database schema

Failed, please check [/var/log/katello/katello-configure/db_migrate.log]
Report errors using # katello-debug tool.

$ cat /var/log/katello/katello-configure/db_migrate.log
rake aborted!
uninitialized constant Rake::DSL
/usr/share/katello/Rakefile:7
/usr/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:2382:in load' /usr/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:2382:inraw_load_rakefile'
/usr/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:2016:in load_rakefile' /usr/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:2067:instandard_exception_handling'
/usr/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:2015:in load_rakefile' /usr/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:1999:inrun'
/usr/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:2067:in standard_exception_handling' /usr/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:1997:inrun'
/usr/lib/ruby/gems/1.8/gems/rake-0.8.7/bin/rake:31
/usr/bin/rake:19:in `load'
/usr/bin/rake:19
(in /usr/share/katello)

[xsuchy]

Mike all issues you had, are already fixed in master:
#348
#355

[mccun934]

@xsuchy thanks, merged those fixes in manually for testing and katello-configure worked great. it was able to run multiple times and deal with the fact that previous runs failed.

is katello-configure --reset_data=RESET_DATA supposed to work? I ran it and it returned in 1-2 seconds and didn't seem todo anything

[lzap]

Its --reset-data=YES. I have to admit the --help page is not super clear, but man page is definitely better. Pushed a fix that will add this to the --help screen.

[lzap]

https://fedorahosted.org/katello/wiki/Install updated, next nightly is today 14 PM EDT