Multi Core Upgrader Failed or stuck in half-upgraded state A14.B? #77

Closed
zkrebs opened this Issue Sep 24, 2010 · 7 comments

Comments

Projects
None yet
2 participants

zkrebs commented Sep 24, 2010

Followed UPGRADE.txt and I see that a new host master was made in ~/host_master/002/sites however it is not present in my Aegir UI, it still is using ~/host_master/001 and my aliases are still present in ~/host_master/001/sites/.

Upgrade on Ubuntu 10.04 had issues:

mime.types' ->mime.types-pre-A12.D'
invoke-rc.d nginx start failed. Error (if any): 0

Displaying the last 15 lines of /opt/tmp/aegir-install.log to help troubleshoot this problem:
zip is already the newest version.
zlib1g-dev is already the newest version.
zlibc is already the newest version.
The following packages were automatically installed and are no longer required:
libldns1 libopendkim1 libev3 liblua5.1-0 libdb4.7 libunbound2
Use 'apt-get autoremove' to remove them.
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
No Pyrex, trying Cython...

The python package 'Pyrex' is not available. If the .c files are available,
they will be built, but modifying the .pyx files will not rebuild them.

Starting php_fpm . done
Starting nginx: nginx.

Additionally, after this failed upgrade Nginx wouldn't start.

Starting nginx: [emerg]: open() "/var/aegir/config/includes/fastcgi_params.conf" failed (2: No such file or directory) in /var/aegir/config/

How do I roll back the upgrade? Or, how do I complete it?

zkrebs commented Sep 24, 2010

I went to ~/backups and restored my /drush_make-pre-002 and /provision-pre-002 and provision_boost-pre-002 to their proper places, however cron runs, but my original 001 platform won't verify again, because cron doesn't seem to be working.. Here's the output..

drush core-cron --debug
Drush bootstrap phase : _drush_bootstrap_drupal_root() [0.07 sec, 3.97 MB] [bootstrap]
Loading drushrc "/var/aegir/host_master/001/drushrc.php" into "drupal" scope. [0.07 sec, 3.98 MB] [bootstrap]
Initialized Drupal 6.19 root directory at /var/aegir/host_master/001 [0.08 sec, 4.72 MB] [notice]
Drush bootstrap phase : _drush_bootstrap_drupal_site() [0.09 sec, 4.72 MB] [bootstrap]
Initialized Drupal site aegir.openashland.com at sites/aegir.openashland.com [0.09 sec, 4.72 MB] [notice]
Loading drushrc "/var/aegir/host_master/001/sites/xxx.yyy.com/drushrc.php" into "site" scope. [0.09 sec, 4.72 MB] [bootstrap]
Drush bootstrap phase : _drush_bootstrap_drupal_configuration() [0.11 sec, 4.95 MB] [bootstrap]
Drush bootstrap phase : _drush_bootstrap_drupal_database() [0.12 sec, 4.96 MB] [bootstrap]
Successfully connected to the Drupal database. [0.12 sec, 4.96 MB] [bootstrap]
Drush bootstrap phase : _drush_bootstrap_drupal_full() [0.14 sec, 5.2 MB] [bootstrap]
Drush bootstrap phase : _drush_bootstrap_drupal_login() [0.24 sec, 11.03 MB] [bootstrap]
Found command: core-cron (commandfile=core) [0.25 sec, 11.03 MB] [bootstrap]
Initializing drush commandfile: drush_make [0.25 sec, 11.03 MB] [bootstrap]
Initializing drush commandfile: drush_make_d_o [0.25 sec, 11.03 MB] [bootstrap]
Initializing drush commandfile: provision [0.25 sec, 11.04 MB] [bootstrap]
Loading drushrc "/var/aegir/host_master/001/sites/xxx.yyy.com/drushrc.php" into "site" scope. [0.25 sec, 11.04 MB] [bootstrap]
Load alias @server_master [0.25 sec, 11.04 MB] [notice]
Loading mysql driver for the db service [0.25 sec, 11.05 MB] [notice]
Loading nginx_ssl driver for the http service [0.25 sec, 11.06 MB] [notice]
WD cron: Cron run completed. [0.27 sec, 11.17 MB] [notice]
Cron run successfully. [0.27 sec, 11.16 MB] [success]
Command dispatch complete [0.27 sec, 11.14 MB] [notice]
Timer Cum (sec) Count Avg (msec)
page 0.164 1 164.39

Peak memory usage was 11.63 MB [0.27 sec, 11.14 MB]

Owner

omega8cc commented Sep 25, 2010

If you see errors while upgrading your system, stop Barracuda and run first what is displayed, like $ apt-get autoremove and then run Barracuda again.

If your hostmaster upgrade failed, revert to files archived in backup and try the hard restart for nginx: $ killall nginx; service nginx start. This should help and then you should be able to run Barracuda again and upgrade your core hostmaster. If not, you also need to restore your hostmaster database from last backup. It is stored in /data/disk/arch/sql.

zkrebs commented Sep 25, 2010

When I run the upgrade script is a new core made as 002 ? and then switched to be the default hostmaster? I followed your advise, and got further along in the install - where it was installing the wildcard SSL aspect and CGP, but it doesn't seem like aegir has updated. Cron still isn't working. IS there any way to tell what version I am running?

Owner

omega8cc commented Sep 27, 2010

Yes, you can see the install logs for both Octopus and Barracuda, it was mentioned in the comment linked in the README.txt: http://groups.drupal.org/node/89594#comment-297894

zkrebs commented Sep 27, 2010

My barracuda_log.txt only mentions:

Sat Sep 25 18:09:22 UTC 2010 / Ubuntu.lucid i686 / Aegir HEAD / Barracuda A14.B / Nginx 0.8.50 / PHP 5.2.14

Cron isn't working.

When I run su-s /bin/bash -aegir
and then drush '@hostmaster' hosting-dispatch -d

i get

Drush bootstrap phase : _drush_bootstrap_drupal_root() [0.09 sec, 4.21 MB] [bootstrap]
Loading drushrc "/var/aegir/host_master/001/drushrc.php" into "drupal" scope. [0.09 sec, 4.21 MB] [bootstrap]
Initialized Drupal 6.19 root directory at /var/aegir/host_master/001 [0.1 sec, 4.95 MB] [notice]
Drush bootstrap phase : _drush_bootstrap_drupal_site() [0.12 sec, 4.95 MB] [bootstrap]
Initialized Drupal site aegir.openashland.com at sites/aegir.openashland.com [0.12 sec, 4.95 MB] [notice]
Loading drushrc "/var/aegir/host_master/001/sites/aegir.openashland.com/drushrc.php" into "site" scope. [0.12 sec, 4.95 MB] [bootstrap]
Drush bootstrap phase : _drush_bootstrap_drupal_configuration() [0.13 sec, 4.95 MB] [bootstrap]
Drush bootstrap phase : _drush_bootstrap_drupal_database() [0.15 sec, 4.96 MB] [bootstrap]
Successfully connected to the Drupal database. [0.15 sec, 4.96 MB] [bootstrap]
Drush bootstrap phase : _drush_bootstrap_drupal_full() [0.18 sec, 5.2 MB] [bootstrap]
Found command: hosting-dispatch (commandfile=hosting) [0.3 sec, 11.04 MB] [bootstrap]
Initializing drush commandfile: drush_make [0.3 sec, 11.04 MB] [bootstrap]
Initializing drush commandfile: drush_make_d_o [0.3 sec, 11.05 MB] [bootstrap]
Initializing drush commandfile: provision [0.3 sec, 11.05 MB] [bootstrap]
Load alias @platform_001 [0.3 sec, 11.05 MB] [notice]
Load alias @server_master [0.3 sec, 11.06 MB] [notice]
Loading mysql driver for the db service [0.31 sec, 11.07 MB] [notice]
Loading nginx_ssl driver for the http service [0.31 sec, 11.08 MB] [notice]
Including profiles/hostmaster/modules/hosting/dispatch.hosting.inc [0.31 sec, 11.12 MB] [bootstrap]
dispatching queues [0.31 sec, 11.15 MB] [notice]
dispatching disabled [0.33 sec, 12.06 MB] [notice]
Command dispatch complete [0.33 sec, 12.04 MB] [notice]
Timer Cum (sec) Count Avg (msec)
page 0.195 1 195.44

Peak memory usage was 12.15 MB [0.33 sec, 12.04 MB]

zkrebs commented Sep 27, 2010

my hosting-dispatch was disabled,
to fix:

su-s /bin/bash -aegir
drush vset hosting_dispatch_enabled 1 -y
drush '@hostmaster' hosting-dispatch -d (to test for errors)

thanks omega8cc

Owner

omega8cc commented Sep 27, 2010

OK, since you got logged out on irc when I attempted to answer the question, I will post it here instead:

Q: what the HOT_SAUCE and shared code in octopus is?

A: you need to start using Octopus for a few instances and try also a few upgrades to see why it is useful to have shared code and why hot sauce check is important. You could also just read the changelog - it is explained there. Of course the script also includes a short explanation, but it is easier to understand when you are using it than when you are just reading the readme.

omega8cc closed this May 22, 2014

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment