Passenger causes kernel alignment check errors at apache restart (amd64, xen)

[FooBarWidget]

From jekhor on May 23, 2012 13:21:17

1. Install passenger to amd64 Linux system under Xen (I am using Debian testing/unstable, kernel 2.6.32-5-xen-amd64).
2. Restart apache.

Passenger crashes with kernel message:

[3205881.384351] alignment check: 0000 [#2] SMP
[3205881.384359] last sysfs file: /sys/devices/virtual/net/lo/operstate
[3205881.384363] CPU 1
[3205881.384365] Modules linked in: ext4 mbcache jbd2 crc16 dm_mod processor thermal_sys acpi_processor aacraid 3w_9xxx 3w_xxxx raid10 raid456 async_raid6_recov async_pq raid6_pq async_xor xor async_memcpy async_tx raid1 raid0 md_mod ahci sd_mod crc_t10dif sata_nv sata_sil sata_via libata scsi_mod xen_netfront xen_blkfront
[3205881.384398] Pid: 11432, comm: PassengerWatchd Tainted: G D 2.6.32-5-xen-amd64 #1
[3205881.384402] RIP: e030:[] [] convert_from_fxsr+0x11e/0x13b
[3205881.384410] RSP: e02b:ffff88007db97d48 EFLAGS: 00250246
[3205881.384413] RAX: 0000000000000000 RBX: ffff88007d06b020 RCX: ffff88007d06b020
[3205881.384417] RDX: ffff88007db97d84 RSI: ffff88007db97dd4 RDI: ffff88007db97d68
[3205881.384421] RBP: ffff88007db97d84 R08 : 0000000000000008 R09 : 0000000000000010
[3205881.384424] R10 : 00000000ffffffff R11 : 0000000000000000 R12 : ffff88007d06b000
[3205881.384428] R13 : ffff88007db97ef8 R14 : ffff88004bc4f5c8 R15 : 000000000000000a
[3205881.384434] FS: 0000000000000000(0000) GS:ffff8800038bc000(0063) knlGS:0000000000000000
[3205881.384438] CS: e033 DS: 002b ES: 002b CR0: 000000008005003b
[3205881.384441] CR2: 00000000f75ebb70 CR3: 000000007ec15000 CR4: 0000000000002660
[3205881.384445] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[3205881.384448] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[3205881.384452] Process PassengerWatchd (pid: 11432, threadinfo ffff88007db96000, task ffff88004bc4f100)
[3205881.384456] Stack:
[3205881.384458] 00000000f73ed204 00000000f73ed204 ffff88007d06b000 ffffffff81018f82
[3205881.384463] <0> ffff0020ffff037f 080c2493ffffffff ff9435c80000e033 000000000000002b
[3205881.384470] <0> 0000000000000001 ffffffff8105d6cd 0000000000000040 000000000000000a
[3205881.384476] Call Trace:
[3205881.384480] [] ? save_i387_fxsave+0x23/0x82
[3205881.384485] [] ? __dequeue_signal+0xfb/0x124
[3205881.384490] [] ? xen_restore_fl_direct_end+0x0/0x1
[3205881.384494] [] ? xen_mc_issue+0x21/0x23
[3205881.384498] [] ? save_i387_xstate_ia32+0xc4/0x14e
[3205881.384502] [] ? get_signal_to_deliver+0x328/0x339
[3205881.384507] [] ? get_sigframe+0x8b/0xc2
[3205881.384510] [] ? ia32_setup_frame+0x38/0x18c
[3205881.384514] [] ? do_notify_resume+0x1ae/0x73f
[3205881.384519] [] ? vfs_read+0xca/0xff
[3205881.384523] [] ? sys_read+0x60/0x6e
[3205881.384526] [] ? int_signal+0x12/0x17
[3205881.384529] Code: 0f b7 96 12 04 00 00 48 8b 86 f8 03 00 00 81 ca 00 00 ff ff 89 57 18 48 8b 40 e0 48 8d 75 50 89 47 10 48 89 ea 48 89 d9 48 8b 01 <48> 89 02 66 8b 41 08 48 83 c1 10 66 89 42 08 48 83 c2 0a 48 39
[3205881.384573] RIP [] convert_from_fxsr+0x11e/0x13b
[3205881.384578] RSP
[3205881.384581] ---[ end trace 404ccc3ec3ee62ec ]---

Passenger version: libapache2-mod-passenger 3.0.11debian-1
Distro: amd64 Debian GNU/Linux testing/unstable.
Kernel: 2.6.32-5-xen-amd64 from Debian.
Xen version: 4.0.1 (preserve-AD)

Original issue: http://code.google.com/p/phusion-passenger/issues/detail?id=761

[FooBarWidget]

From jekhor on May 23, 2012 10:06:27

Reproduced with 3.0.12.

Clarification: my kernel is amd64, but userspace is i386.

[FooBarWidget]

From honglilai on May 31, 2012 23:20:31

I don't think this is a Phusion Passenger issue. I've encountered similar issues when I run an x86 userland on an x86_64 kernel even when I don't use Phusion Passenger. Most likely this is a kernel bug. You should report this to your OS vendor or the kernel developers.

Status: Invalid

[FooBarWidget]

From mh-google@edition-software.de on July 20, 2013 03:58:13

Similiar issue here, loaded passenger module results in immediate crash dump on the first request. Even if it is not a request for a rails app, so passenger should not be involved. The crash immediately occurs when the passenger module is activated.

I indeed run a 64bit kernel and 32bit userspace:
SunOS hp 5.11 oi_151a7 i86pc i386 i86pc Solaris

But this should not trigger any problem, the Solaris userspace always was 32bit except for a few kernel related programs or those really large programs which need 64bit. This is the case since introduction of the 64bit kernel some 15 or 20 years ago and never caused any issues, not on sparc and not on the x86 platform.

Apache/2.2.17 (Unix) mod_ssl/2.2.17 OpenSSL/0.9.8x DAV/2 SVN/1.7.7 PHP/5.3.18 Phusion_Passenger/4.0.10 configured

apache is configured to generate real coredumps:
Core was generated by `/usr/local/apache/2.2.17_php536/bin/httpd -k start'.
Program terminated with signal 11, Segmentation fault.
[New process 71487 ]
#0 0xfc303d5c in Passenger::DirConfig::isEnabled (this=0x0) at ext/apache2/Configuration.hpp:188
188 return enabled != DISABLED;
(gdb) bt
#0 0xfc303d5c in Passenger::DirConfig::isEnabled (this=0x0) at ext/apache2/Configuration.hpp:188
#1 0xfc301fc4 in Hooks::prepareRequestWhenInHighPerformanceMode (this=0x8aafab0, r=0x8ac0388)
at ext/apache2/Hooks.cpp:1332
#2 0xfc2f5901 in prepare_request_when_in_high_performance_mode (r=0x8ac0388) at ext/apache2/Hooks.cpp:1667
#3 0x0807f4b6 in ap_run_map_to_storage ()
#4 0x0807fd3c in ap_process_request_internal ()
#5 0x080914b8 in ap_process_request ()
#6 0x0808ed2f in ap_process_http_connection ()
#7 0x0808b136 in ap_run_process_connection ()
#8 0x080962a5 in child_main ()
#9 0x0809646b in make_child ()
#10 0x0809654d in startup_children ()
#11 0x0809688f in ap_mpm_run ()
#12 0x0807150e in main ()

Maybe this gdb backtraces helps?

Don't think we should report to some kernel developers, the linux and openindiana/illumos kernel developers aren't really the same guys ;-)