mail/rspamd: Daemon sometimes doesn't start after reboot #466
Comments
|
@mimugmail no I did not - but from the line |
|
setup.sh handling could be a bit unstable depending on which plugin branch and core branch you are using... _opnsense_bootup_run= was renamed to _var_script= |
|
yeah, I'm running on 18.1_r2. Script is there and executable but /var/run/rspamd doesn't exist. |
|
could be the reason because I am on |
|
Not entirely sure this is the same issue as #276 ? |
|
@fichtner probably not, this is because the preparation script (setup.sh) is not called on @mimugmail machine for some reason. I never had this issue on stable 17.7. |
|
Okay, waiting for a clean test on a 18.1-RC1... |
|
@fichtner I think you broke something on 18.1: This I a fresh installed image of OPNsense and the only thing that happend is the installation of rspamd. |
|
For me it's only the call of setup.sh at bootup. When starting manually it works. Switch back from 18.1.r_2 to 17.7 fix this. I can test latest patch when the kids sleep :) |
|
@mimugmail this is the fresh image, not an updated version - just FYI. Those images were released yesterday. This screenshot is made on VirtualBox. |
|
and if you try the old package? |
|
same result... |
|
funny, where should that come from then? |
|
@fichtner manual start shows the issue: it is a PAX issue: |
|
Looks like rspamd is crashing repeatedly, causing SEGVGUARD to fire. We need to figure out why rspamd is segfaulting. |
|
no that's because it crashed before. there are no changes in the ports if both 18.1 and 17.7 version segfaults.... only leaves the kernel or the base system |
|
@lattera it's the kernel :/ 17.7.10 kernel works fine |
|
@fichtner, what in the kernel is causing the segfault? Is there a mismerge somewhere? |
|
I do not know. I just switched the kernel to the older version on the 18.1 VM. no changes to rspamd package or base system binaries. |
|
@fichtner how should I handle the issue now. At the moment the only reasonable action can be holding back rspamd or the FreeBSD 11.1 kernel... |
|
you can help debug the kernel :P |
|
@fichtner did never debug a FreeBSD kernel - until I find out the place to start it will be too late ;) |
|
@fabianfrz for now sit back and relax :) |
|
Not the same as #276 ... I can reproduce when switching betweens 17.7.11
and 18.1.
Will try to debug at work today. My test machine has to many plugins
installed, hard to follow the screen on boot up :)
2018-01-11 16:10 GMT+01:00 Franco Fichtner <notifications@github.com>:
… Not entirely sure this is the same issue as #276
<#276> ?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#466 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/ATfeobpaq7-KfBHkZ9KFQkq6Jc5vdfIFks5tJiRvgaJpZM4RS_U2>
.
|
|
make sure to test with the 11.0 kernel (17.7-based) for now 👍 |
|
I found the issue: rspamd calls mmap(MAP_32BIT), which was allowed in the 17.x kernel, but disallowed now in the 18.1 kernel. The output from Until I come up with a permanent fix, I'm going to drop the feature that disallows calls to mmap with the MAP_32BIT flag set. From the perspective of 17.x to 18.x, we're not losing anything by dropping the feature. This will give me a good excuse to finally integrate exploit mitigation toggling directly into pkg. Note that this particular feature (the pkg integration) will take a while to implement (maybe for 18.7). |
|
More info: Turns out that rspamd itself isn't calling Even on 64-bit systems, LuaJIT uses 32-bit pointers. They enforce 32-bit pointers by passing in the MAP_32BIT flag to In OPNsense 17.x, we didn't enable the HardenedBSD feature that disallows MAP_32BIT. Thus, LuaJIT worked fine. We enabled the feature in 18.1, which is why the LuaJIT implementation within rspamd crashes. |
|
So, what I'm going to do is: In OPNsense: hardening.pax.disallow_map32bit.status will default to 1 (application must opt in). Advanced users who also don't need rspamd can set it to 2 (application must opt out) via a tunable. In the meantime, you can work around this by setting the hardening.pax.disallow_map32bit.status tunable to 1 via /boot/loader.conf.local. For example, login via ssh and run the following command (as root): |
LuaJIT creates 32-bit memory maps in a 64-bit execution environment, which is disabled by default in HardenedBSD. We never ported the MAP_32BIT disallow logic over from HardenedBSD to OPNsense prior to 18.1. The merge of HardenedBSD's ASLR implementation for 18.1 also brought in the new logic. Instead of removing disallow_map32bit, simply set it to 1 by default. This will allow those who don't use LuaJIT applications to set it to 2 in /boot/loader.conf.local. Note that this commit is specific to OPNsense. OPNsense ships without the PAX_SYSCTLS kernel option, thus it's impossible to toggle via sysctl.conf(5) and must be set via loader.conf.local. Signed-off-by: Shawn Webb <shawn@opnsense.org> github-issue: opnsense/plugins#466
LuaJIT creates 32-bit memory maps in a 64-bit execution environment, which is disabled by default in HardenedBSD. We never ported the MAP_32BIT disallow logic over from HardenedBSD to OPNsense prior to 18.1. The merge of HardenedBSD's ASLR implementation for 18.1 also brought in the new logic. Instead of removing disallow_map32bit, simply set it to 1 by default. This will allow those who don't use LuaJIT applications to set it to 2 in /boot/loader.conf.local. Note that this commit is specific to OPNsense. OPNsense ships without the PAX_SYSCTLS kernel option, thus it's impossible to toggle via sysctl.conf(5) and must be set via loader.conf.local. Signed-off-by: Shawn Webb <shawn@opnsense.org> github-issue: opnsense/plugins#466 (cherry picked from commit 5a1bbae)
|
@fichtner, I'm not sure how you want to handle this. Now that the commit has been made, we should push out a new kernel. Then users can retest and report. If the retest is successful, we can close this ticket. |
|
@lattera means we have a workaround and a sticky solution. How about we land this commit with the final 18.1 images and go with the workaround for 18.1-RC2? So far rspamd/luajit is the only report of this failing. And it's a security feature after all. |
PR: opnsense/plugins#466 (cherry picked from commit 90d8ef6)
|
@fichtner be careful, haproxy is also including lua. |
|
on RC2 + reboot it will work with the RC1 kernel... |
|
Fixed a long time ago :) |
LuaJIT creates 32-bit memory maps in a 64-bit execution environment, which is disabled by default in HardenedBSD. We never ported the MAP_32BIT disallow logic over from HardenedBSD to OPNsense prior to 18.1. The merge of HardenedBSD's ASLR implementation for 18.1 also brought in the new logic. Instead of removing disallow_map32bit, simply set it to 1 by default. This will allow those who don't use LuaJIT applications to set it to 2 in /boot/loader.conf.local. Note that this commit is specific to OPNsense. OPNsense ships without the PAX_SYSCTLS kernel option, thus it's impossible to toggle via sysctl.conf(5) and must be set via loader.conf.local. Signed-off-by: Shawn Webb <shawn@opnsense.org> github-issue: opnsense/plugins#466
LuaJIT creates 32-bit memory maps in a 64-bit execution environment, which is disabled by default in HardenedBSD. We never ported the MAP_32BIT disallow logic over from HardenedBSD to OPNsense prior to 18.1. The merge of HardenedBSD's ASLR implementation for 18.1 also brought in the new logic. Instead of removing disallow_map32bit, simply set it to 1 by default. This will allow those who don't use LuaJIT applications to set it to 2 in /boot/loader.conf.local. Note that this commit is specific to OPNsense. OPNsense ships without the PAX_SYSCTLS kernel option, thus it's impossible to toggle via sysctl.conf(5) and must be set via loader.conf.local. Signed-off-by: Shawn Webb <shawn@opnsense.org> github-issue: opnsense/plugins#466
Hi,
this is the third time rspamd didn't start after a reboot. @fabianfrz did you experienced this too?
Seems to be something with the pid file:
The text was updated successfully, but these errors were encountered: