New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
"stack smashing" segmentation fault with znc-git on armv7h #1459
Comments
|
It seems that this issue occurs even on stable ZNC 1.6.5. I am now inclined to suspect not ZNC, but an external module I use, znc-clientbuffer. I will rebuild that package with debug symbols, report the issue there too, and see how this develops. |
|
Sorry, but I don't think that coredump is of much use. Coredumps are specific to the binary in use, so in this case this depends on your Also, don't upload your coredumps to the internet. For example, I am fairly sure that your I didn't find anything that looks like a password in there, but it might be possible to find passwords. And SSL private keys. And other secrets. Edit: And you are using systemd's resolved, so ZNC ended up connecting to DBus. Edit: Okay, your password for rizonirpg is in there and also something that looks like a secret to enter some special channel. |
|
Perhaps that was short sighted. I've taken it down. Thanks for the injection of common sense. I'll have a look at this again later on. |
|
What I've had chance to test so far:
I've closed the ticket over at znc-clientbuffer. I'll report back here once again as soon as I have the time to do some more thorough debugging. I'll also see if I can get in touch with any other ALARM users - the issue may be isolated to there (quirks in memory management for some core system package, or just badly built in some way etc.). |
|
Iam also experiencing this same error with raspberry 2, os: Arch Linux ARM Fault code: |
|
I think we have grounds now to raise this with the Archlinux ARM maintainers. https://archlinuxarm.org/forum/viewtopic.php?f=15&t=12288 |
|
Okay, so on that list of upgrades: The znc upgrade has the commit message "icu 60.1 rebuild", so does not change any code. The following could be related: openssl, icu, (libidn2?) Now on something more productive:
|
|
Huh, I'm surprised you got to see a commit message for that anywhere. On the archlinuxarm website some packages, ZNC included, just display an error instead of the PKGBUILD or changelog. I figure they must also have a git repo I'm not aware of. |
|
The upgrade is:
and this is just what https://git.archlinux.org/svntogit/community.git/commit/trunk?h=packages/znc&id=0a4fcc82a47522079462178a168b1f7161af3326 does. I didn't even know that archlinuxarm had its own website, but I would guess it does not deviate much from archlinux PKGBUILDs, no? |
|
I'm facing the same issue on ArchLinuxArm running on an ARMv5 machine. I rebuilt znc with the --disable-charset option, and could not reproduce the bug after that. |
|
So, apparently it is |
|
I can confirm @Bonstra's findings, and after adding --disable-charset to the build options, ZNC survives overnight, which it hasn't been able to do since the affected upgrades. I've tried to keep some of the ALARM maintainers up-to-date on IRC but I suspect this might take some communication between them and the znc devs. I'll update the ALARM forum thread later today. |
|
I don't have much to add aside from a 'me too' - running ALARM on a Pi 3. Does the --disable-charset build option affect functionality at all? I'm happy to use it as a workaround until we get a proper fix! |
Well, all charset conversion is disabled. So if you configured charsets in ZNC, they will no longer be applied. |
|
I'm me too-ing this as well, ALARM on a Pi 2. I decided to downgrade ICU and znc by one release, instead of the build flag. I've got a module that requires modpython, which fails the build --disable-charset. Standard Arch warnings about partial upgrades apply, ICU did receive a soname bump this upgrade. |
|
@psychon came here from http://bugs.icu-project.org/trac/ticket/13490#comment:1 — there's not a lot of detail here. What are the two converters being used? |
|
@srl295 I don't really know details either and I am still hoping for more myself. Most of what I know is from https://gist.github.com/Bonstra/f38a184d0af66ee71d0dc9f8936db65c which is a gdb Wait, what? So, if I understand the above correctly, the code is trying to convert from UTF-8 to UTF-8 in strict mode when the stack overflow occurs. To summarise:
@Bonstra What are your ZNC's encoding settings? Specifically for the Mozilla IRC server where someone is in However, all of the above can very well be wrong. The The difference between these two pointers is huge, but is supposed to be beginning and end of the input string to be converted...?? (well, And another random comment: The only interesting thing that I can see in https://wiki.debian.org/ArchitectureSpecificsMemo is that arm has unsigned char while amd64 has signed char. That could be part of why this is only seen on arm...?
I cannot reproduce the issue. I just tried to get the people who can reproduce this to provide some more information and eventually gave mostly up. Edit: Another random bit of information that I get from the |
|
@psychon can you pull out the call site, the source code that calls ICU at this point? Hm, doing a utf-8 to utf-8 conversion? Might be better to just skip conversion there. As of ICU 60, UTF-8 error handling follows the w3c spec. This might result in longer error strings on output, but only in the presence of malformed input. A crash as a result would be most likely from a latent bug in calling ICU. |
|
is the code here? https://github.com/jimloco/Csocket/blob/master/Csocket.cc I haven't evaluated the loop there yet, but I think it's doing a round about way of trying to see if something is valid UTF-8. |
|
Yes, that's the code. https://github.com/jimloco/Csocket/blob/054857fe8304277ad6ab1182d854ad6ee4f24a65/Csocket.cc#L2498 is the relevant line. There is a setting which says "receive as UTF-8 and as ___, send as ___", or "receive as UTF-8 and as ___, send as UTF-8", or "receive and send in ___ only". edit: I missed the "send as UTF-8" option, duplicating the other one twice |
|
m_cnvInt and m_cnvIntStrict stand for "internal", because internally ZNC assumes everything is UTF-8, and all conversion happens on the edge. |
|
@srl295 My (longish) comment above has a link to the function and a description of what its parameters are in the list starting with "to summarize" |
|
maybe jimloco/CSocket should be involved in this discussion? since the crash is there. the CSocket loop (the call to ICU) seems buggy on first glance. It doesn't detect the case where the pivotbuf has run out of space. It seems like it might get stuck in a loop continuously appending data. |
|
Maybe he should. But it was me who added encodings support to Csocket. @jimloco
In what cases 100 bytes can be not enough for the pivot? |
I have I'm making network captures in order to inject the data received just before the crash into an ICU sample program to see if it is enough to trigger the bug. |
|
@DarthGandalf my err, sorry, pivotbuf is managed by the call. this ought to be OK. Still, is the idea (in the utf-8/utf-8 case) to validate whether it is valid UTF-8? that can be done without as many moving parts |
Okay, so this function is called with Do the other people who can reproduce this have similar settings? Does it also happen without the |
|
It appears to be a bug in ICU, where multibyte sequences may be written beyond the end of the output buffer, erasing part of the stack canary. |
|
Thanks a tons Bonstra! I applied the following patch to your --- poc.cpp.orig 2017-12-08 09:18:27.039252381 +0100
+++ poc.cpp 2017-12-08 09:19:16.010615391 +0100
@@ -11,7 +11,7 @@
#include "unicode/uloc.h"
#include "unicode/unistr.h"
-int main(int argc, char *argv[])
+int main()
{
const uint8_t bytes[110] = {
/* First 100 bytes */
@@ -33,7 +33,6 @@ int main(int argc, char *argv[])
UChar pivot[100];
char target[104];
UErrorCode status = U_ZERO_ERROR;
- int32_t len;
UConverter *src_cnv = ucnv_open("utf-8", &status);
assert(status == U_ZERO_ERROR);
UConverter *tgt_cnv = ucnv_open("utf-8", &status);
@@ -54,7 +53,7 @@ int main(int argc, char *argv[])
ucnv_convertEx(tgt_cnv, src_cnv, &outdata, &target[100], &indata, &source[110], &pivot[0], &pivotsrc, &pivottgt, pivotlimit, true, true, &status);
assert(status == U_BUFFER_OVERFLOW_ERROR);
- if (target[100] == 0xba && target[101] == 0xd && target[102] == 0xca && target[103] == 0xfe) {
+ if (target[100] == '\xba' && target[101] == '\x0d' && target[102] == '\xca' && target[103] == '\xfe') {
printf("Ok! No overflow in target buffer.\n");
} else {
printf("Buffer overflow detected:\nExpected ba d ca fe\nGot %x %x %x %x\n",Building and running the result works fine: I'm on Debian testing, so the above was with libicu-dev 57.1-8. I then installed libicu-dev 60.1-1 from experimental and the result changed: Indeed, the three byte sequence Since this pretty much confirms that the issue is in ICU, I am closing this bug. |
|
Even though the bug is in ICU, perhaps we should stop attempting conversion from UTF-8 to UTF-8 this way. |
|
for reasons unclear to me, there's a new ICU issue (with a commit against it) https://ssl.icu-project.org/trac/ticket/13510 |
I'm running znc-git from the AUR on my ODROID-XU4, running Arch Linux ARM, after adding armv7h to the architecture field. Things were fine until I decided to update/rebuild yesterday.
After between 15m-1h of being online, the service suddenly stops.
I would report this in IRC, but none of my clients are set up for direct connection, only through my ZNC session.
If there's any additional info which would be helpful, please just ask; I'll be more than happy to generate/provide it.
The text was updated successfully, but these errors were encountered: