Bus error since 4.0.4 on a Raspberry Pi

[merdok]

Hi,

the recent update (4.0.4) is causing my nodejs app to crash, i am using https://www.npmjs.com/package/websocket as a dependency to connect to a websocket endpoint, which has this package as dependency.
I was tracking down the issue and found out that the "Bus error" happens during the mask function here https://github.com/theturtle32/WebSocket-Node/blob/master/lib/WebSocketFrame.js#L265

source = <Buffer 7b 22 69 64 22 3a 22 62 61 63 30 33 38 30 36 30 30 30 31 22 2c 22 74 79 70 65 22 3a 22 72 65 71 75 65 73 74 22 2c 22 75 72 69 22 3a 22 73 73 61 70 3a ... 37 more bytes>
mask = <Buffer 48 29 6e 2b>
output = <Buffer 81 d7 48 29 6e 2b 00 00 08 94 21 02 01 00 00 00 88 0f 00 00 2c 00 00 00 88 90 21 02 68 ad 21 02 73 65 74 4e 6f 44 65 6c 61 79 21 02 00 00 00 00 94 3d ... 43 more bytes>
offset = 6
length =87

I have no idea what you changed in version 4.0.4 but when i rollback to version 4.0.3 then it still works.
As far as I understand that some build c program is called to execute those functions faster but apparently it is now broken on some machines. I am running on a Raspberry Pi:
Linux raspberrypi 5.10.52-v7l+ #1441 SMP Tue Aug 3 18:11:56 BST 2021 armv7l GNU/Linux

Can you tell me what the issue here is, and how to fix this? Can it be that the latest update broke this package for raspberry pi devices?

[lpinca]

We rebuilt the binary but that is automated and we made no changes. Maybe the compiler used in https://github.com/dockcross/dockcross/tree/master/linux-armv7 was updated and it produced a different binary.

Can you try to run this script https://github.com/websockets/bufferutil/blob/v4.0.4/package.json#L13, copy the generated binary to your Raspberry Pi and see if it works?

This issue might be similar to #132.

[merdok]

I have tried that and the issue is still there.
But i am pretty certain that something is broken with the binary after your recent update...
To be sure about that the binary has an issue, I renamed the node.napi.armv7.node on my Raspberry Pi to some random name blabla.node, and the "Bus error" is gone and everything works again. I guess when the correct binary cannot be found then it defaults to fallback.js, right?

Maybe something with the docker container changed since you automatically build the binary last time?

[lpinca]

I guess when the correct binary cannot be found then it defaults to fallback.js, right?

Correct.

Maybe something with the docker container changed since you automatically build the binary last time?

Maybe but that is outside of our direct control. What we can do is stop shipping prebuilt binaries for armv6 and armv7 but I don't know if it is a good idea.

[merdok]

I actually did not notice any change in performance with the fallback.js so I am not sure if that is a concern (but I guess that would depend on how heavy it is used)

I mean the prebuilt library was working for arm7 in version 4.0.3. I think you can continue to ship them but you need to actually make sure that they are working especially when you build them automatically, just building and hoping that they will work is quite a bad idea...

The question is now if you can provide a quick fix (by temporary removing the broken binaries) or if you want to find the root cause of the broken binaries and fix the problem there?

[lpinca]

I mean the prebuilt library was working for arm7 in version 4.0.3. I think you can continue to ship them but you need to actually make sure that they are working especially when you build them automatically, just building and hoping that they will work is quite a bad idea...

Testing them on the same machine where they are built will obviously work.

[lpinca]

I think I'll just cut a new patch or minor version removing prebuilt binaries for armv6 and armv7.

[merdok]

That would be fine with me, but you could also revert back to the binaries from 4.0.3

[lpinca]

That would be fine with me, but you could also revert back to the binaries from 4.0.3

Then the next time we cut a new release the binaries will be broken again.

[merdok]

That's why you would need to investigate the root cause of the issue.
Anyway, I am fine with any solution as long as it will work again 😊

[lpinca]

I don't have an armv6 or armv7 device so it's not easy for me to investigate.

[merdok]

The "Bus error" is not really something what tells you what exactly the issue is... if you tell me how I can debug that then I could help you.

[lpinca]

According to https://stackoverflow.com/questions/212466/what-is-a-bus-error-is-it-different-from-a-segmentation-fault, it is a memory alignment issue and if I have to guess, it is caused by dockcross/dockcross@7b63720#diff-f9e3abded6708e2d856adcdaeb77508db9fb89eb724ac8d330cfcac014a5dab0.

[merdok]

I wonder if also other binaries are affected which were then build using the same way. I mean it should not only affect then this package but many more.

[lpinca]

I mean it should not only affect then this package but many more.

Yes, other modules might be affected by the same issue.

Anyway, we are only cross compiling ARM binaries.

[lpinca]

Can I ask you to build the binary on your Raspberry Pi and see if it works?

git clone https://github.com/websockets/bufferutil.git
cd bufferutil
npm install
npm test