bmcweb service crashes when using KVM over GUI #80
Comments
|
@eddiejames @gtmills ^^^ |
|
@edtanous You seen this before? Any thoughts? |
|
Yep, we've seen it. It's a flow control issue with how KVM was architected. My understanding of the problem here is that libvncserver has no flow control, and pushes frames as fast as the hardware can produce them. This is fine if bmcweb and the TLS connection can keep up with the rate the frames are coming. The problem shows up if the TLS and network connections go slower than the available frames (think cellular connection speeds). In that case, bmcweb has no way to adjust the flow control or block frames, as they would otherwise just stack up in the unix socket, which is just as fatal. In my original design, I had put the RFB server inside bmcweb, so it would have information about the framerate, and the buffer sizes, and only pull frames out of the hardware as the link allowed bandwidth for. This idea was dropped in Lieu of the libvncserver -> unix socket-> bmcweb -> websocket approach. Jae is working on some fixes in this area, using the existing design, and I'm really interested to see what he comes up with. If he's not successful, we might have to resurrect some of the code below to implement flow control. https://gerrit.openbmc-project.xyz/c/openbmc/bmcweb/+/16976/8/include/web_kvm.hpp#b16 |
|
Please keep us in the loop on any updates @yoojae |
|
Patches submitted: Please review and test it. Thanks! |
|
@yoojae the changes looks good when I tested them out. Thanks |
|
We did another downstream build with the latest Master and seems we hit BMC web crashing out but only when we do few the BMC web GUI refresh and switching GUI options back and forth a bit. any recent changes we should be looking at ? @yoojae |
|
On 6/13/2019 6:56 AM, George Keishing wrote:
We did another downstream build with the latest Master and seems we hit
BMC web crashing out but only when we do few the BMC web GUI refresh and
switching GUI options back and forth a bit.
|Jun 13 05:33:21 openbmc obmc-ikvm[1264]: 13/06/2019 05:33:21
SetPixelFormat : 1 | 20/ 20 ( 0.0%) Jun 13 05:33:21 openbmc
obmc-ikvm[1264]: 13/06/2019 05:33:21 TOTALS : 42 | 488/ 488 ( 0.0%) Jun
13 05:33:22 openbmc systemd[1]: bmcweb.service: Main process exited,
code=killed, status=11/SEGV Jun 13 05:33:22 openbmc systemd[1]:
bmcweb.service: Failed with result 'signal'. Jun 13 05:33:30 openbmc
systemd[1]: Started Start bmcweb server. |
any recent changes we should be looking at ? @yoojae
<https://github.com/yoojae>
Hi George,
Did you apply these patches?
https://gerrit.openbmc-project.xyz/c/openbmc/meta-phosphor/+/21482
https://gerrit.openbmc-project.xyz/c/openbmc/obmc-ikvm/+/21483
These patches are not merged to main tree so you should cherry pick the
patches manually.
Thanks,
Jae
…
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#80?email_source=notifications&email_token=AH6AZ33TT3YO23SLGI6BQCLP2JGYDA5CNFSM4HIWCZQKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODXTYVBQ#issuecomment-501713542>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AH6AZ37B6XXC2DX644WDJGTP2JGYDANCNFSM4HIWCZQA>.
|
|
Yup we picked those commits when I re-ran the test again. |
All I did was following:
and and I see the bmcweb service crashing out
The text was updated successfully, but these errors were encountered: