HHVM 3.15 package for Ubuntu 14.04/16.04 crashes and it is unstable #7333
Comments
|
I've got this issue with fbctf(by vagrant up). I attached some debug log of HHVM-nightly-dgb.
#0 0000000004d2575f |
|
HHVM 3.14.5 is no problem on FBCTF VBox. |
|
I am having this problem even with HHVM 3.14.5. It consistently stops running the hhvm service upon restart of the virtual machine (VM) running Ubuntu 14.04 on both Azure and Google Cloud. It works fine right after the initial provisioning script execution of FBCTF (at least so far I haven't noticed any problems), but upon stop/start of the VM on either cloud service HHVM fails to run (confirmed via 'sudo service hhvm status') and thus NGIX produces it's "502 Bad Gateway" error page to the client browser. I separately reported this Issue to HHVM at #7387. |
|
Can you reproduce this with a debug package? That might get a usable backtrace. |
|
Fred, I'm assuming you mean a debug package of HHVM 3.14.5. Where can I get this? |
|
3.15.0 - just install the hhvm-dbg package with apt instead of hhvm |
|
Ok, I installed hhvm-dbg v3.15 per HHVM install instructions for Ubuntu 14.04. There was a warning message that caught my eye, see my console output during the install below. Note: I had received some suggested commands to try from the FBCTF team that turned out to be a work-around for this problem, here is what they are:sudo rm /var/run/hhvm/hhvm.hhbc sudo service hhvm startAfter running the above cli, hhvm v3.14.5 service shows as running and everything else works fine again, altho IF the machine is restarted hhvm service fails to run and the problem is back until these commands are run again. Before the hhvm-dbg v3.15 installation, hhvm v3.14.5 service was running and everything was working fine. After the installation, the hhvm 3.15 service was not running, and worse yet I could not run the work-around cli commands above to get it to stay running, ie 'service hhvm status' would show it as running but at some point soon after that it showed not running again. Ie, there's something about v3.15 that makes it behave worse than v3.14.5, which is a confirmation of the Issue that the FBCTF team opened elsewhere here about that. Fred, help me get started on how to do a backtrace for you with this newly installed hhvm-dbg 3.15, I don't know how to do that. Meanwhile here's my console output during the installation of hhvm-dbg 3.15, notice the warning near the end about "clashing" NGINX config in it (an error I don't see when installing the FBCTF package with HHVM 3.14.5, thus further indication that something may have changed in hhvm between 3.14.5 and 3.15 that affects this). (nginx, NOT appache, is being used as the webserver for fbctf)eeeehaw@cyberwars-vm1:
eeeehaw@cyberwars-vm1:
|
|
Fred, sorry about the large bold font in above paragraphs. I didn't specify that, somehow this github platform decided to do that for me. I'm NOT trying to yell or anything :-) |
|
|
Ok, maybe I'm not clear on what you mean by "backtrace". I haven't run a hhvm debug package before, nor run an explicit trace (that I know of). How to do that? Thx for tip on triple backticks. |
This kind of thing; the debug package should make it so you get function names in that instead of just memory addresses |
I found a single stacktrace file in /tmp, but it is empty and shows as 0 bytes size. I also see where the HHVM error.log in /var/log/ shows that it should have written 3 stacktrace files to /tmp? But only one empty one is there. See my console output for that: |
|
Try run running hhvm in gdb, and when it crashes , |
Installed GDB. I found that hhvm process running as #2479, even though the hhvm service is not running. So, attached gdb to that process#. Here's console output, inc the 'thread apply al bt': |
|
That's not useful until hhvm has crashed; it automatically pauses when gdb attaches, but that's not the crash. |
|
Hmm...ok, then I take it that hhvm hasn't "crashed" (which is promising?) but the hhvm service is not running, when I attempt to restart that service it refuses to run, and because of that my environment is non-functional. Exploring some more, I did find what may be an interesting config file in the /home/eeeehaw/fbctf/extra/ folder, hhvm.conf; perhaps that may provide some clues why I saw that "clashing" config warning during the install of hhvm-dbg v3.15. Here's the contents of that file: |
|
You also might find this easier to debug/reproduce by removing nginx from the mix, and just using HHVM's built-in http server (hhvm.server.type=proxygen) |
|
This issue has been waiting for more info for more than 2 weeks. I'm closing it for now, but feel free to re-open it if you can provide more info. https://github.com/facebook/hhvm/wiki/Human-Timeouts I also have reason to believe it is fixed, at least @eeeehaw's problem should be fixed on master. |
HHVM Version
The package installed in both Ubuntu 14.04 and 16.04 from the HHVM repositories do not work properly. This is the output of
hhvm --version:Standalone code, or other way to reproduce the problem
I am seeing this problem with the fbctf project, since the 3.15 package was released, the platform is not working at all. Tried to isolate the problem but it was completely unstable and I was not able to isolate. HHVM was just crashing.
A simple
index.phpfile with justphpinfo()would work but it does return the same 502 Bad gateway intermittently.Actual result
HHVM is used as fastcgi within nginx and an error (502 Bad Gateway) is returned to nginx. The output of
/var/log/hhvm/error.logis:Being the content of the log in
/tmpempty.Tried to remove the 3.15 version and install manually the 3.14 package from http://dl.hhvm.com/ubuntu/pool/main/h/hhvm/ and there was no issues at all, working as expected.
The text was updated successfully, but these errors were encountered: