New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
HHVM fixes - tests pass hunting [WIP] #165
Conversation
$this->findTests(__DIR__ . '/find-tests/*.phptx'); | ||
Assert::count(1, $this->jobs); | ||
}); | ||
if (defined('HHVM_VERSION')) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This hack should not be needed anymore in future, since facebook/hhvm#1203 is fixed (currently only in master, but hopefully also in 3.3.0).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@Majkl578 Thanks for the tip. I'm still waiting for HHVM 3.3.0 release to test all properly and release Tester 1.3.0.
I upgraded HHVM on my VPS (which immediately got broken because of hhvm/packaging#72).
Figured out it is related to parallel tests run (and it was working on 3.2!), so this is what I got with single thread (hell, HHVM is so painfully slow in CLI):
And here for this PR, first parallel run, then serial run:
Anyway, even HHVM's stable channel is totally unreliable, broken and buggy. And this is supposed to be LTS release. Sad. |
Without this option, Fatal Error stack trace is printed even the set_exception_handler() is used (e.g. AssertException printing).
ca5f64b
to
8f4fd4e
Compare
👍 |
@Majkl578 If you can debug HHVM parallel jobs problem, please, open a new issue. |
@milo: Strange. :) It's Debian Wheezy and HHVM 3.3.0 stable channel (their official repo). Not sure how reliably debug such thing remotely... |
@milo: Ok, I tried, but I give up. I don't know how to debug such thing. Tester is run by PHP 5.6.0 from Dotdeb, tests by HHVM 3.3.0. The problem is somewhere in Runner or Job. When I use e.g. 10 thread instead of default 33, it works. With 10, it is still ok, with 15, it starts to fail randomly (like 6 tests out of 48), the higher concurrency, the more random failures. Also, some tests died with the following error:
Edit: Checked free/top and realized, it might also be related to low memory, since HHVM is extremely expensive on RAM, so 33 threads may easily exhaust all free RAM. |
@Majkl578 It really looks like free memory problem. HHVM consumes 6x more memory than PHP in average. |
Maybe we could set default threads count lower for HHVM? Like, say, 12? It's not only memory expensive, but also CPU expensive, way more than PHP. |
At first, we need feedback from more than 2 people. My VPS and server are OK ;) |
I've just tested the lastest HHVM compiled on CentOS 7 and it looks OK:
|
BTW, 33 jobs is not a good default for HHVM, it eats a lot of memory with no positive impact. The number of the system's CPU cores would fit much better. |
@vojta: 💯 Exactly what I said above. :) |
@vojta crucial is, at which number of threads will be lowest elaped time |
@dg depends on the environment. In example, we are using a shared development server. A developer running a testcase with such concurrency completely stalls that machine for the other users. Another thing is swapping out some apps or wiping the VM cache. |
On my virtual Ubuntu (64bit, 4cores) are best results for 4…7 (on Tester testsuite). |
OK, "number of cpus" (+2) sounds reasonable to me - if the test case is CPU bound, not i/o bound. There's always a possibility that the test run can run out of memory, but is much lower than with parallel 33 jobs. |
@vojta: Sounds reasonable, although on VPS it will be often 1(+2) as there is usually one virtual cpu with a single thread. |
Server with Intel Xeon X3360 (4 cores) Debian: The speed inceasing is very small with threads > 8. And as I can see, it is similar on your side. So, let's say, default threads to 8? |
I am fine with 8. |
The reason is memory saving. Based on #165 discussion.
If anyone is interested, here is current status for Nette under HHVM. All components master + Tester 1.3.0@RC + HHVM 3.4.0-dev@gd54eed7 (including merged fix for facebook/hhvm#2684).
|
@Majkl578 Could you paste somewhere a detailed dump of the failures? |
@JanJakes: |
@Majkl578 Sweet, thanks! |
One last assertion is not passing due to #162. Thinking about fix/workaround.
1st commit note: HHVM 3.3 should be out next Thursday.