Added regression test for bug 8778

[staabm]

covers #8778

[staabm]

we grep for the debug output and because of the leading ! of this line is the exit code of the last pipe inverted.

means we are getting only a successfull exit code, when the text was not matched

[ondrejmirtes]

This grep business can break really easily.

What I'd like instead:

1. Add isResultCacheUsed to AnalysisResult (pass $resultCache->isFullAnalysis() to the constructor)
2. Create FailWithoutResultCacheErrorFormatter. After it checks that result cache was used, we can just call TableErrorFormatter.
3. Use this error formatter here in this test instead of grep.

[staabm]

added another test case, because I realized phpstan is also not using the result cache the other way arround

[ondrejmirtes]

This grep business can break really easily.

What I'd like instead:

1. Add isResultCacheUsed to AnalysisResult (pass $resultCache->isFullAnalysis() to the constructor)
2. Create FailWithoutResultCacheErrorFormatter. After it checks that result cache was used, we can just call TableErrorFormatter.
3. Use this error formatter here in this test instead of grep.

[ondrejmirtes]

I just checked AnalyseCommand and the codepath with --generate-baseline actually isn't influenced by the selected error formatter. So this isn't going to fail.

I really like --error-format=failWithoutResultCache and am looking forward to use it in other E2E tests, but not sure what to do about it here.

[staabm]

I had no better idea either. I went back using the grep way for the --generate-baseline case.

to make it more robust, I am doing a initial run, which does not expect the error message beeing in stdout and the subsequent run with the very same string.

hopefully thats good enough.

[staabm]

Sleeping over it I got a different idea.

What about adding a new cli option, e.g --assert-result-cached or --fail-when-no-result-cache instead of using a error-formatter?

---

edit: maybe instead we could trigger it via ENV var, so the enduser won't see this testing-only infrastructure

[staabm]

implemented a ENV var approach in phpstan/phpstan-src#2611

[ondrejmirtes]

Thank you.