[Testing] Some cache is not resetted between tests

[canvural]

I'm refactoring Larastan's test suite to use RuleTestCase instead of our own slow solution.

But I hit a wall during the refactor 😄

We have a custom TypeNodeResolverExtension. It's logic depends on a parameter defined in the config. And registered in config to change it accordingly.

This test provides a custom config file to override the checkModelProperties

The problem

When the test is ran with ./vendor/bin/phpunit --filter='testModelPropertyRuleOnModelFactory' it fails. Because the expected error did not happen. But if I run PHPStan directly with ./vendor/bin/phpstan analyse -vvv --debug -c tests/Rules/Data/modelPropertyConfig.neon tests/Rules/Data/model-property-model-factory.php -l9 I see the correct error.

I debugged it with XDebug little bit. And I suspect that it's related to the caching in LazyTypeNodeResolverExtensionRegistryProvider, PHPStan is doing while testing. Because when phpunit first runs, it collects data in the data providers. Since Larastan has some dataProviders with gatherAssertTypes, it triggers the LazyTypeNodeResolverExtensionRegistryProvider. Then when the actual test runs, it uses the old data.

What I tried?

• I tried to make the active flag in ModelPropertyTypeNodeResolverExtension static. Didn't help. Called the new static method in setUp methods in test classes. Didn't help.
• I added autowired: false to the ModelPropertyTypeNodeResolverExtension, tried setup to set the active flag. Didn't help.

It's easily reproducible in the test-refactors of Larastan. I did not spend time to create a new repository, because I'm sure the rest of the Larastan has no effect on this bug. I verified this by removing the include in the config file.

Does anyone have any idea? Because I'm going little bit crazy 😅

[ondrejmirtes]

And I suspect that it's related to the caching in LazyTypeNodeResolverExtensionRegistryProvider

I don't think so, because it's based on instance, it's not static. Every DI container cached by PHPStanTestCase is hashed by the used config files so it shouldn't be a problem. Every LazyTypeNodeResolverExtensionRegistryProvider should have the correct registry cached.

I'm not sure at the first sight, I'll have to dive deep into it later :)

[ondrejmirtes]

Maybe there some static caching property somewhere else in the codebase, and that causes the problem.

[canvural]

I spent some time pressing F8 (step over 😄) and found that this line is called only when PHPUnit is gathering data from data providers (so before PHPUnit 9.5.27 by Sebastian Bergmann and contributors. is printed to the output)

Since the custom types are coming from stubs, I thought maybe PHPStan only parses stubs once. And when the actual test runs, it uses this old data. No idea if this true, just guessing.

Also this disproves my previous findings. $this->active is true during this time. But it just never reaches that line, that's the problem.

[canvural]

I reproduced this in a separate repo: https://github.com/canvural/bug

I think my suspicion about stub files only parsed once is true. But no idea how to resolve it.

[canvural]

@ondrejmirtes Do you have any idea about this?

[ondrejmirtes]

There was Christmas break right after this message. I've been slowly chipping away at my email queue, while also working on stuff I'm myself interested in. So please wait a bit more :)

[ondrejmirtes]

This drove me nuts for 30 minutes, but I managed to fix it.

Reproduction: cfe9e2a
Fix: phpstan/phpstan-src@b13ce59

[canvural]

Hah, awesome. Thank you very much!