[8.x] Memory leak on testing #39255
Replies: 18 comments 14 replies
-
After some overnight digging, I tried to make some barebone calls by just requiring the bootstrap file manually, instantiating the Console Kernel, and bootstraping the app. for ($i = 1; $i <= 10000; ++$i) {
$app = include __DIR__.'/../../bootstrap/app.php';
$app->make(Kernel::class)->bootstrap();
} This still makes the test suite crash around 128MB, which is the limit set to PHP globally. But, if I don't use for ($i = 1; $i <= 10000; ++$i) {
$app = include __DIR__.'/../../bootstrap/app.php';
$app->make(Kernel::class);
} Then, I decided to setup the memory limit 512MB while bootstraping the Console Kernel, and after some rounds, it hit the memory limit, so I can confirm it's a memory leak by just bootstraping the Console Kernel constantly. While using for ($i = 1; $i <= 10000; ++$i) {
$app = include __DIR__.'/../../bootstrap/app.php';
$app->make(Kernel::class)->bootstrap();
$app->flush();
} I'll try later to check if there is any Service Provider that leaks from being booted again and again, even after the container flushes itself. |
Beta Was this translation helpful? Give feedback.
-
I can confirm that disabling /**
* The bootstrap classes for the application.
*
* @var string[]
*/
protected $bootstrappers = [
\Illuminate\Foundation\Bootstrap\LoadEnvironmentVariables::class,
\Illuminate\Foundation\Bootstrap\LoadConfiguration::class,
\Illuminate\Foundation\Bootstrap\HandleExceptions::class,
\Illuminate\Foundation\Bootstrap\RegisterFacades::class,
\Illuminate\Foundation\Bootstrap\SetRequestForConsole::class,
\Illuminate\Foundation\Bootstrap\RegisterProviders::class,
// \Illuminate\Foundation\Bootstrap\BootProviders::class,
]; This array is called when bootstraping the Console Kernel to prepare the application before adding the rest of services into the container, which is done by the last two Service Providers. Pne provider being booted is causing havoc and doesn't "frees itself" appropriately. |
Beta Was this translation helpful? Give feedback.
-
Well, after checking manually each Service Provider (tested larger groups until I found the group that contained the culprit, I ended up with the Even when tested in isolation, the /**
* Define your route model bindings, pattern filters, etc.
*
* @return void
*/
public function boot()
{
$this->configureRateLimiting();
$this->routes(function () {
Route::prefix('api')
->middleware('api')
->namespace($this->namespace)
->group(base_path('routes/api.php'));
Route::middleware('web')
->namespace($this->namespace)
->group(base_path('routes/web.php'));
});
}
/**
* Configure the rate limiters for the application.
*
* @return void
*/
protected function configureRateLimiting()
{
RateLimiter::for('api', function (Request $request) {
return Limit::perMinute(60)->by(optional($request->user())->id ?: $request->ip());
});
} Disabling the Meddling with how the routes are created, the Router First, I tried to make a bunch of routes using just static closures returning void, something like this: Route::middleware('web')
->namespace($this->namespace)
->group(static function(): void {
Route::get('foo', fn(): void { });
Route::get('bar', fn(): void { });
Route::get('baz', fn(): void { });
// ...
}); This made no difference. It radically changed my approach and decided to make a sea of files which the Route Route::middleware('web')
->namespace($this->namespace)
->group(base_path('routes/test.php')
// routes/test.php
Route::group(__DIR__ . 'test/foo.php');
// routes/test/foo.php
Route::group(__DIR__ . 'bar.php');
// You catch my drift... Memory leak found |
Beta Was this translation helpful? Give feedback.
-
Thinking that caching the routes with If I remove all routes, I can easily get 5x times more runs, so it seems that something doesn't flush and its kept there. Again, everything goes AWOL once There are also Deferred Providers that are run after the other registered services providers are booted, which may pose another point of contention. I can confirm there is no memory leaks if I comment some Facades, disable loading the Deferred Providers, and only leaving from the Console Kernel bootstrappers:
Once I enable So, I'm back to square one. It's not that the Routes are a huge memory leak, but rather, something in PHP is leaking memory each time the application is bootstrapped. Kinda of out ideas. |
Beta Was this translation helpful? Give feedback.
-
This isn't a realistic example as it's only expected to run the Look, I appreciate it that you've been writing this down in such length but if there isn't an issue with a real life scenario then I don't think there's an actual problem here. This is also the first report we've ever gotten about this. If you really believe there's an issue or memory leak and you can find a solution then feel free to PR a fix. I'm moving this to discussions in the meantime. |
Beta Was this translation helpful? Give feedback.
-
We also ran into this issue today hitting the 128MB memory limit with 225 tests. With a basic Laravel install the memory increase is quite slow (2MB every 20 tests). Registering event listeners and routes increases the memory consumption faster. Currently we just increased the memory limit for our CI. |
Beta Was this translation helpful? Give feedback.
-
We are running into the same issue since the beginning in our project. We have a large test suite, and after around 150 test runs, we run out of memory. We too found out that the issue was in the application's RouteServiceProvider, more specifically when loading the routes from our files. All other application service providers are disabled. When enabling all except the RouteServiceProvider, we can run the test without any issue. There is something not getting released properly when setting up the routes. This is a REAL LIFE scenario, app is in production. Problem exists at least since 2018 on our side, this is not specific to Laravel 8. |
Beta Was this translation helpful? Give feedback.
-
I'm not sure really how relevant this actually is in real world applications because if we take the loop as a sample and create 1000 actual tests, it just works with memory usage of 16MB, it runs out of memory only on this sample where you bootstrap it multiple times in the same test, if we really want to dig into this then the first culprit is: framework/src/Illuminate/Foundation/Bootstrap/HandleExceptions.php Lines 45 to 47 in a9b3fb2
I'm really not sure if this is worth going deeper unless real case scenario is presented, which points to the framework being at fault and not some application code that is never cleaned up. |
Beta Was this translation helpful? Give feedback.
-
@donnysim I also thought that the leak is coming from not calling I was using this snippet in the protected function tearDown(): void
{
parent::tearDown();
gc_collect_cycles();
gc_collect_cycles();
dump('Memory in use: (' . ((int) (memory_get_usage() / 1024 / 1024)) . 'M)');
} |
Beta Was this translation helpful? Give feedback.
-
I got rid of quite a lot of leaking memory by removing some large config files we had in the I am wondering if the configuration repository also does not keep some stuff in between tests ? Or if that is simply a side effect from some other leak somewhere else. |
Beta Was this translation helpful? Give feedback.
-
@OrkhanAlikhanov ooh, that's nice to know. The exception handler was partially fixed to keep only single Application instance in memory so that saved some memory, but because of possible issues with Octane and Vapor the HandleExceptions is still kept in memory for every test you run, so 1k tests = 1k HandleExceptions instances in memory, fixing that can drop ~20MB of memory for 10k tests, But most likely Octane needs to be updated and I think they didn't want to get into that. I'm not familiar with how Octane does things so can't tell if it actually has a memory leak because of this. What I found is that PHPUnit is really hard to debug memory with, because it in itself keeps a lot of objects in memory before actually starting any test so the initial memory is jumping depending by the number of tests available - it creates a new test class instance for each test, so UserTest having 1k test declarations/function will result in 1k UserTest class instances before it actually starts processing them, which I'd say is more of a inefficiency from the PHPUnit side, the more tests you have the higher the starting memory will be. I thought of trying to replicate something like a test case without all the PHPUnit bootstrapping, but that's an undertaking in itself to find what where and when needs to be executed to match current behavior which not sure I want to get into, at least for now. For what else can eat up memory - everything, it's hard to tell what other static variables there are that are not reset between tests or state that lingers and the tooling is not making it easier to discover them. I believe I saw an issue about middleware also keeping it's "skipCallbacks" between tests, but that's relevant only if someone registers them. |
Beta Was this translation helpful? Give feedback.
-
So kind of did mostly extract the logic needed for testing and the result, at least for the OOB experience (L9.1, without executing a request), only HandleExceptions has memory leak without handler restore and the shutdown function. What else I found is PHP is weird and you don't want anonymous functions in a plain file that you include many times 😅 Route::get('/', function () {
return view('welcome');
}); believe it or not, including this file over and over again for each test actually is a memory leak. Making it <?php
function () {}; same behavior persists. It somehow for some reason cannot garbage collect anonymous functions created in a plain file. Convert it to Route::get('/', [SomeController::class, 'someMethod']); and the memory issue is solved. Now take into account all the group, channels, console, api routes that contain functions and it adds up. I'm honestly baffled by this. |
Beta Was this translation helpful? Give feedback.
-
Seems like there is an old PHP issue https://bugs.php.net/bug.php?id=76982 that was partially fixed in PHP 8.1 |
Beta Was this translation helpful? Give feedback.
-
Had this issue with a large set of scenarios via dataProvider.
Maybe it is the result itself, that takes large amount of space? |
Beta Was this translation helpful? Give feedback.
-
Jumping in here to add that we've been experiencing this same issue lately using the latest release at the time of writing (v10.42.0) and PHP 8.2.15. In a fresh application, we can craft a test case that reinstantiates the app and flushes it 1K times, revealing the same gradual 2MB increase mentioned by @pxlrbt. Unfortunately, none of the suggested workarounds mentioned here has any effect. In our real-world codebase, this presents itself as an eventual out-of-memory error about 2/3 through our test suite of ~1500 files. |
Beta Was this translation helpful? Give feedback.
-
After upgrading to Laravel 11, all my Feature tests stopped working: With out-of-the-box PHPUnit 10.5, tests just get stuck. CPU usage at 100%, no memory increase in System Monitor. Upgrading to PHP 11, I am getting Out-of-memory errors right away. Is anyone experiencing anything similar after upgrading to L11? |
Beta Was this translation helpful? Give feedback.
-
newrelic exception handler was the culprit. removed. also don't forget to rename your schema dump: mine became |
Beta Was this translation helpful? Give feedback.
-
I've encountered the same issue with New Relic and PHPUnit 11. You do not necessarily need to remove New Relic. See this issue for the concept of the solution: In practice define the following in your phpunit bootstrap file if (extension_loaded('newrelic')) {
newrelic_end_transaction();
} and the following in your custom test classes that extend public function setup(): void {
parent::setup();
if (extension_loaded('newrelic')) {
newrelic_start_transaction(ini_get('newrelic.appname'));
}
}
public function teardown(): void {
parent::teardown();
if (extension_loaded('newrelic')) {
newrelic_end_transaction();
}
} |
Beta Was this translation helpful? Give feedback.
-
Description:
On extensive PHPUnit testing, the application suddenly stops with the following message:
After further investigation, we couldn't pin-point at time of writing the culprit on application-code. Our tests failed on different places, but this line was the most common. We couldn't end a single test run without a memory leak.
We suspect that something is kept while refreshing the application with
$this->createApplication()
, as we could replicate the same behaviour using a loop.Steps To Reproduce:
Just simply make a new Laravel Framework in your PC.
Edit the included
tests/Feature/ExampleTest.php
first test with a loop refreshing the application.Once done, call artisan to test (or PHPUnit directly).
You should receive this line after a short while:
You should get the same problem, at least on Windows. We haven't tested on MacOS or Linux because we're poor AF and noob AF. We couldn't decipher the Xdebug profiler snapshots we made under that test because we're dumb AF too.
Beta Was this translation helpful? Give feedback.
All reactions