[Yii2] Used database connections are not getting closed #4926
Comments
|
Okay I did some deep diving. One solution would be to allow PDO reuse without using transactions (this is not hard from a technical perspective). For now several workarounds should fix it for you:
|
|
@SamMousa In my case transactions are enabled, component still called 'db'. |
|
#4928 @StalkAlex your issue should be fixed |
|
Thanks! I'll try after merge |
|
Where would I add the trigger for the The reason I had Transactions as False because we heavily use them in our app. So if something failed all the rest of the tests would fail and it would be difficult to find the problem. |
|
But seems it doesn't work for acceptance tests with webdriver. |
|
@ilgiz-badamshin I think that would work yes, I would however check if the db component is initialized first. If you have another issue please create a minimal test case as a PR to https://github.com/codeception/yii2-tests That way your error can become part of the test suite and we can prevent future regressions. |
|
@SamMousa Done: Codeception/yii2-tests#1 |
|
I'll look at the pr |
|
After update from Codeception |
|
@SamMousa, please reopen the ticket. |
|
Agree - there is still a problem with 2.4.2 (I have reconnect: true in modules: enabled: Db:) |
|
Debug logs? |
|
Part of debug log: ...
✔ PushNotificationSenderTest: Count pending | #14 (0.07s)
[Transaction] Rolling back 0 transactions
Destroying application
Destroying application
Starting application
[Fixtures] Loading fixtures
[Fixtures] Done
[yii\db\Connection::open] 'Opening DB connection: mysql:host=test_db;port=3306;dbname=klip_test'
✔ PushNotificationSenderTest: Count pending | #15 (0.07s)
[Transaction] Rolling back 0 transactions
Destroying application
Destroying application
Starting application
[Fixtures] Loading fixtures
[Fixtures] Done
[yii\db\Connection::open] 'Opening DB connection: mysql:host=test_db;port=3306;dbname=klip_test'
E BasePushNotificationCommandTest: Execute | "successful sending for android" (0.01s)
[Transaction] Rolling back 0 transactions
Destroying application
Suite done, restoring $_SERVER to original
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Time: 16.94 seconds, Memory: 72.00MB
There was 1 error:
---------
1) BasePushNotificationCommandTest: Execute | "successful sending for android"
Test tests/unit/components/notifications/commands/BasePushNotificationCommandTest.php:testExecute
[yii\db\Exception] SQLSTATE[HY000] [1040] Too many connections
#1 /app/vendor/yiisoft/yii2/db/Connection.php:624
#2 /app/vendor/yiisoft/yii2/db/Connection.php:996
#3 /app/vendor/yiisoft/yii2/db/Command.php:255
#4 /app/vendor/yiisoft/yii2/db/Command.php:1070
#5 /app/vendor/yiisoft/yii2/test/InitDbFixture.php:96
#6 /app/vendor/yiisoft/yii2/test/InitDbFixture.php:78
#7 /app/vendor/yiisoft/yii2/test/FixtureTrait.php:117
#8 Codeception\Module\Yii2->haveFixtures
#9 /app/tests/_support/_generated/UnitTesterActions.php:585
#10 /app/tests/unit/components/notifications/commands/BasePushNotificationCommandTest.php:33
ERRORS!
Tests: 180, Assertions: 285, Errors: 1.
actor: Tester
paths:
tests: tests
log: tests/_output
data: tests/_data
helpers: tests/_support
settings:
bootstrap: _bootstrap.php
memory_limit: 1024M
colors: true
modules:
config:
Yii2:
configFile: 'config/test.php'
cleanup: false
class_name: UnitTester
modules:
enabled:
- Asserts
- Yii2:
part: [orm, fixtures] |
|
Current quick workaround for this issue is to add to my test.php config file. |
|
@scottix can you confirm this is still an issue on latest version from GIT? (ie 2.4 branch). |
|
@SamMousa, the issues still exists :-( |
|
Hmm, it has something to do with fixtures... |
|
Some examples are below. class AvatarCest
{
/**
* Initializes fixtures.
*
* @param \ApiTester $I Tester
*/
public function _before(\ApiTester $I)
{
$I->haveFixtures([
UserFixture::class,
TokenFixture::class,
]);
}
// ...
}class UserExperimentTest extends \Codeception\Test\Unit
{
/**
* @var \UnitTester Tester
*/
protected $tester;
/**
* Declares the fixtures.
*/
protected function _before()
{
$this->tester->haveFixtures([
'experiments' => ExperimentFixture::class,
]);
}
// ...
}#unit.suite.yml
class_name: UnitTester
modules:
enabled:
- Asserts
- Yii2:
part: [orm, fixtures]#codeception.yml
actor: Tester
paths:
tests: tests
log: tests/_output
data: tests/_data
helpers: tests/_support
settings:
bootstrap: _bootstrap.php
memory_limit: 1024M
colors: true
modules:
config:
Yii2:
configFile: 'config/test.php'
cleanup: falseNote that some DB calls are transactional. I have 1500+ tests in total. |
|
I think I have fixed it here: #5045 Could you give this branch a try? |
namespace app\tests\fixtures;
use yii\test\ActiveFixture;
/**
* Token fixture
*
* @author Oleksandr Roslov <oleksandr.roslov@midealab.co>
*/
class TokenFixture extends ActiveFixture
{
public $modelClass = 'app\\modules\\api\\v2\\models\\Token';
public $depends = [
'app\\tests\\fixtures\\AppKeyFixture',
'app\\tests\\fixtures\\OAuthFixture',
'app\\tests\\fixtures\\UserFixture',
];
}# /tests/fixtures/data/token.php
return [
[
'user_id' => 1,
'app_key_id' => 1,
'oauth_id' => 1,
'token' => 'fjhdsidhgrlkruh974y5hghdfkghdfkg',
],
// ...
]; |
|
I downloaded https://github.com/SamMousa/Codeception/tree/yii2-event-handlers But the issue still exists. |
|
Debug output? |
|
Looks like I found where the possible problem is. The connections are not closed after each data provider test. SET GLOBAL max_connections = 17;
SHOW VARIABLES LIKE "max_connections";<?php
namespace app\tests\functional\admin;
use app\tests\fixtures\EventFixture;
use app\tests\fixtures\PermissionFixture;
use app\tests\fixtures\AdminUserFixture;
use Codeception\Example;
/**
* Tests Events page of admin panel.
*/
class EventsCest
{
/**
* Initializes fixtures.
*
* @param \FunctionalTester $I Tester
*/
public function _before(\FunctionalTester $I)
{
$I->haveFixtures([
'users' => AdminUserFixture::class,
'permissions' => PermissionFixture::class,
'events' => EventFixture::class,
]);
}
/**
* Tests available elements on pages
*
* @param \FunctionalTester $I Tester
* @param Example $example Data provider example
*
* @dataprovider dataProvider
*/
public function tryToFindButtons(\FunctionalTester $I, Example $example)
{
$I->amLoggedInAs($example['user']);
$route = $example['route'][0];
$params = $example['route'];
unset($params[0]);
$I->amOnRoute($route, $params);
if ($example['visible']) {
$I->see($example['text']);
} else {
$I->dontSee($example['text']);
}
}
/**
* User data provider
*
* @return array Data
*/
protected function dataProvider()
{
return [
'superadmin index visible button create' => [
'user' => 1,
'route' => ['admin/event'],
'text' => 'Create Event',
'visible' => true,
],
'superadmin view visible button update' => [
'user' => 1,
'route' => ['admin/event/1'],
'text' => 'Edit',
'visible' => true,
],
'superadmin view visible button delete' => [
'user' => 1,
'route' => ['admin/event/1'],
'text' => 'Delete',
'visible' => true,
],
'moderator index visible button create' => [
'user' => 2,
'route' => ['admin/event'],
'text' => 'Create Event',
'visible' => true,
],
'moderator view visible button update' => [
'user' => 2,
'route' => ['admin/event/1'],
'text' => 'Edit',
'visible' => true,
],
'moderator view visible button delete' => [
'user' => 2,
'route' => ['admin/event/1'],
'text' => 'Delete',
'visible' => false,
],
'analyst index visible button create' => [
'user' => 3,
'route' => ['admin/event'],
'text' => 'Create Event',
'visible' => false,
],
'analyst view visible button update' => [
'user' => 3,
'route' => ['admin/event/1'],
'text' => 'Edit',
'visible' => false,
],
'analyst view visible button delete' => [
'user' => 3,
'route' => ['admin/event/1'],
'text' => 'Delete',
'visible' => false,
],
'moderator analyst index visible button create' => [
'user' => 4,
'route' => ['admin/event'],
'text' => 'Create Event',
'visible' => true,
],
'moderator analyst view visible button update' => [
'user' => 4,
'route' => ['admin/event/1'],
'text' => 'Edit',
'visible' => true,
],
'moderator analyst view visible button delete' => [
'user' => 4,
'route' => ['admin/event/1'],
'text' => 'Delete',
'visible' => false,
],
'owner index visible button create' => [
'user' => 5,
'route' => ['admin/event'],
'text' => 'Create Event',
'visible' => true,
],
'owner view visible button update' => [
'user' => 5,
'route' => ['admin/event/1'],
'text' => 'Edit',
'visible' => true,
],
'owner view visible button delete' => [
'user' => 5,
'route' => ['admin/event/1'],
'text' => 'Delete',
'visible' => false,
],
];
}
}Run tests with Tests pass. Then either decrease MySQL 'owner view visible button delete TEST' => [
'user' => 5,
'route' => ['admin/event/1'],
'text' => 'Delete',
'visible' => false,
],Run tests with You’ll get error |
|
Strange, that is exactly the kind of test i'm using here: Codeception/yii2-tests#2 |
|
Debug output of https://github.com/SamMousa/Codeception/tree/yii2-event-handlers |
|
Hmm, it seems after connections are closed they get reopened. Do you have |
|
Disabled ( |
|
But the behavior is the same with |
|
Hmm, weird, if you look at this: https://travis-ci.org/Codeception/yii2-tests/jobs/397764560 Those tests actually check if database connections are being closed properly... *-) |
|
I found workaround using helper. Create file: <?php
namespace Helper;
use Codeception\Module;
use Codeception\TestInterface;
class ConnectionClose extends Module
{
public function _after(TestInterface $test)
{
if (isset(\Yii::$app->db)) {
\Yii::$app->db->close();
}
parent::_after($test);
}
}and add module to suite: modules:
enabled:
- Helper\ConnectionClose |
|
2.4.4 still does not close all connections: |
|
Of course, not closed. Because: https://github.com/Codeception/Codeception/blob/2.4/src/Codeception/Lib/Connector/Yii2.php#L93 Application destroyed without closing connection, without any destructors. |
|
@m8rge destructors are called by PHP itself, it's called garbage collection... Anyway, we watch all connections and close them after each test here: |
|
Nice, now we know the cause I think we can reproduce it as well right? Creating an app that bootstraps the db component should do it. I'll try to make a test case in the test repo. |
|
In my case It was reproducible in |
|
Okay, I looked into it some more and feel there's some information still missing.
public function getApplication()
{
if (!isset(Yii::$app)) {
$this->startApp();
}
return Yii::$app;
}So I started looking for places where
@roslov could you check all your code for anything that's unsetting Also, I've created a PR that does some code cleanup. One of the things it changes is that it doesn't recreate the application to close the session. (If there's no application, there should be no session) Could you try that branch to see if that also resolves your issue? |
I don’t know why it is so. |
|
You put session close into |
|
This means that public function closeSession()
{
if (isset(\Yii::$app) && \Yii::$app->has('session', true)) {
\Yii::$app->session->close();
}
}does initialize DB connections. |
What are you trying to achieve?
Trying to run suite of tests
What do you get instead?
Failures
Eventually get this
--------- 300) ApiUserTesterCest: Test the forgot password request Test tests/api/ApiUserTesterCest.php:ForgotPasswordRequest [yii\db\Exception] SQLSTATE[08006] [7] FATAL: sorry, too many clients already /srv/www/xyzweb/src/private/protected/vendor/yiisoft/yii2/db/Connection.php:624 /srv/www/xyzweb/src/private/protected/vendor/yiisoft/yii2/db/Connection.php:687 /srv/www/xyzweb/src/private/protected/vendor/yiisoft/yii2/db/Connection.php:613 /srv/www/xyzweb/src/private/protected/vendor/yiisoft/yii2/db/Connection.php:996 /srv/www/xyzweb/src/private/protected/vendor/yiisoft/yii2/db/Connection.php:983 /srv/www/xyzweb/src/private/protected/vendor/yiisoft/yii2/db/Schema.php:463 /srv/www/xyzweb/src/private/protected/vendor/yiisoft/yii2/db/Connection.php:881 /srv/www/xyzweb/src/private/protected/vendor/yiisoft/yii2/db/Command.php:209 /srv/www/xyzweb/src/private/protected/vendor/yiisoft/yii2/db/Command.php:1099 /srv/www/xyzweb/src/private/protected/vendor/yiisoft/yii2/db/Command.php:1120 /srv/www/xyzweb/src/private/protected/vendor/yiisoft/yii2/db/Command.php:442 /srv/www/xyzweb/src/private/protected/vendor/yiisoft/yii2/db/pgsql/Schema.php:182 /srv/www/xyzweb/src/private/protected/vendor/yiisoft/yii2/db/Schema.php:237 /srv/www/xyzweb/src/private/protected/vendor/yiisoft/yii2/db/pgsql/QueryBuilder.php:200 /srv/www/xyzweb/src/private/protected/vendor/yiisoft/yii2/db/Command.php:959 /srv/www/xyzweb/src/private/protected/vendor/yiisoft/yii2/test/InitDbFixture.php:96 /srv/www/xyzweb/src/private/protected/vendor/yiisoft/yii2/test/InitDbFixture.php:78 /srv/www/xyzweb/src/private/protected/vendor/yiisoft/yii2/test/FixtureTrait.php:117 /srv/www/xyzweb/src/private/protected/vendor/codeception/codeception/src/Codeception/Module/Yii2.php:467 /srv/www/xyzweb/src/private/protected/vendor/codeception/codeception/src/Codeception/Module/Yii2.php:285 /srv/www/xyzweb/src/private/protected/vendor/codeception/codeception/src/Codeception/Module/Yii2.php:259 /srv/www/xyzweb/src/private/protected/vendor/codeception/codeception/src/Codeception/Subscriber/Module.php:56 /srv/www/xyzweb/src/private/protected/vendor/symfony/event-dispatcher/EventDispatcher.php:212 /srv/www/xyzweb/src/private/protected/vendor/symfony/event-dispatcher/EventDispatcher.php:44 /srv/www/xyzweb/src/private/protected/vendor/codeception/phpunit-wrapper/src/Listener.php:133 /srv/www/xyzweb/src/private/protected/vendor/codeception/phpunit-wrapper/src/Listener.php:102 /srv/www/xyzweb/src/private/protected/vendor/phpunit/phpunit/src/Framework/TestResult.php:395 /srv/www/xyzweb/src/private/protected/vendor/codeception/codeception/src/Codeception/Test/Test.php:75 /srv/www/xyzweb/src/private/protected/vendor/phpunit/phpunit/src/Framework/TestSuite.php:755 /srv/www/xyzweb/src/private/protected/vendor/codeception/phpunit-wrapper/src/Runner.php:106 /srv/www/xyzweb/src/private/protected/vendor/codeception/codeception/src/Codeception/SuiteManager.php:157 /srv/www/xyzweb/src/private/protected/vendor/codeception/codeception/src/Codeception/Codecept.php:189 /srv/www/xyzweb/src/private/protected/vendor/codeception/codeception/src/Codeception/Codecept.php:158 /srv/www/xyzweb/src/private/protected/vendor/codeception/codeception/src/Codeception/Command/Run.php:466 /srv/www/xyzweb/src/private/protected/vendor/codeception/codeception/src/Codeception/Command/Run.php:361 /srv/www/xyzweb/src/private/protected/vendor/symfony/console/Command/Command.php:252 /srv/www/xyzweb/src/private/protected/vendor/symfony/console/Application.php:946 /srv/www/xyzweb/src/private/protected/vendor/symfony/console/Application.php:248 /srv/www/xyzweb/src/private/protected/vendor/symfony/console/Application.php:148 /srv/www/xyzweb/src/private/protected/vendor/codeception/codeception/src/Codeception/Application.php:108 /srv/www/xyzweb/src/private/protected/vendor/codeception/codeception/codecept:42Details
composer show)alexandernst/yii2-device-detect 0.0.12 Yii2 extension for Mobile-Detect library
behat/gherkin v4.5.1 Gherkin DSL parser for PHP 5.3
bower-asset/bootstrap v3.3.7 The most popular front-end framework for developing responsive, mobile first projects on the web.
bower-asset/inputmask 3.3.11 Inputmask is a javascript library which creates an input mask. Inputmask can run against vanilla javascript, jQuery and jql...
bower-asset/jquery 3.2.1
bower-asset/jquery-ui 1.12.1
bower-asset/punycode v1.3.2
bower-asset/yii2-pjax 2.0.7.1
braintree/braintree_php 3.30.0 Braintree PHP Client Library
cebe/markdown 1.1.2 A super fast, highly extensible markdown parser for PHP
codeception/codeception 2.4.1 BDD-style testing framework
codeception/phpunit-wrapper 6.0.9 PHPUnit classes used by Codeception
codeception/specify 1.0 BDD code blocks for PHPUnit and Codeception
codeception/stub 1.0.2 Flexible Stub wrapper for PHPUnit's Mock Builder
codeception/verify 1.0.0 BDD assertion library for PHPUnit
crossjoin/css v1.0.3 CSS reader and writer with full CSS3 support, already supporting huge parts of the current CSS4 spec. It supports media quer...
crossjoin/pre-mailer v1.0.5 Crossjoin\PreMailer converts CSS in a given HTML source to inline styles and optimizes it for sending it via e-mail. It also...
doctrine/instantiator 1.0.5 A small, lightweight utility to instantiate objects in PHP without invoking their constructors
ezyang/htmlpurifier v4.10.0 Standards compliant HTML filter written in PHP
facebook/webdriver 1.5.0 A PHP client for Selenium WebDriver
firebase/php-jwt v5.0.0 A simple library to encode and decode JSON Web Tokens (JWT) in PHP. Should conform to the current spec.
flow/jsonpath 0.4.0 JSONPath implementation for parsing, searching and flattening arrays
guzzlehttp/guzzle 6.3.2 Guzzle is a PHP HTTP client library
guzzlehttp/promises v1.3.1 Guzzle promises library
guzzlehttp/psr7 1.4.2 PSR-7 message implementation that also provides common utility methods
imagine/imagine v0.7.1 Image processing for PHP 5.3
kamranahmedse/php-geocode v2.0 A wrapper around the Google Geocoding API to get different details such as latitude, longitude, country, city, district, pos...
mobiledetect/mobiledetectlib 2.8.31 Mobile_Detect is a lightweight PHP class for detecting mobile devices. It uses the User-Agent string combined with specific ...
myclabs/deep-copy 1.7.0 Create deep copies (clones) of your objects
phar-io/manifest 1.0.1 Component for reading phar.io manifest information from a PHP Archive (PHAR)
phar-io/version 1.0.1 Library for handling version information and constraints
phpdocumentor/reflection-common 1.0.1 Common reflection classes used by phpdocumentor to reflect the code structure
phpdocumentor/reflection-docblock 4.3.0 With this component, a library can provide support for annotations via DocBlocks or otherwise retrieve information that is e...
phpdocumentor/type-resolver 0.4.0
phpmailer/phpmailer v5.2.26 PHPMailer is a full-featured email creation and transfer class for PHP
phpoffice/phpexcel 1.8.1 PHPExcel - OpenXML - Read, Create and Write Spreadsheet documents in PHP - Spreadsheet engine
phpspec/prophecy 1.7.5 Highly opinionated mocking framework for PHP 5.3+
phpunit/php-code-coverage 5.3.2 Library that provides collection, processing, and rendering functionality for PHP code coverage information.
phpunit/php-file-iterator 1.4.5 FilterIterator implementation that filters files based on a list of suffixes.
phpunit/php-text-template 1.2.1 Simple template engine.
phpunit/php-timer 1.0.9 Utility class for timing
phpunit/php-token-stream 2.0.2 Wrapper around PHP's tokenizer extension.
phpunit/phpunit 6.5.7 The PHP Unit Testing framework.
phpunit/phpunit-mock-objects 5.0.6 Mock Object library for PHPUnit
piwik/piwik-php-tracker 1.2.0 PHP Client for Piwik Analytics Tracking API
psr/http-message 1.0.1 Common interface for HTTP messages
psr/log 1.0.2 Common interface for logging libraries
sebastian/code-unit-reverse-lookup 1.0.1 Looks up which function or method a line of code belongs to
sebastian/comparator 2.1.3 Provides the functionality to compare PHP values for equality
sebastian/diff 2.0.1 Diff implementation
sebastian/environment 3.1.0 Provides functionality to handle HHVM/PHP environments
sebastian/exporter 3.1.0 Provides the functionality to export PHP variables for visualization
sebastian/global-state 2.0.0 Snapshotting of global state
sebastian/object-enumerator 3.0.3 Traverses array structures and object graphs to enumerate all referenced objects
sebastian/object-reflector 1.1.1 Allows reflection of object attributes, including inherited and non-public ones
sebastian/recursion-context 3.0.0 Provides functionality to recursively process PHP variables
sebastian/resource-operations 1.0.0 Provides a list of PHP built-in functions that operate on resources
sebastian/version 2.0.1 Library that helps with managing the version number of Git-hosted PHP projects
symfony/browser-kit v3.4.8 Symfony BrowserKit Component
symfony/console v3.4.8 Symfony Console Component
symfony/css-selector v2.8.38 Symfony CssSelector Component
symfony/debug v3.4.8 Symfony Debug Component
symfony/dom-crawler v3.4.8 Symfony DomCrawler Component
symfony/event-dispatcher v3.4.8 Symfony EventDispatcher Component
symfony/finder v3.4.8 Symfony Finder Component
symfony/polyfill-mbstring v1.7.0 Symfony polyfill for the Mbstring extension
symfony/process v3.4.8 Symfony Process Component
symfony/yaml v3.4.8 Symfony Yaml Component
theseer/tokenizer 1.1.0 A small library for converting tokenized PHP source code into XML and potentially other formats
webmozart/assert 1.3.0 Assertions to validate method input/output with nice error messages.
yiidoc/yii2-redactor 2.0.1 Extension redactor for Yii2 Framework.
yiisoft/yii2 2.0.15.1 Yii PHP Framework Version 2
yiisoft/yii2-bootstrap 2.0.8 The Twitter Bootstrap extension for the Yii framework
yiisoft/yii2-composer 2.0.6 The composer plugin for Yii extension installer
yiisoft/yii2-imagine 2.1.1 The Imagine integration for the Yii framework
yiisoft/yii2-jui 2.0.7 The Jquery UI extension for the Yii framework
yiisoft/yii2-redis 2.0.8 Redis Cache, Session and ActiveRecord for the Yii framework
yurkinx/yii2-image dev-master 2737b1e Yii2 extension for image manipulating using Kohana Image Library.
The text was updated successfully, but these errors were encountered: