Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Tests fail not in debug mode #893

Closed
1 of 3 tasks
nishimura-d opened this issue Mar 31, 2022 · 1 comment · Fixed by #894
Closed
1 of 3 tasks

Tests fail not in debug mode #893

nishimura-d opened this issue Mar 31, 2022 · 1 comment · Fixed by #894
Milestone
4.x

Comments

@nishimura-d
Copy link
Contributor

This is a (multiple allowed):

  • bug

  • enhancement

  • feature-discussion (RFC)

  • CakePHP Application Skeleton Version: 4.3.1

What you did

  1. change config/app_local.php to 'debug' => false (or remove this and do not set DEBUG environment variable)
  2. composer test

What happened

There were 4 failures:

1) App\Test\TestCase\ApplicationTest::testBootstrap
plugins has DebugKit
Failed asserting that false is true.

/var/www/html/tests/TestCase/ApplicationTest.php:47
phpvfscomposer:///var/www/html/vendor/phpunit/phpunit/phpunit:97

2) App\Test\TestCase\Controller\PagesControllerTest::testMultipleGet
Possibly related to Cake\Http\Exception\NotFoundException: "Please replace templates/Pages/home.php with your own version or re-enable debug mode." 
#0 /var/www/html/vendor/cakephp/cakephp/src/View/View.php(1181): include()
#1 /var/www/html/vendor/cakephp/cakephp/src/View/View.php(1138): Cake\View\View->_evaluate('/var/www/html/t...', Array)
#2 /var/www/html/vendor/cakephp/cakephp/src/View/View.php(769): Cake\View\View->_render('/var/www/html/t...')
#3 /var/www/html/vendor/cakephp/cakephp/src/Controller/Controller.php(764): Cake\View\View->render()
#4 /var/www/html/src/Controller/PagesController.php(65): Cake\Controller\Controller->render('home')
#5 /var/www/html/vendor/cakephp/cakephp/src/Controller/Controller.php(539): App\Controller\PagesController->display('home')
#6 /var/www/html/vendor/cakephp/cakephp/src/Controller/ControllerFactory.php(140): Cake\Controller\Controller->invokeAction(Object(Closure), Array)
#7 /var/www/html/vendor/cakephp/cakephp/src/Controller/ControllerFactory.php(115): Cake\Controller\ControllerFactory->handle(Object(Cake\Http\ServerRequest))
#8 /var/www/html/vendor/cakephp/cakephp/src/Http/BaseApplication.php(317): Cake\Controller\ControllerFactory->invoke(Object(App\Controller\PagesController))
#9 /var/www/html/vendor/cakephp/cakephp/src/Http/Runner.php(77): Cake\Http\BaseApplication->handle(Object(Cake\Http\ServerRequest))
#10 /var/www/html/vendor/cakephp/cakephp/src/Http/Middleware/CsrfProtectionMiddleware.php(164): Cake\Http\Runner->handle(Object(Cake\Http\ServerRequest))
#11 /var/www/html/vendor/cakephp/cakephp/src/Http/Runner.php(73): Cake\Http\Middleware\CsrfProtectionMiddleware->process(Object(Cake\Http\ServerRequest), Object(Cake\Http\Runner))
#12 /var/www/html/vendor/cakephp/cakephp/src/Http/Middleware/BodyParserMiddleware.php(157): Cake\Http\Runner->handle(Object(Cake\Http\ServerRequest))
#13 /var/www/html/vendor/cakephp/cakephp/src/Http/Runner.php(73): Cake\Http\Middleware\BodyParserMiddleware->process(Object(Cake\Http\ServerRequest), Object(Cake\Http\Runner))
#14 /var/www/html/vendor/cakephp/cakephp/src/Routing/Middleware/RoutingMiddleware.php(161): Cake\Http\Runner->handle(Object(Cake\Http\ServerRequest))
#15 /var/www/html/vendor/cakephp/cakephp/src/Http/Runner.php(73): Cake\Routing\Middleware\RoutingMiddleware->process(Object(Cake\Http\ServerRequest), Object(Cake\Http\Runner))
#16 /var/www/html/vendor/cakephp/cakephp/src/Routing/Middleware/AssetMiddleware.php(68): Cake\Http\Runner->handle(Object(Cake\Http\ServerRequest))
#17 /var/www/html/vendor/cakephp/cakephp/src/Http/Runner.php(73): Cake\Routing\Middleware\AssetMiddleware->process(Object(Cake\Http\ServerRequest), Object(Cake\Http\Runner))
#18 /var/www/html/vendor/cakephp/cakephp/src/Error/Middleware/ErrorHandlerMiddleware.php(126): Cake\Http\Runner->handle(Object(Cake\Http\ServerRequest))
#19 /var/www/html/vendor/cakephp/cakephp/src/Http/Runner.php(73): Cake\Error\Middleware\ErrorHandlerMiddleware->process(Object(Cake\Http\ServerRequest), Object(Cake\Http\Runner))
#20 /var/www/html/vendor/cakephp/cakephp/src/Http/Runner.php(58): Cake\Http\Runner->handle(Object(Cake\Http\ServerRequest))
#21 /var/www/html/vendor/cakephp/cakephp/src/Http/Server.php(90): Cake\Http\Runner->run(Object(Cake\Http\MiddlewareQueue), Object(Cake\Http\ServerRequest), Object(App\Application))
#22 /var/www/html/vendor/cakephp/cakephp/src/TestSuite/MiddlewareDispatcher.php(142): Cake\Http\Server->run(Object(Cake\Http\ServerRequest))
#23 /var/www/html/vendor/cakephp/cakephp/src/TestSuite/IntegrationTestTrait.php(477): Cake\TestSuite\MiddlewareDispatcher->execute(Array)
#24 /var/www/html/vendor/cakephp/cakephp/src/TestSuite/IntegrationTestTrait.php(363): App\Test\TestCase\Controller\PagesControllerTest->_sendRequest('/', 'GET')
#25 /var/www/html/tests/TestCase/Controller/PagesControllerTest.php(39): App\Test\TestCase\Controller\PagesControllerTest->get('/')
#26 /var/www/html/vendor/phpunit/phpunit/src/Framework/TestCase.php(1545): App\Test\TestCase\Controller\PagesControllerTest->testMultipleGet()
#27 /var/www/html/vendor/phpunit/phpunit/src/Framework/TestCase.php(1151): PHPUnit\Framework\TestCase->runTest()
#28 /var/www/html/vendor/phpunit/phpunit/src/Framework/TestResult.php(726): PHPUnit\Framework\TestCase->runBare()
#29 /var/www/html/vendor/phpunit/phpunit/src/Framework/TestCase.php(903): PHPUnit\Framework\TestResult->run(Object(App\Test\TestCase\Controller\PagesControllerTest))
#30 /var/www/html/vendor/phpunit/phpunit/src/Framework/TestSuite.php(677): PHPUnit\Framework\TestCase->run(Object(PHPUnit\Framework\TestResult))
#31 /var/www/html/vendor/phpunit/phpunit/src/Framework/TestSuite.php(677): PHPUnit\Framework\TestSuite->run(Object(PHPUnit\Framework\TestResult))
#32 /var/www/html/vendor/phpunit/phpunit/src/Framework/TestSuite.php(677): PHPUnit\Framework\TestSuite->run(Object(PHPUnit\Framework\TestResult))
#33 /var/www/html/vendor/phpunit/phpunit/src/TextUI/TestRunner.php(673): PHPUnit\Framework\TestSuite->run(Object(PHPUnit\Framework\TestResult))
#34 /var/www/html/vendor/phpunit/phpunit/src/TextUI/Command.php(143): PHPUnit\TextUI\TestRunner->run(Object(PHPUnit\Framework\TestSuite), Array, Array, true)
#35 /var/www/html/vendor/phpunit/phpunit/src/TextUI/Command.php(96): PHPUnit\TextUI\Command->run(Array, true)
#36 phpvfscomposer:///var/www/html/vendor/phpunit/phpunit/phpunit(97): PHPUnit\TextUI\Command::main()
#37 /var/www/html/vendor/bin/phpunit(115): include('phpvfscomposer:...')
#38 {main}
Failed asserting that 404 is between 200 and 204.

/var/www/html/vendor/cakephp/cakephp/src/TestSuite/IntegrationTestTrait.php:773
/var/www/html/tests/TestCase/Controller/PagesControllerTest.php:40
phpvfscomposer:///var/www/html/vendor/phpunit/phpunit/phpunit:97

3) App\Test\TestCase\Controller\PagesControllerTest::testDisplay
Possibly related to Cake\Http\Exception\NotFoundException: "Please replace templates/Pages/home.php with your own version or re-enable debug mode." 
#0 /var/www/html/vendor/cakephp/cakephp/src/View/View.php(1181): include()
#1 /var/www/html/vendor/cakephp/cakephp/src/View/View.php(1138): Cake\View\View->_evaluate('/var/www/html/t...', Array)
#2 /var/www/html/vendor/cakephp/cakephp/src/View/View.php(769): Cake\View\View->_render('/var/www/html/t...')
#3 /var/www/html/vendor/cakephp/cakephp/src/Controller/Controller.php(764): Cake\View\View->render()
#4 /var/www/html/src/Controller/PagesController.php(65): Cake\Controller\Controller->render('home')
#5 /var/www/html/vendor/cakephp/cakephp/src/Controller/Controller.php(539): App\Controller\PagesController->display('home')
#6 /var/www/html/vendor/cakephp/cakephp/src/Controller/ControllerFactory.php(140): Cake\Controller\Controller->invokeAction(Object(Closure), Array)
#7 /var/www/html/vendor/cakephp/cakephp/src/Controller/ControllerFactory.php(115): Cake\Controller\ControllerFactory->handle(Object(Cake\Http\ServerRequest))
#8 /var/www/html/vendor/cakephp/cakephp/src/Http/BaseApplication.php(317): Cake\Controller\ControllerFactory->invoke(Object(App\Controller\PagesController))
#9 /var/www/html/vendor/cakephp/cakephp/src/Http/Runner.php(77): Cake\Http\BaseApplication->handle(Object(Cake\Http\ServerRequest))
#10 /var/www/html/vendor/cakephp/cakephp/src/Http/Middleware/CsrfProtectionMiddleware.php(164): Cake\Http\Runner->handle(Object(Cake\Http\ServerRequest))
#11 /var/www/html/vendor/cakephp/cakephp/src/Http/Runner.php(73): Cake\Http\Middleware\CsrfProtectionMiddleware->process(Object(Cake\Http\ServerRequest), Object(Cake\Http\Runner))
#12 /var/www/html/vendor/cakephp/cakephp/src/Http/Middleware/BodyParserMiddleware.php(157): Cake\Http\Runner->handle(Object(Cake\Http\ServerRequest))
#13 /var/www/html/vendor/cakephp/cakephp/src/Http/Runner.php(73): Cake\Http\Middleware\BodyParserMiddleware->process(Object(Cake\Http\ServerRequest), Object(Cake\Http\Runner))
#14 /var/www/html/vendor/cakephp/cakephp/src/Routing/Middleware/RoutingMiddleware.php(161): Cake\Http\Runner->handle(Object(Cake\Http\ServerRequest))
#15 /var/www/html/vendor/cakephp/cakephp/src/Http/Runner.php(73): Cake\Routing\Middleware\RoutingMiddleware->process(Object(Cake\Http\ServerRequest), Object(Cake\Http\Runner))
#16 /var/www/html/vendor/cakephp/cakephp/src/Routing/Middleware/AssetMiddleware.php(68): Cake\Http\Runner->handle(Object(Cake\Http\ServerRequest))
#17 /var/www/html/vendor/cakephp/cakephp/src/Http/Runner.php(73): Cake\Routing\Middleware\AssetMiddleware->process(Object(Cake\Http\ServerRequest), Object(Cake\Http\Runner))
#18 /var/www/html/vendor/cakephp/cakephp/src/Error/Middleware/ErrorHandlerMiddleware.php(126): Cake\Http\Runner->handle(Object(Cake\Http\ServerRequest))
#19 /var/www/html/vendor/cakephp/cakephp/src/Http/Runner.php(73): Cake\Error\Middleware\ErrorHandlerMiddleware->process(Object(Cake\Http\ServerRequest), Object(Cake\Http\Runner))
#20 /var/www/html/vendor/cakephp/cakephp/src/Http/Runner.php(58): Cake\Http\Runner->handle(Object(Cake\Http\ServerRequest))
#21 /var/www/html/vendor/cakephp/cakephp/src/Http/Server.php(90): Cake\Http\Runner->run(Object(Cake\Http\MiddlewareQueue), Object(Cake\Http\ServerRequest), Object(App\Application))
#22 /var/www/html/vendor/cakephp/cakephp/src/TestSuite/MiddlewareDispatcher.php(142): Cake\Http\Server->run(Object(Cake\Http\ServerRequest))
#23 /var/www/html/vendor/cakephp/cakephp/src/TestSuite/IntegrationTestTrait.php(477): Cake\TestSuite\MiddlewareDispatcher->execute(Array)
#24 /var/www/html/vendor/cakephp/cakephp/src/TestSuite/IntegrationTestTrait.php(363): App\Test\TestCase\Controller\PagesControllerTest->_sendRequest('/pages/home', 'GET')
#25 /var/www/html/tests/TestCase/Controller/PagesControllerTest.php(52): App\Test\TestCase\Controller\PagesControllerTest->get('/pages/home')
#26 /var/www/html/vendor/phpunit/phpunit/src/Framework/TestCase.php(1545): App\Test\TestCase\Controller\PagesControllerTest->testDisplay()
#27 /var/www/html/vendor/phpunit/phpunit/src/Framework/TestCase.php(1151): PHPUnit\Framework\TestCase->runTest()
#28 /var/www/html/vendor/phpunit/phpunit/src/Framework/TestResult.php(726): PHPUnit\Framework\TestCase->runBare()
#29 /var/www/html/vendor/phpunit/phpunit/src/Framework/TestCase.php(903): PHPUnit\Framework\TestResult->run(Object(App\Test\TestCase\Controller\PagesControllerTest))
#30 /var/www/html/vendor/phpunit/phpunit/src/Framework/TestSuite.php(677): PHPUnit\Framework\TestCase->run(Object(PHPUnit\Framework\TestResult))
#31 /var/www/html/vendor/phpunit/phpunit/src/Framework/TestSuite.php(677): PHPUnit\Framework\TestSuite->run(Object(PHPUnit\Framework\TestResult))
#32 /var/www/html/vendor/phpunit/phpunit/src/Framework/TestSuite.php(677): PHPUnit\Framework\TestSuite->run(Object(PHPUnit\Framework\TestResult))
#33 /var/www/html/vendor/phpunit/phpunit/src/TextUI/TestRunner.php(673): PHPUnit\Framework\TestSuite->run(Object(PHPUnit\Framework\TestResult))
#34 /var/www/html/vendor/phpunit/phpunit/src/TextUI/Command.php(143): PHPUnit\TextUI\TestRunner->run(Object(PHPUnit\Framework\TestSuite), Array, Array, true)
#35 /var/www/html/vendor/phpunit/phpunit/src/TextUI/Command.php(96): PHPUnit\TextUI\Command->run(Array, true)
#36 phpvfscomposer:///var/www/html/vendor/phpunit/phpunit/phpunit(97): PHPUnit\TextUI\Command::main()
#37 /var/www/html/vendor/bin/phpunit(115): include('phpvfscomposer:...')
#38 {main}
Failed asserting that 404 is between 200 and 204.

/var/www/html/vendor/cakephp/cakephp/src/TestSuite/IntegrationTestTrait.php:773
/var/www/html/tests/TestCase/Controller/PagesControllerTest.php:53
phpvfscomposer:///var/www/html/vendor/phpunit/phpunit/phpunit:97

4) App\Test\TestCase\Controller\PagesControllerTest::testCsrfAppliedOk
Failed asserting that `200` matches response status code `404`.

/var/www/html/vendor/cakephp/cakephp/src/TestSuite/IntegrationTestTrait.php:819
/var/www/html/tests/TestCase/Controller/PagesControllerTest.php:123
phpvfscomposer:///var/www/html/vendor/phpunit/phpunit/phpunit:97

FAILURES!
Tests: 10, Assertions: 19, Failures: 4.

What you expected to happen

No failures.

Testing in production environment is nonsense, but testing production behavior is significant.
Some test cases set debug mode by itself.

app/tests/TestCase/Controller/PagesControllerTest.php

Line 79 in a151dcf

Configure::write('debug', true);

  1. App\Test\TestCase\ApplicationTest::testBootstrap
    DebugKit should not be added in production.
  2. App\Test\TestCase\Controller\PagesControllerTest::testMultipleGet
    Added at ec4b498.
    This seems test PHP_SELF in tests/bootstrap.php, but now it is not required. Set PHP_SELF in the integration test case. cakephp#12076
    Still need this test?
  3. App\Test\TestCase\Controller\PagesControllerTest::testDisplay
    Will be resolved by adding Configure::write('debug', true);
    This repository does not have pages displayed in production.
  4. App\Test\TestCase\Controller\PagesControllerTest::testCsrfAppliedOk
    CSRF protection should be applied in production too.
    How about asserting the status is not 403?
@markstory markstory added this to the 4.x milestone Mar 31, 2022
@markstory
Copy link
Member

Sounds like you have several changes already mapped out. Any chance you'd like to put together a pull request with the changes you have in mind.

nishimura-d added a commit to nishimura-d/app that referenced this issue Apr 1, 2022
@nishimura-d
PHP_SELF in tests/bootstrap is no longer necessary.
dbbed7c 
The problem is resolved at CakePHP 3.6.3.
Remove a test added at
ec4b498.

Refs cakephp#893
nishimura-d added a commit to nishimura-d/app that referenced this issue Apr 1, 2022
@nishimura-d
Make PagesControllerTest stable against debug mode.
aa06c04 
Run testDisplay in debug mode.
Pages in this repo can only be displayed in debug mode.

Do not assert normal responses at testCsrfAppliedOk.
Responses may change by debug mode,
but that is not a concern of this test.
Assert the opposite of testCsrfAppliedError.

Refs cakephp#893
nishimura-d added a commit to nishimura-d/app that referenced this issue Apr 1, 2022
@nishimura-d
Make ApplicationTest stable against debug mode.
44658f3 
Make testBootstrap run in production mode,
and add a case to test bootstrap in debug mode.
DebugKit should not be added in production,
but should be added in debug mode.

Refs cakephp#893
@nishimura-d nishimura-d mentioned this issue Apr 1, 2022
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
4.x
Development

Successfully merging a pull request may close this issue.

Make tests stable against debug mode. nishimura-d/app
2 participants
@markstory @nishimura-d