enable incomplete/skipped/risky/warning for logger #3890

Merged
merged 14 commits into from Mar 8, 2017

Conversation

Projects
None yet
6 participants
@mario-naether
Contributor

mario-naether commented Jan 3, 2017

  • enable "logIncompleteSkipped" for JUnit logger
  • enable config "beStrictAboutTestsThatDoNotTestAnything" and "beStrictAboutOutputDuringTests"

Mario Näther added some commits Jan 3, 2017

Mario Näther
enable incomplete/skipped/risky/warning for logger
- enable "logIncompleteSkipped" for JUnit logger
- enable config "beStrictAboutTestsThatDoNotTestAnything" and
"beStrictAboutOutputDuringTests"
Mario Näther
enable beStrictAboutTestsThatDoNotTestAnything
- enable phpunit config "beStrictAboutTestsThatDoNotTestAnything"
src/Codeception/PHPUnit/Runner.php
-
+ $result->beStrictAboutTestsThatDoNotTestAnything(true);
+ $result->beStrictAboutOutputDuringTests(true);
+

This comment has been minimized.

@Nitpick-CI

Nitpick-CI Jan 3, 2017

Spaces must be used to indent lines; tabs are not allowed

@Nitpick-CI

Nitpick-CI Jan 3, 2017

Spaces must be used to indent lines; tabs are not allowed

@edno

This comment has been minimized.

Show comment
Hide comment
@edno

edno Jan 3, 2017

Contributor

I suggest to make beStrictAboutTestsThatDoNotTestAnything and beStrictAboutOutputDuringTests configurable instead of hardcoding the value to true.
Also why not extending this feature to all parameters for risky tests?

Contributor

edno commented Jan 3, 2017

I suggest to make beStrictAboutTestsThatDoNotTestAnything and beStrictAboutOutputDuringTests configurable instead of hardcoding the value to true.
Also why not extending this feature to all parameters for risky tests?

@Naktibalda

This comment has been minimized.

Show comment
Hide comment
@Naktibalda

Naktibalda Jan 3, 2017

Member

I think that it should be configurable with current behavior as default.
It isn't nice to change rules in the patch release.

Member

Naktibalda commented Jan 3, 2017

I think that it should be configurable with current behavior as default.
It isn't nice to change rules in the patch release.

@mario-naether

This comment has been minimized.

Show comment
Hide comment
@mario-naether

mario-naether Jan 6, 2017

Contributor

I made the phpunit settings configurable in codeception settings and updated main yml file + documentation.

Contributor

mario-naether commented Jan 6, 2017

I made the phpunit settings configurable in codeception settings and updated main yml file + documentation.

@edno

This comment has been minimized.

Show comment
Hide comment
@edno

edno Jan 6, 2017

Contributor

You should default the parameters to false in Codecept.php, otherwise that will generate errors (index xxxx does not exist in array) while running tests if the parameters are not set in codeception.yml.

Contributor

edno commented Jan 6, 2017

You should default the parameters to false in Codecept.php, otherwise that will generate errors (index xxxx does not exist in array) while running tests if the parameters are not set in codeception.yml.

@mario-naether

This comment has been minimized.

Show comment
Hide comment
@mario-naether

mario-naether Jan 7, 2017

Contributor

ok, I added the default values as false in options array.

Contributor

mario-naether commented Jan 7, 2017

ok, I added the default values as false in options array.

@mario-naether

This comment has been minimized.

Show comment
Hide comment
@mario-naether

mario-naether Jan 10, 2017

Contributor

ok, I fixed the last stuff and added an method-check for old phpunit version

Contributor

mario-naether commented Jan 10, 2017

ok, I fixed the last stuff and added an method-check for old phpunit version

@DavertMik

Thanks, this looks like a very good addition, making Codeception more compatible with PHPUnit. However, I have some concerns regarding this changes

codeception.yml
@@ -12,6 +12,10 @@ settings:
memory_limit: 1024M
log: true
lint: true
+ reportUselessTests: false

This comment has been minimized.

@DavertMik

DavertMik Jan 16, 2017

Member

by convention and by other settings we use underscore_keys in yaml config files

@DavertMik

DavertMik Jan 16, 2017

Member

by convention and by other settings we use underscore_keys in yaml config files

This comment has been minimized.

@mario-naether

mario-naether Jan 25, 2017

Contributor

ok, i changed the names settings to underscore.

@mario-naether

mario-naether Jan 25, 2017

Contributor

ok, i changed the names settings to underscore.

docs/reference/Configuration.md
+
+ # PHPUnit can be strict about tests that manipulate global state.
+ # See https://phpunit.de/manual/current/en/risky-tests.html#risky-tests.global-state-manipulation
+ beStrictAboutChangesToGlobalState: false

This comment has been minimized.

@DavertMik

DavertMik Jan 16, 2017

Member

tooooooooooooooooooooo long

what does it mean to be strict?
no_global_changes, avoid_global_changes,...
naming can be improved

@DavertMik

DavertMik Jan 16, 2017

Member

tooooooooooooooooooooo long

what does it mean to be strict?
no_global_changes, avoid_global_changes,...
naming can be improved

This comment has been minimized.

@mario-naether

mario-naether Jan 25, 2017

Contributor

this is a copy of the officially phpunit documentation

@mario-naether

mario-naether Jan 25, 2017

Contributor

this is a copy of the officially phpunit documentation

src/Codeception/Codecept.php
+ 'interactive' => true,
+ 'no-rebuild' => false,
+ 'quiet' => false,
+ 'reportUselessTests' => false,

This comment has been minimized.

@DavertMik

DavertMik Jan 16, 2017

Member

this is not a configuration options but a runner options passed with parameters like --fail-fast, --no-rebuild. This is a config option so probably it should be set inside Codeception\Configuration class. Somewhere here https://github.com/Codeception/Codeception/blob/2.2/src/Codeception/Configuration.php#L92

Yes, I know that might sound confusing 🙃

@DavertMik

DavertMik Jan 16, 2017

Member

this is not a configuration options but a runner options passed with parameters like --fail-fast, --no-rebuild. This is a config option so probably it should be set inside Codeception\Configuration class. Somewhere here https://github.com/Codeception/Codeception/blob/2.2/src/Codeception/Configuration.php#L92

Yes, I know that might sound confusing 🙃

This comment has been minimized.

@mario-naether

mario-naether Jan 25, 2017

Contributor

yes, i understand this and moved to Configuration.php

@mario-naether

mario-naether Jan 25, 2017

Contributor

yes, i understand this and moved to Configuration.php

@Naktibalda Naktibalda requested a review from DavertMik Feb 24, 2017

@DavertMik

This comment has been minimized.

Show comment
Hide comment
@DavertMik

DavertMik Mar 8, 2017

Member

Yes, looks good. Thank you!

Member

DavertMik commented Mar 8, 2017

Yes, looks good. Thank you!

@DavertMik DavertMik merged commit 91a30f0 into Codeception:2.2 Mar 8, 2017

4 checks passed

continuous-integration/appveyor/pr AppVeyor build succeeded
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
semaphoreci The build passed on Semaphore.
Details
wercker/build Wercker pipeline passed
Details

chris1312 added a commit to chris1312/Codeception that referenced this pull request Jun 16, 2017

enable incomplete/skipped/risky/warning for logger (#3890)
* enable incomplete/skipped/risky/warning for logger

- enable "logIncompleteSkipped" for JUnit logger
- enable config "beStrictAboutTestsThatDoNotTestAnything" and
"beStrictAboutOutputDuringTests"

* enable beStrictAboutTestsThatDoNotTestAnything

- enable phpunit config "beStrictAboutTestsThatDoNotTestAnything"

* - enable phpunit option BeStrictAboutChangesToGlobalState

* - added settings for reporting Incomplete/Skipped/Risky Tests

* - added default option for incomplete/skipped/risky tests in codecept.php

* - added isset check for "beStrictAboutChangesToGlobalState"

* - fixed setting "beStrictAboutChangesToGlobalState"

* - check if method "setbeStrictAboutChangesToGlobalState" exists for phpunit 4.8

* Fixed mistype

* - changed settings to underscore for naming conventions

* - moved phpunit settings to default configuration array

* - fixed "Undefined index: ..." Warning in SuiteManagerTest
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment