Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enhancement: Implement integer increment and decrement mutators #152

Merged

Conversation

localheinz
Copy link
Member

@localheinz localheinz commented Feb 4, 2018

This PR

  • implements IntegerIncrement and IntegerDecrement mutators
  • reduces the minimum required MSI to 58%

πŸ’β€β™‚οΈ Not sure if something similar already exists and I missed it, but oftentimes tests do not verify boundaries, so maybe this helps to avoid off-by-one errors.

@localheinz
Copy link
Member Author

Build failure appears to be unrelated, see https://travis-ci.org/infection/infection/jobs/337077633#L632-L640.

@localheinz localheinz mentioned this pull request Feb 4, 2018
1 task
@maks-rafalko
Copy link
Member

maks-rafalko commented Feb 4, 2018

Thanks for creating this mutator. You are right, we didn't have it.

To reduce the number of duplicated mutations, I suggest to not mutate 0 to 1 and 1 to 0 because OneZeroInteger mutator already does it: https://infection.github.io/guide/mutators.html#Literal-Numbers

Another approach is to remove OneZeroInteger in favor of this new mutator, but probably having two of them is more flexible solution (especially after implementing Profiles, where we can use one but not another)

What do you think guys?

@maks-rafalko maks-rafalko added this to the 0.8.0 milestone Feb 4, 2018
@localheinz
Copy link
Member Author

@borNfreee

Ha, nice!

Yes, I was thinking that maybe one reason for this kind of mutator not to exist was that it might cause too many mutations. Great you like it, happy to hear suggestions! I will wait with modifications for now, then.

@theofidry
Copy link
Member

Great addition! I would skip the 0 & 1 but adding a comment mentioning the literal number to explain why. For now as we neither have profiles or a way to alter a mutator configuration depending of which mutators are present (e.g. we could allow 0 & 1 here if Literal-Numbers is not enabled

@localheinz
Copy link
Member Author

@borNfreee @theofidry

Build is failing as the minimum required MSI percentage has dropped below the configured 59%.

Before

    ____      ____          __  _
   /  _/___  / __/__  _____/ /_(_)___  ____
   / // __ \/ /_/ _ \/ ___/ __/ / __ \/ __ \
 _/ // / / / __/  __/ /__/ /_/ / /_/ / / / /
/___/_/ /_/_/  \___/\___/\__/_/\____/_/ /_/

Running initial test suite...

PHPUnit version: 6.1.0

  330 [============================] 15 secs

Generate mutants...

Processing source code files: 168/168
Creating mutated files and processes: 1038/1038
.: killed, M: escaped, S: uncovered, E: fatal error, T: timed out

S.S....E..M..E..M...M.....M.M........MMM...SSSSS..   (  50 / 1038)
M.MM....SSS.S.........S...MM...EEE.E.............M   ( 100 / 1038)
.SSSSS...MMMM....SSS...........M...MMMSM...S.MM...   ( 150 / 1038)
.SMM...S..SM.MM.MMMMMSSSSSSSSSSSSMMM..M...M...M.E.   ( 200 / 1038)
E..E.EE....EE.....M.SSSSSSS.E....EEEE...E......E..   ( 250 / 1038)
.EM..E......E.......E.......E....EE......E...E....   ( 300 / 1038)
....EE..S.E.....E........E.......E....EE.EEE.EE.E.   ( 350 / 1038)
...EE.EE...E.E..SS..E....EEE....M.EEE..EEEE.EE.EEE   ( 400 / 1038)
.EEE.EE.E.EE...EEE..EEE.EEE..E..EE.E.EEEE.EE.EE..E   ( 450 / 1038)
EEE.E.EEE.EE.EE..EEEE.E.E.EEE.....E.E..SSSS.S.E...   ( 500 / 1038)
.....S.M....MMMMMMMMMMMM.SSSSSSSS..SSSSSSSSSSSSSSS   ( 550 / 1038)
SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS   ( 600 / 1038)
SSSSSSSSSSSSSS..SSSSSSSSSSSSS.MS.M............S...   ( 650 / 1038)
.....MME..M......M..MME.......E.EEE...E......M...M   ( 700 / 1038)
.....MSSS......M.S.......EE....E.E........EEEE...M   ( 750 / 1038)
M.MM............M...........SSSS....M...SSSSSM....   ( 800 / 1038)
......M.................................S.SSSSSSSS   ( 850 / 1038)
SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS   ( 900 / 1038)
SSSSSSSSSSSS...E...SMMSSS.S.SSEMSSE.ME..MSSM.MMMMM   ( 950 / 1038)
MMSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSMMM..S.EM   (1000 / 1038)
....M.SSSSSSSSSSSSSSSSSSSSSSSSSSSSS..E               (1038 / 1038)

1038 mutations were generated:
     485 mutants were killed
     319 mutants were not covered by tests
      95 covered mutants were not detected
     139 errors were encountered
       0 time outs were encountered

Metrics:
         Mutation Score Indicator (MSI): 60%
         Mutation Code Coverage: 69%
         Covered Code MSI: 87%

Please note that some mutants will inevitably be harmless (i.e. false positives).

After

    ____      ____          __  _
   /  _/___  / __/__  _____/ /_(_)___  ____
   / // __ \/ /_/ _ \/ ___/ __/ / __ \/ __ \
 _/ // / / / __/  __/ /__/ /_/ / /_/ / / / /
/___/_/ /_/_/  \___/\___/\__/_/\____/_/ /_/

Running initial test suite...

PHPUnit version: 6.1.0

  334 [============================] 15 secs

Generate mutants...

Processing source code files: 170/170
Creating mutated files and processes: 1187/1187
.: killed, M: escaped, S: uncovered, E: fatal error, T: timed out

S.S.....E..M...E..M..M....M..M........MMMM...SSSSS   (  50 / 1187)
SSS.M...M.M..SSS.S.........S...MMM..EE..EE......M.   ( 100 / 1187)
....M..MM...SSSSS...MMMM....SSS............MM.....   ( 150 / 1187)
.MMMMSM....SM.M....M..SSMM...S..SM.MM.MMMMMMMMMMSS   ( 200 / 1187)
SSSSSSSSSSSMMM...MM.....ME..E....EEE.....EE.....M.   ( 250 / 1187)
SSSSSSS..E.E..EEE....E....E....ME...E........E....   ( 300 / 1187)
......E....E....E....E.......E....E.....EE....E...   ( 350 / 1187)
E.........E.E..S.E......E.......E.....E......E.EE.   ( 400 / 1187)
EEE..EE....EEE.E...EE...SS..EM...E.EE.....MEEE.EE.   ( 450 / 1187)
EE..EEEEE.EEE.EE.EE.E...EEE.E.EEE..EEE...EEE.EE..E   ( 500 / 1187)
EEE.E.EEE..EEEE..EEEE.EE..EE.EEEE.E.EE....E.E...SS   ( 550 / 1187)
SSES...........S...M....MMMM.MMMMMMMMMMMMMMM.SSSSS   ( 600 / 1187)
SSS..SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS   ( 650 / 1187)
SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS..   ( 700 / 1187)
SSSSSSSSSSSSS.MMS.M............SS...........MMEMM.   ( 750 / 1187)
...M........M..MEM.........EEEE.......E......MM...   ( 800 / 1187)
.M....M.M.SSSS......M.S.......EE....EMM.E........E   ( 850 / 1187)
EEE...MMM.MMM.............M............MSSSSSM...M   ( 900 / 1187)
...SSSSSM...........M.............................   ( 950 / 1187)
.............S.SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS   (1000 / 1187)
SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS..E....SMMSS   (1050 / 1187)
S.SESSSM.SSEM.ME..MSSSM.MMMMMMMMMMMSSSSSSSSSSSSSSS   (1100 / 1187)
SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSMMM..S.ME....   (1150 / 1187)
.MSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS..E                (1187 / 1187)

1187 mutations were generated:
     550 mutants were killed
     362 mutants were not covered by tests
     132 covered mutants were not detected
     143 errors were encountered
       0 time outs were encountered

Metrics:
         Mutation Score Indicator (MSI): 58%
         Mutation Code Coverage: 70%
         Covered Code MSI: 84%

Please note that some mutants will inevitably be harmless (i.e. false positives).


 [ERROR] The minimum required MSI percentage should be 59%, but actual is 58%. Improve your tests!

@maks-rafalko
Copy link
Member

Just set it to 58 in .travis.yml please

This is something that bothers me a lot that we are decreasing it from PR to PR, but MSI is not that important as covered MSI as for me.

@maks-rafalko
Copy link
Member

Thank you @localheinz!

@maks-rafalko maks-rafalko merged commit 8f494c1 into infection:master Feb 6, 2018
@localheinz localheinz deleted the feature/increment-decrement branch February 6, 2018 21:49
@localheinz
Copy link
Member Author

Thank you, @borNfreee and @theofidry!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants