Camp Fire

This release is mainly a preparation of introducing mutation schemata by refactoring the analyse phase into more steps. A side effect of this is that mutation schematas requires the mutants to be valid. A problem that has existed is that the mutants produced by the SDL mutation operator had a medium probability of introducing either invalid code or undefined behaviour. This has been handled by the refactored analyser.

New features

• introduce profiling of the analyse and report stage. This can be used by a user to understand "why" certain operations take time.
• Print a progress message when removing orphaned mutants. This can take a long time thus this printing avoid e.g. a Jenkins timeout and helps a user not get impatient.
• Mutations that would occur inside a C/C++ macro is now removed except a small list of hard coded exceptions such as TRUE and FALSE.
• The test command runner has been able to execute the tests in parallel since the v2.0.0. It is now possible to configure it to stop execution as soon as one of the test commands signal failure. This can reduce the time it takes to test a mutant at the expense of the report of the test cases being less accurate.
• The test command runner automatically prioritize test cases to execute by how often they kill mutants. It means that it will over time execute the best test commands first.

Fixes

• speedup database access by adding indexes. How much it helps depends on the size of the database but empirical data shows between a 20-90% speedup.
• don't generate SDL mutants that remove returns in functions that has a return type other than void.
• slow down the analyse worker threads if the database thread that store the result is overloaded.
• set the mutations operators to use to "any" if the user doesn't specify an operator.
• Reset the timeout iterator to zero when a mutation operator has finished testing all its mutants. Previously the value where kept thus it created a strange timeout behaviour when switching between operators.
• Eliminate all SDL mutants that resulted in undefined behaviour.
• Eliminate useless mutants generated from the UOI mutation operator. Instead of fully implementing the academical definition of UOI it now only removes negation. This makes UOI useful because most mutants that it generates are productive.

Deprecation

• remove the COR mutation operator. It was rarely used and produced hard to understand mutants.

Continuous Flow

This release focus on improving the integration in a CI workflow.

New features:

• Speed-up mutation testing workflow by allowing the user to use a diff to direct dextool to what files to analyse and test. The diff can be from git diff.
• Allow a user to speed-up the analyze phase when saving to the database by reducing the SQLite safety settings (--fast-db-store)
• Speed-up saving the analyze result to by only saving those files that have changed.
• Speed-up the analyzer by parallelising it.
• Allow the user to specify multiple test_cmd that is executed in parallel.
• It is now possible to specify a directory test_cmd_dir which dextool will scan, after compilation, for binaries. These binaries are added as test_cmds.
• Allow a user to specify a max runtime when testing mutants. This is to be able to give "fast" feedback on a pull request.
• Add a report section which display what mutants a test case uniquely kill and those that has no unique kills.
• Allow a user to exclude files from analysis.
• The admin command group now has an option to mark mutants with a specific status. This is for the users that can't modify the source code to add // NOMUT on lines with mutants that should be ignored.
• Which mutants that are tested in a diff is deterministic (always the same over a week) to make it possible for a user to e.g. add or change test cases to kill mutants in the diff.
• Merge the mutants that a suite of parameterisized GoogleTest test cases kill to the "top name". This helps with unique and similarity anaylsis.
• Execute test case binaries in parallel.

Breaking change:

• test_cmd, build_cmd and analyze_cmd in the configuration file now requires that the user write the path to the command. Such as ./build.sh.

Fixes:

• Only write the output from test cases to the filesystem when the user has specified an external analyze_cmd. The builtins no longer requires this temporary storage.
• A compilation_commands.json is only required when generating a report if the report contains sections which requires it.
• Repair the short term view and rename it to diff view.
• Only reset and test old mutants when there is "time left" to do so.
• Fix bug when parsing compilation_commands.json when paths contain whitespaces.
• Allow a user to Ctrl+C when running the analyze without corrupting the database.
• Fix bug where current working directory was used instead of root from the configuration file which lead to "strange behavior" when analysing files.
• NOMUT should not count as killed.
• Fix the algorithm for how the timeout is incrementally increased to allow multiple instances of dextool to better cooperate.
• Limit the number of iterations of mutants that are marked as timeout to three. This should help in those circumstances wherein there are many timeout mutants that continuously "reset" the iterator.

Continuous Flow

• fix livelock when a test suite do not end its output to stdout with a newline

Continuous Flow

• Fix livelocks in the process handling when executing test commands.

Continuous Flow

This release focus on improving the integration in a CI workflow.

New features:

• Speed-up mutation testing workflow by allowing the user to use a diff to direct dextool to what files to analyse and test. The diff can be from git diff.
• Allow a user to speed-up the analyze phase when saving to the database by reducing the SQLite safety settings (--fast-db-store)
• Speed-up saving the analyze result to by only saving those files that have changed.
• Speed-up the analyzer by parallelising it.
• Allow the user to specify multiple test_cmd that is executed in parallel.
• It is now possible to specify a directory test_cmd_dir which dextool will scan, after compilation, for binaries. These binaries are added as test_cmds.
• Allow a user to specify a max runtime when testing mutants. This is to be able to give "fast" feedback on a pull request.
• Add a report section which display what mutants a test case uniquely kill and those that has no unique kills.
• Allow a user to exclude files from analysis.
• The admin command group now has an option to mark mutants with a specific status. This is for the users that can't modify the source code to add // NOMUT on lines with mutants that should be ignored.
• Which mutants that are tested in a diff is deterministic (always the same over a week) to make it possible for a user to e.g. add or change test cases to kill mutants in the diff.

Breaking change:

• test_cmd, build_cmd and analyze_cmd in the configuration file now requires that the user write the path to the command. Such as ./build.sh.

Fixes:

• Only write the output from test cases to the filesystem when the user has specified an external analyze_cmd. The builtins no longer requires this temporary storage.
• A compilation_commands.json is only required when generating a report if the report contains sections which requires it.
• Repair the short term view and rename it to diff view.
• Only reset and test old mutants when there is "time left" to do so.
• Fix bug when parsing compilation_commands.json when paths contain whitespaces.
• Allow a user to Ctrl+C when running the analyze without corrupting the database.
• Fix bug where current working directory was used instead of root from the configuration file which lead to "strange behavior" when analysing files.
• NOMUT should not count as killed.
• Fix the algorithm for how the timeout is incrementally increased to allow multiple instances of dextool to better cooperate.

Continuous Flow

This release focus on improving the integration in a CI workflow.

New features:

• Speed-up mutation testing workflow by allowing the user to use a diff to direct dextool to what files to analyse and test. The diff can be from git diff.
• Allow the user to specify multiple test_cmd that is executed in parallel.
• It is now possible to specify a directory test_cmd_dir which dextool will scan, after compilation, for binaries. These binaries are added as test_cmds.
• Allow a user to specify a max runtime when testing mutants. This is to be able to give "fast" feedback on a pull request.
• Add a report section which display what mutants a test case uniquely kill and those that has no unique kills.
• Allow a user to exclude files from analysis.
• The admin command group now has an option to mark mutants with a specific status. This is for the users that can't modify the source code to add // NOMUT on lines with mutants that should be ignored.

Breaking change:

• test_cmd, build_cmd and analyze_cmd in the configuration file now requires that the user write the path to the command. Such as ./build.sh.

Fixes:

• Only write the output from test cases to the filesystem when the user has specified an external analyze_cmd. The builtins no longer requires this temporary storage.
• A compilation_commands.json is only required when generating a report if the report contains sections which requires it.
• Repair the short term view and rename it to diff view.
• Only reset and test old mutants when there is "time left" to do so.
• Fix bug when parsing compilation_commands.json when paths contain whitespaces.
• Allow a user to Ctrl+C when running the analyze without corrupting the database.
• Fix bug where current working directory was used instead of root from the configuration file which lead to "strange behavior" when analysing files.
• NOMUT should not count as killed.
• Fix the algorithm for how the timeout is incrementally increased to allow multiple instances of dextool to better cooperate.

Liu Stable

Release verified to work with the lab material for the Liu course TDDD04.

Pretty Pictures

This release focus on the mutation testing plugin and the html report that is produced.

Mutation testing features:

• Produce a tree map in the html view over the directory/files to visualize the mutation score
• Update the html view of individual files
• Enrich the view of a mutant in a file with the test cases that killed it.

Dancing Fox

The plugin mutate has been updated to:

• reuse data between mutants that introduce the same source code change.
• gather and report statistics about individual test cases in a test suite.
  • find tests that are ineffective (do not kill any mutants).
  • find tests that fully overlap in their testing.

This release marks the mutate plugin as ready to use in production.

The Sad Panda

This release introduces a new plugin architecture akin to how git operates.

Feature

• Plugins are separate executables discovered at runtime.
• C/C++ testdouble plugin. Make it easier to grep for the dextool version in generated files by always prefixing with DEXTOOL_VERSION:.
• C/C++ testdouble plugin. New magic keyword $dextool_version$ can be used in custom headers.
• GraphML plugin. Add static as typeAttr to global variables to make it easier to find globals that are NOT local to a translation unit.

Bug Fixes

• C/C++ testdouble plugin. Header files can occur multiple times in the generated #includes in the generated code.
• C/C++ testdouble plugin. Fix -h to be more consistent with 80 columns.
• GraphML plugin. Add fallback nodes to ensure a valid graphml file is generated.