Skip to content
master
Switch branches/tags
Code

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
Jul 23, 2020
Oct 24, 2021
Oct 24, 2021
Jun 21, 2019
Oct 24, 2021
Jun 20, 2020

covertable logo

https://circleci.com/gh/walkframe/covertable.svg?style=shield

Time is limited.

It is not realistic to create a test case that satisfies all the multiple factors, and above all, it is tedious.

covertable is a flexible pairwise tool to create a two (or more) factor covered combination.

Now it has 2 implementations.

Python:
TypeScript:

Go see the detail from these links.

Performance

Note

  • The following data was measured in Python 3.7.7 and 3.1 GHz 6 Cores Intel Core i5.
  • coverage number is 2.
Number and time of combinations.
Combination Default Minimum case Fastest case
3^4
  • num: 9
  • cond: default
  • time: 0.0006s
  • num: 9
  • cond: default
  • time: 0.0006s
  • num: 14
  • cond: sorter: random, criterion: simple
  • time: 0.0005s
3^13
  • num: 19
  • cond: default
  • time: 0.03s
  • num: 17
  • cond: seed: 1084
  • time: 0.03s
  • num: 21
  • cond: sorter: random, criterion: simple
  • time: 0.003s
4^15 + 3^17 + 2^29
  • num: 36
  • cond: default
  • time: 7.41s
  • num: 34
  • cond: seed: 19
  • time: 7.47s
  • num: 42
  • cond: sorter: random, criterion: simple
  • time: 0.40s
4^1 + 3^39 + 2^35
  • num: 27
  • cond: default
  • time: 15.19s
  • num: 26
  • cond: seed: 14
  • time: 14.70s
  • num: 30
  • cond: sorter: random, criterion: simple
  • time: 0.51s
2^100
  • num: 14
  • cond: default
  • time: 23.97s
  • num: 12
  • cond: seed: 6, criterion: simple
  • time: 0.63s
  • num: 13
  • cond: sorter: random, criterion: simple
  • time: 0.48s
10^20
  • num: 198
  • cond: default
  • time: 14.28s
  • num: 195
  • cond: seed: 1139
  • time: 14.48s
  • num: 284
  • cond: sorter: random, criterion: simple
  • time: 0.53s

In general, as the number of elements or coverage increases, the number of combinations have a tendency to increase significantly.

Tolerance

If you use greedy criterion and specify a positive integer to tolerance option, it can increase the speed at the expense of the number of combinations.

The greater the tolerance, the shorter the speed and bigger the number of combinations.

Table for the case when combinations are created from 10^20 test cases.
tolerance num time
0 (default) 195 14.48s
1 199 12.45s
2 201 9.48s
3 201 7.17s
4 207 5.70s
5 212 4.58s
6 212 3.65s
7 216 3.07s
8 223 2.57s
9 226 2.14s
10 233 1.84s
11 237 1.61s
12 243 1.43s
13 249 1.28s
14 254 1.19s

History

2.0.x:
  • sorter option was splitted into sorter and criterion.
    • e.g. greedy -> hash sorter + greedy criterion.
  • greedy method is much faster than before.
  • greedy method got an option tolerance to balance speed and results.
  • sequenctial sorter was dropped.
    • Because The number of combinations would be huge in TypeScript.
1.1.x:
  • Greedy sorter improved in both implementations.
    • It got increased in speed.
1.0.x:
  • First release 🎉

Note

It moved from twopairs.