Skip to content

Commit

Permalink
Add an example for parameter definition re-use
Browse files Browse the repository at this point in the history
  • Loading branch information
@jaccz
jaccz committed Nov 6, 2016
1 parent fe1be29 commit 69320f9
Showing 1 changed file with 51 additions and 59 deletions.
110 changes: 51 additions & 59 deletions doc/pict.md
View file
Expand Up @@ -88,7 +88,7 @@ Constraints allow you to specify limitations on the domain. In the example with

##Conditional Constraints

A term **[parameter] relation value** is an atomic part of a predicate. The following relations can be used: =, <>, >, >=, <, <=, and LIKE. LIKE is a wildcard-matching operator (* - any character, ? – one character).
A term **[parameter] relation value** is an atomic part of a constraint expression. The following relations can be used: =, <>, >, >=, <, <=, and LIKE. LIKE is a wildcard-matching operator (* - any character, ? – one character).

[Size] < 10000
[Compression] = "OFF"
Expand All @@ -99,7 +99,7 @@ Operator IN allows specifying a set of values that satisfy the relation explicit
IF [Cluster size] in {512, 1024, 2048} THEN [Compression] = "Off";
IF [File system] in {"FAT", "FAT32"} THEN [Compression] = "Off";

The IF, THEN, and ELSE parts of a predicate may contain multiple terms joined by logical operators: NOT, AND, and OR. Parentheses are also allowed in order to change the default operator priority:
The IF, THEN, and ELSE parts of an expression may contain multiple terms joined by logical operators: NOT, AND, and OR. Parentheses are also allowed in order to change the default operator priority:

IF [File system] <> "NTFS" OR
( [File system] = "NTFS" AND [Cluster size] > 4096 )
Expand All @@ -114,16 +114,16 @@ Parameters can also be compared to other parameters, like in this example:
#
# Machine 1
#
OS_1: Win2000, WinXP
SKU_1: Professional, Server, Datacenter, WinPowered
LANG_1: EN, DE
OS_1: Win7, Win8, Win10
SKU_1: Home, Pro
LANG_1: English, Spanish, Chinese

#
# Machine 2
#
OS_2: Win2000, WinXP
SKU_2: Professional, Server, Datecenter
LANG_2: EN, DE
OS_2: Win7, Win8, Win10
SKU_2: Home, Pro
LANG_2: English, Spanish, Chinese, Hindi

IF [LANG_1] = [LANG_2]
THEN [OS_1] <> [OS_2] AND [SKU_1] <> [SKU_2];
Expand Down Expand Up @@ -159,12 +159,32 @@ String comparison is lexicographical and case-insensitive by default. Numerical

By default, PICT does all its comparisons and checks case-insensitively. For instance, if there are two parameters defined: *OS* and *os*, a duplication of names will be detected (parameter names must be unique). Constraints are also resolved case-insensitively by default:

IF [OS] = "Win2K" THEN ...
IF [OS] = "Win10" THEN ...

will match both *Win2K* and *win2k* values (values of a parameter are not required to be unique). Option **/c** however, makes the model evaluation fully case-sensitive.
will match both *Win10* and *win10* values (values of a parameter are not required to be unique). Option **/c** however, makes the model evaluation fully case-sensitive.

#Advanced modelling features
#Advanced Modelling Features

##Re-using Parameter Definitions

Once a parameter is defined, it can help in defining other parameters.

#
# Machine 1
#
OS_1: Win7, Win8, Win10
SKU_1: Home, Pro
LANG_1: English, Spanish, Chinese

#
# Machine 2
#
OS_2: <OS_1>
SKU_2: <SKU_1>
LANG_2: <LANG_1>, Hindi

Less typing and better maintainability.

##Sub-Models

Sub-models allow the bundling of certain parameters into groups that get their own combinatory orders. This can be useful if combinations of certain parameters need to be tested more thoroughly or must be combined in separation from the other parameters in the model. The sub-model definition has the following format:
Expand All @@ -173,13 +193,13 @@ Sub-models allow the bundling of certain parameters into groups that get their o

For example, sub-modeling is useful when hardware and software parameters are combined together. Without sub-models, each test case would produce a new, unique hardware configuration. Placing all hardware parameters into one sub-model produces fewer distinct hardware configurations and potentially lowers the cost of testing. The order of combinations that can be assigned to each sub-model allows for additional flexibility.

PLATFORM: x86, ia64, amd64
CPUS: Single, Dual, Quad
RAM: 128MB, 1GB, 4GB, 64GB
PLATFORM: x86, x64, arm
CPUS: 1, 2, 4
RAM: 1GB, 4GB, 64GB
HDD: SCSI, IDE
OS: NT4, Win2K, WinXP, Win2K3
IE: 4.0, 5.0, 5.5, 6.0
APP: SQLServer, Exchange, Office
OS: Win7, Win8, Win10
Browser: Edge, Opera, Chrome, Firefox
APP: Word, Excel, Powerpoint

{ PLATFORM, CPUS, RAM, HDD } @ 3
{ OS, IE } @ 2
Expand All @@ -203,43 +223,15 @@ Notes:

##Aliasing

Aliasing is a way of specifying multiple names for one value. Multiple names do not change the combinatorial complexity of the model. No matter how many names a value has, it is treated as one entity. The only difference will be in the output; any test case that would normally have that one value will have one of its names instead. Names are rotated among the test cases. One practical application can be shown by the following example: In most of the cases, the difference between Windows 2000 Server and Windows 2000 Advanced Server can safely be ignored, so a tester may opt to test just one version, thereby reducing the number of test cases. However, for sanity sake, both versions could be tested interchangeably. Specifying one value with two names will result in having either Server or Advanced Server in the output without adding extra test cases.
Aliasing is a way of specifying multiple names for a single value. Multiple names do not change the combinatorial complexity of the model. No matter how many names a value has, it is treated as one entity. The only difference will be in the output; any test case that would normally have that one value will have one of its names instead. Names are rotated among the test cases. Specifying one value with two names will result in having them both show up in the output without additional test cases.

By default, names should be separated by | character but this can be changed with option **/a**.

#
# Machine 1
#
OS_1: Win2000, WinXP
SKU_1: Professional, Server|AdvServer, Datacenter, WinPowered
LANG_1: EN, DE

#
# Machine 2
#
OS_2: Win2000, WinXP
SKU_2: Professional, Server|AdvServer, Datecenter
LANG_2: EN, DE

#
# WinXP is always Professional in our case
#
if [OS_1] = "WinXP" then [SKU_1] = "Professional";
if [OS_2] = "WinXP" then [SKU_2] = "Professional";

#
# No German WinPowered
#
if [SKU_1] = "WinPowered" then [LANG_1] = "EN";

#
# Let’s not test the same OS on both sides
#
[OS_1] <> [OS_2];

Notes:
OS_1: Win2008, Win2012, Win2016
SKU_1: Professional, Server | Datacenter

1. When evaluating constraints, only the first name is evaluated. For instance, **[SKU_1] = "Server"** will match the second value but **[SKU_1] = "AdvServer"** will match nothing. Also, only the first name is used to determine whether a value is negative or a numeric type.
Note:
When evaluating constraints, only the first name counts. For instance, **[SKU_1] = "Server"** will result in a match but **[SKU_1] = "Datacenter"** will not. Also, only the first name is used to determine whether a value is negative or a numeric type.

##Negative Testing

Expand Down Expand Up @@ -303,7 +295,7 @@ The generation mechanism can be forced to prefer certain values. This is done by
Format method: quick, slow
File system: FAT, FAT32, NTFS (10)

Important note:
Note:
Weight values have no intuitive meaning. For example, when a parameter is defined like this:

File system: FAT, FAT32, NTFS (10)
Expand All @@ -312,7 +304,7 @@ it does not mean that NTFS will appear in the output ten times more often than F

The reason for this is that we deal with two contradictory requirements:
1. To cover all combinations in the smallest number of test cases.
2. To choose values according to their weights.
2. To choose values proportionally to their weights.

(1) will always take precedence over (2) and weights will only be honored when the choice of a value is not determined by a need of satisfying (1). More specifically, during the test case production, candidate values are evaluated and one that covers most of the still unused combinations is always picked. Sometimes there is a tie among candidate values and really no choice is better than another. In those cases, weights will be used to determine the final choice.

Expand All @@ -331,9 +323,9 @@ Seeding rows must be defined in a separate file (a seeding file). Use new option
Seeding files have the same format as any PICT output. First line contains parameter names separated by tab characters and each following line contains one seeding row with values also separated by tabs. This format can easily be prepared from scratch (scenario 1) either in Notepad or in Excel and also allows for quick and direct reuse of any prior results (scenario 2).

Ver SKU Lang Arch
Win2k Pro EN x86
Win2k DE x86
WinXP Pro EN ia64
Win7 Pro EN x86
Win7 FR x86
Win10 Pro EN x64

Any seeding row may be complete i.e. with values specified for all parameters, or partial, like the second seeding row above which does not have a value for the SKU parameter. In this case, the generation process will take care of choosing the best value for SKU.

Expand All @@ -346,12 +338,12 @@ There are a few rules of matching seeding rows with the current model:

PICT will issue warnings if (1) or (2) occurs.

In addition to that, there are a few things that are normally allowed in PICT models buy may lead to ambiguities when seeding is used:
1. Blank parameter and value names.
In addition, there are a few things that are normally allowed in PICT models but may lead to ambiguities when seeding is used:
1. Blank parameter and value names.
2. Parameter and value names containing tab characters.
3. Value names and all their aliases not unique within a parameter.
3. Values and all their aliases not unique within a parameter.

When seeding is used, you will be warned if any of those problems exist in your model.
When seeding is used, you will be warned if any of the above problems are detected in your model.

# Constraints Grammar

Expand Down

0 comments on commit 69320f9

Please sign in to comment.