From 69320f9a7e7378dd640d32a85f630d72bdea98e5 Mon Sep 17 00:00:00 2001 From: Jacek Czerwonka Date: Sat, 5 Nov 2016 23:39:12 -0700 Subject: [PATCH] Add an example for parameter definition re-use --- doc/pict.md | 110 ++++++++++++++++++++++++---------------------------- 1 file changed, 51 insertions(+), 59 deletions(-) diff --git a/doc/pict.md b/doc/pict.md index 272e900..2326fb9 100644 --- a/doc/pict.md +++ b/doc/pict.md @@ -88,7 +88,7 @@ Constraints allow you to specify limitations on the domain. In the example with ##Conditional Constraints -A term **[parameter] relation value** is an atomic part of a predicate. The following relations can be used: =, <>, >, >=, <, <=, and LIKE. LIKE is a wildcard-matching operator (* - any character, ? – one character). +A term **[parameter] relation value** is an atomic part of a constraint expression. The following relations can be used: =, <>, >, >=, <, <=, and LIKE. LIKE is a wildcard-matching operator (* - any character, ? – one character). [Size] < 10000 [Compression] = "OFF" @@ -99,7 +99,7 @@ Operator IN allows specifying a set of values that satisfy the relation explicit IF [Cluster size] in {512, 1024, 2048} THEN [Compression] = "Off"; IF [File system] in {"FAT", "FAT32"} THEN [Compression] = "Off"; -The IF, THEN, and ELSE parts of a predicate may contain multiple terms joined by logical operators: NOT, AND, and OR. Parentheses are also allowed in order to change the default operator priority: +The IF, THEN, and ELSE parts of an expression may contain multiple terms joined by logical operators: NOT, AND, and OR. Parentheses are also allowed in order to change the default operator priority: IF [File system] <> "NTFS" OR ( [File system] = "NTFS" AND [Cluster size] > 4096 ) @@ -114,16 +114,16 @@ Parameters can also be compared to other parameters, like in this example: # # Machine 1 # - OS_1: Win2000, WinXP - SKU_1: Professional, Server, Datacenter, WinPowered - LANG_1: EN, DE + OS_1: Win7, Win8, Win10 + SKU_1: Home, Pro + LANG_1: English, Spanish, Chinese # # Machine 2 # - OS_2: Win2000, WinXP - SKU_2: Professional, Server, Datecenter - LANG_2: EN, DE + OS_2: Win7, Win8, Win10 + SKU_2: Home, Pro + LANG_2: English, Spanish, Chinese, Hindi IF [LANG_1] = [LANG_2] THEN [OS_1] <> [OS_2] AND [SKU_1] <> [SKU_2]; @@ -159,12 +159,32 @@ String comparison is lexicographical and case-insensitive by default. Numerical By default, PICT does all its comparisons and checks case-insensitively. For instance, if there are two parameters defined: *OS* and *os*, a duplication of names will be detected (parameter names must be unique). Constraints are also resolved case-insensitively by default: - IF [OS] = "Win2K" THEN ... + IF [OS] = "Win10" THEN ... -will match both *Win2K* and *win2k* values (values of a parameter are not required to be unique). Option **/c** however, makes the model evaluation fully case-sensitive. +will match both *Win10* and *win10* values (values of a parameter are not required to be unique). Option **/c** however, makes the model evaluation fully case-sensitive. -#Advanced modelling features +#Advanced Modelling Features +##Re-using Parameter Definitions + +Once a parameter is defined, it can help in defining other parameters. + + # + # Machine 1 + # + OS_1: Win7, Win8, Win10 + SKU_1: Home, Pro + LANG_1: English, Spanish, Chinese + + # + # Machine 2 + # + OS_2: + SKU_2: + LANG_2: , Hindi + +Less typing and better maintainability. + ##Sub-Models Sub-models allow the bundling of certain parameters into groups that get their own combinatory orders. This can be useful if combinations of certain parameters need to be tested more thoroughly or must be combined in separation from the other parameters in the model. The sub-model definition has the following format: @@ -173,13 +193,13 @@ Sub-models allow the bundling of certain parameters into groups that get their o For example, sub-modeling is useful when hardware and software parameters are combined together. Without sub-models, each test case would produce a new, unique hardware configuration. Placing all hardware parameters into one sub-model produces fewer distinct hardware configurations and potentially lowers the cost of testing. The order of combinations that can be assigned to each sub-model allows for additional flexibility. - PLATFORM: x86, ia64, amd64 - CPUS: Single, Dual, Quad - RAM: 128MB, 1GB, 4GB, 64GB + PLATFORM: x86, x64, arm + CPUS: 1, 2, 4 + RAM: 1GB, 4GB, 64GB HDD: SCSI, IDE - OS: NT4, Win2K, WinXP, Win2K3 - IE: 4.0, 5.0, 5.5, 6.0 - APP: SQLServer, Exchange, Office + OS: Win7, Win8, Win10 + Browser: Edge, Opera, Chrome, Firefox + APP: Word, Excel, Powerpoint { PLATFORM, CPUS, RAM, HDD } @ 3 { OS, IE } @ 2 @@ -203,43 +223,15 @@ Notes: ##Aliasing -Aliasing is a way of specifying multiple names for one value. Multiple names do not change the combinatorial complexity of the model. No matter how many names a value has, it is treated as one entity. The only difference will be in the output; any test case that would normally have that one value will have one of its names instead. Names are rotated among the test cases. One practical application can be shown by the following example: In most of the cases, the difference between Windows 2000 Server and Windows 2000 Advanced Server can safely be ignored, so a tester may opt to test just one version, thereby reducing the number of test cases. However, for sanity sake, both versions could be tested interchangeably. Specifying one value with two names will result in having either Server or Advanced Server in the output without adding extra test cases. +Aliasing is a way of specifying multiple names for a single value. Multiple names do not change the combinatorial complexity of the model. No matter how many names a value has, it is treated as one entity. The only difference will be in the output; any test case that would normally have that one value will have one of its names instead. Names are rotated among the test cases. Specifying one value with two names will result in having them both show up in the output without additional test cases. By default, names should be separated by | character but this can be changed with option **/a**. - # - # Machine 1 - # - OS_1: Win2000, WinXP - SKU_1: Professional, Server|AdvServer, Datacenter, WinPowered - LANG_1: EN, DE - - # - # Machine 2 - # - OS_2: Win2000, WinXP - SKU_2: Professional, Server|AdvServer, Datecenter - LANG_2: EN, DE - - # - # WinXP is always Professional in our case - # - if [OS_1] = "WinXP" then [SKU_1] = "Professional"; - if [OS_2] = "WinXP" then [SKU_2] = "Professional"; - - # - # No German WinPowered - # - if [SKU_1] = "WinPowered" then [LANG_1] = "EN"; - - # - # Let’s not test the same OS on both sides - # - [OS_1] <> [OS_2]; - -Notes: + OS_1: Win2008, Win2012, Win2016 + SKU_1: Professional, Server | Datacenter -1. When evaluating constraints, only the first name is evaluated. For instance, **[SKU_1] = "Server"** will match the second value but **[SKU_1] = "AdvServer"** will match nothing. Also, only the first name is used to determine whether a value is negative or a numeric type. +Note: +When evaluating constraints, only the first name counts. For instance, **[SKU_1] = "Server"** will result in a match but **[SKU_1] = "Datacenter"** will not. Also, only the first name is used to determine whether a value is negative or a numeric type. ##Negative Testing @@ -303,7 +295,7 @@ The generation mechanism can be forced to prefer certain values. This is done by Format method: quick, slow File system: FAT, FAT32, NTFS (10) -Important note: +Note: Weight values have no intuitive meaning. For example, when a parameter is defined like this: File system: FAT, FAT32, NTFS (10) @@ -312,7 +304,7 @@ it does not mean that NTFS will appear in the output ten times more often than F The reason for this is that we deal with two contradictory requirements: 1. To cover all combinations in the smallest number of test cases. - 2. To choose values according to their weights. + 2. To choose values proportionally to their weights. (1) will always take precedence over (2) and weights will only be honored when the choice of a value is not determined by a need of satisfying (1). More specifically, during the test case production, candidate values are evaluated and one that covers most of the still unused combinations is always picked. Sometimes there is a tie among candidate values and really no choice is better than another. In those cases, weights will be used to determine the final choice. @@ -331,9 +323,9 @@ Seeding rows must be defined in a separate file (a seeding file). Use new option Seeding files have the same format as any PICT output. First line contains parameter names separated by tab characters and each following line contains one seeding row with values also separated by tabs. This format can easily be prepared from scratch (scenario 1) either in Notepad or in Excel and also allows for quick and direct reuse of any prior results (scenario 2). Ver SKU Lang Arch - Win2k Pro EN x86 - Win2k DE x86 - WinXP Pro EN ia64 + Win7 Pro EN x86 + Win7 FR x86 + Win10 Pro EN x64 Any seeding row may be complete i.e. with values specified for all parameters, or partial, like the second seeding row above which does not have a value for the SKU parameter. In this case, the generation process will take care of choosing the best value for SKU. @@ -346,12 +338,12 @@ There are a few rules of matching seeding rows with the current model: PICT will issue warnings if (1) or (2) occurs. -In addition to that, there are a few things that are normally allowed in PICT models buy may lead to ambiguities when seeding is used: - 1. Blank parameter and value names. +In addition, there are a few things that are normally allowed in PICT models but may lead to ambiguities when seeding is used: + 1. Blank parameter and value names. 2. Parameter and value names containing tab characters. - 3. Value names and all their aliases not unique within a parameter. + 3. Values and all their aliases not unique within a parameter. -When seeding is used, you will be warned if any of those problems exist in your model. +When seeding is used, you will be warned if any of the above problems are detected in your model. # Constraints Grammar