Skip to content

Commit

Permalink
Update sample scenarios with additional clarity and fallbacks.
Browse files Browse the repository at this point in the history
The TESTING ontology works to capture a broad range of testing considerations;
not all of those considerations are apparent or available when ingesting data, so
the scenarios now describe some of the fallback representations when less data is
available.
  • Loading branch information
kquick committed Apr 4, 2023
1 parent 7762b41 commit 2d417be
Showing 1 changed file with 24 additions and 7 deletions.
31 changes: 24 additions & 7 deletions RACK-Ontology/Orienteering.md
Original file line number Diff line number Diff line change
Expand Up @@ -247,9 +247,8 @@ the TESTING.sadl file provides a more complete specification of TESTING elements
The software written during the Program is intended to satisfy a particular
requirement "Req1" (among others). The software is built via the documented
build process and the resulting artifact of the build is an executable,
identified by Target name and version. The software must be verified to meet
Req1, so 5 requirements-based TESTs are developed (the TESTs 'wasGeneratedBy' a
TEST_DEVELOPMENT), and the 'verifies' target of each is the "Req1" REQUIREMENT.
identified by Target name and version. This isn't strictly part of the testing
ontology, this is some information from the software development ontology:

REQ1 is a REQUIREMENT has identifier "Req1".

Expand All @@ -263,6 +262,12 @@ TEST_DEVELOPMENT), and the 'verifies' target of each is the "Req1" REQUIREMENT.
Bld1 has used MakeFile1 has compileInput SOURCE1 has goal TARGET1.
TARGET1 is a FILE.

The software must be verified to meet
Req1, so 5 requirements-based TESTs are developed (the TESTs 'wasGeneratedBy' a
TEST_DEVELOPMENT), and the 'verifies' target of each is the "Req1" REQUIREMENT.

The test development could be captured by another software development dataset:

TDEV1 is a TEST_DEVELOPMENT.
TDEV1 has goal TestSource1.
TestSource1 is a FILE.
Expand All @@ -271,6 +276,10 @@ TEST_DEVELOPMENT), and the 'verifies' target of each is the "Req1" REQUIREMENT.
TestBld1 has compileInput TestSource1 has goal TestExe.
TestExe is a FILE.

Describing the 5 tests within the TESTING ontology portion and--since we
documented the `TEST_DEVELOPMENT` above--linking them back to their development
portion (which is not always present):

TEST1 is a TEST has identifier "Test1".
TEST2 is a TEST has identifier "Test2".
TEST3 is a TEST has identifier "Test3".
Expand All @@ -291,7 +300,9 @@ TEST_STEPs ("Step1", "Step2", "Step3"), where "Step1" 'thisStep' is "Test1",
"Step2" 'thisStep' is "Test2", and "Step3" 'thisStep' is "Test3". There are two
more TEST_STEPs ("Step3b" and "Step3c"), where "Step3" has 'nextStep' to "Step3b"
(which has 'thisStep' to "Test4"), and "Step3b" has 'nextStep' to "Step3c" (which
has 'thisStep' to "Test5").
has 'thisStep' to "Test5"). Note that it is not always apparent which tests are
dependent on other tests; the default approach is to simply assume all tests are
independent until otherwise known, as is described in this scenario.

VR1 is a TEST_PROCEDURE has identifier "VerifyReq1".
Tstep1 is a TEST_STEP has identifier "Step1" has thisStep TEST1.
Expand Down Expand Up @@ -377,10 +388,9 @@ TEST_RESULT for "Test2".
TestResultAnalysis is an ACTIVITY.
TestResultAnalysis has wasAssociatedWith JoeTest has goal Memo.

Memo is an TEST_ANNOTATION has wasGeneratedBy TestResultAnalysis.
Memo is a TEST_ANNOTATION has wasGeneratedBy TestResultAnalysis.
Memo has annotatedResult TRES2.
IgnoredFailureOOB is a ENTITY has wasGeneratedBy TestResultAnalysis.
Memo has testAnnotation IgnoredFailureOOB.
Memo has annotatedValue IgnoredFailureOOB.


## Scenario 2
Expand Down Expand Up @@ -504,6 +514,13 @@ activity.
Flopped2 is a TEST_RESULT has wasGeneratedBy Run2 has confirms Flop.
Flopped2 has result Passed.

Another note to make here is that it's not always the case that this type of
incremental development process is captured in the data provided to RACK. The
minimum scenario is that there is a `TEST_RECORD` that fully matches the
`TEST_SCENARIO` and contains a `TEST_RESULT` that confirms every `TEST`; the
additional information captured in the above multi-stage scenario is enhanced
information that is not always available.


---- temporary below --------------------------------------------------

Expand Down

0 comments on commit 2d417be

Please sign in to comment.