Provide a MODEL concept for attaching different interpretations of RACK data

[glguy]

@AbhaMoitra @russell-d-e I'd like your take on this approach!

I propose to separate out the concept of a project REQUIREMENT and the interpretation of the statements one makes. I think we can almost definitely say what a requirement is from the point of view of a particular project. The various teams are interested in going further and providing interpretations of the semantics of these requirements. This approach would allow has to capture these different interpretations in an extensible way while not trying to subclass the concept of the requirement itself.

The raw natural language text of a requirement can be stored in the description field we already have. Any understanding of that requirement can be tracked in a REQUIREMENT_STATEMENT linked by states. These names are up for discussion, of course.

I think this division of requirement from interpretation helps make our current ifText givenText thenText properties make much more sense as we can link them much closer to the confidence parameters GrammaTech wanted.

In the future we can add other statement subtypes for other interesting formal languages. I'm imagining something like:

TEMPORAL_LOGIC_STATEMENT
    is a type of REQUIREMENT_STATEMENT
    described by logicalEnvironment with values of type string
    described by logicalExpression with values of type string
    described by supportPackage with values of type string.

The idea being that these statements could have a lot more structure to them if that was appropriate. By not merely subclassing REQUIREMENT for each of these we allow different tools and teams to interpret a single REQUIREMENT with multiple statements in different languages without duplicating the requirement.

Thoughts?

[AbhaMoitra]

I like the approach - can you expand on what exactly "REQUIREMENT_STATEMENT linked by states" would look like. This is the critical link to REQUIREMENT.

In your TEMPORAL_LOGIC_STATEMENT example, should there be a link to the REQUIREMENT?

Another discussion point was different types of requirements and requirement decomposition.

[russell-d-e]

In general I like the approach. I can't help but wonder could we make it a little bit more non-specific. What I mean is should we add a concept of a MODEL that can be used to describe an THING something along the lines of:

MODEL (note "a physical, mathematical, or otherwise logical representation of a system, entity, phenomenon, or process")is a ENTITY.
models describes MODEL with values of type THING.
confidence describes MODEL with values of type float.

then when you would be able to assign the confidence in the model. The would allow the same basic constructs to be used for other types of modeling that people may be doing. Part of what I hang-up a bit with this is the idea of having a confidence value in any thing with REQUIREMENT in its name, a requirement should be absolute truth, if you don't have full confidence in your requirements how can your testing or implementation possibly going be objective. By making it clear that you are doing your analysis on a model of the requirement it would make sense to have a confidence. This same basic approach could be used when modeling other things that you may performing analysis on, for example an AADL model of a SYSTEM you may perform timing or bandwidth analysis. At the ontology extension level I would see something like:

AcertRequirementStatement is a type of MODEL.
    models describes AcertRequirementStatement with values of type REQUIREMENT.

    textConfidence (note "confidence that this was requirement text")
        describes AcertRequirementStatement with values of type double.
	
	ifText (note "conditions under which this requirement holds")
	    describes AcertRequirementStatement with values of type string.
    ifTextConfidence (note "confidence that this was the if-portion of the text")
        describes AcertRequirementStatement with values of type double.

	thenText (note "specifies what should hold by this requirement if the requirement conditions are met")
		describes AcertRequirementStatement with values of type string.    
    thenTextConfidence (note "confidence that this was the then-portion of the text")
        describes AcertRequirementStatement with values of type double.
	
	givenText (note "optional part that can be used to set certain conditions apart from other conditions in the ifText part")
		describes AcertRequirementStatement with values of type string.
    givenTextConfidence (note "confidence that this was the given-portion of the text")
        describes AcertRequirementStatement with values of type double.

StrEarsStatement (note "A requirement statement written in the Easy Approach to Requirements Syntax language.")
    is a type of MODEL.
    models describes StrEarsStatement with values of type REQUIREMENT.
    
    earsStatement (note "Text of statement written in the EARS requirements language.")
        describes StrEarsStatement with values of type string.

As you can see from above I also would suggest reversing the direction of the connection between the REQUIREMENT and REQUIREMENT_STATEMENT (or MODEL if we chose to go that way.) This way you could have multiple REQUIREMENT_STATEMENTs (or MODELs) relating back to the same requirement in the event that multiple TA1 have different ways to model the same Boeing requirement (i.e. you may model the same requirement modeled in multiple ways depending on the different analysis that you are doing, one model may be for requirement analysis while a different model may be for test generation).

[glguy]

Yeah, reversing the link makes a lot of sense. The requirement stands on its own and the models are modeling a particular requirement. I'll fix that up now.

Generalizing out to MODEL allows us to model all sorts of rack entities and saves a lot of potential duplication!

[glguy]

@russell-d-e Are you thinking MODEL would relate to a thing because we might not only model and entity, but also an ACTIVITY?

As general as MODEL is going to be, should it live in our PROV-S, or in its own top-level ontology file. It seems to me like it actually fits in the core PROV-S, even though it's not derived from PROV itself, but just because it becomes so core the the way to capture information. Opinions?

[glguy]

@AbhaMoitra I like your MODEL but I think the notion of confidence should live on the subclasses of MODEL rather than on the top-level class itself. I can't imagine that confidence numbers would be comparable between different subclasses. In the case of GrammaTech's requirement model they have many confidences. Other models might not have a notion of confidence, and those that do might use different scales.

[russell-d-e]

@russell-d-e Are you thinking MODEL would relate to a thing because we might not only model and entity, but also an ACTIVITY?

As general as MODEL is going to be, should it live in our PROV-S, or in its own top-level ontology file. It seems to me like it actually fits in the core PROV-S, even though it's not derived from PROV itself, but just because it becomes so core the the way to capture information. Opinions?

I did use THING thinking that at some point someone may want to model an ACTIVITY, I didn't think to hard about it but my initial thought was that might be useful so use THING to keep the option open.

My thinking was that it would be in its own top-level ontology file. Just like we have REQUIREMENTS and TESTING. We would add a MODELING file. I hadn't thought of including it into the PROV-S file, In general I was thinking we would want to keep that file matching PROV-S as much as possible, so I guess I would ask what benefits would you see including it into PROV-S rather than in a separate file?

[glguy]

I think having it as a top-level sadl file makes the most sense having thought about it.

[hbr17]

Also see email: (hbr17 = Howard Reubenstein)

I think this is largely an issue of the physical organization of the Rack as given this representation style we can define queries or utilities like the following:
access-requirements-doc() à String
access-requirements-ears-model() à earsStatement or null

One thing I see missing from the proposal is a way to interrogate what models for a requirement exist, e.g.,
access-models() à set(MODEL)

(I see the link from each model to the requirements, but do we need a link from the requirement to its set of models?
or can this be done by a query: access-models() = all(x:Model) | x.models =

Before implementing this change I would strongly suggest providing some notional pseudo-code showing how this feature would be used by the TA3s. (To evaluate API proposals – and this is sort of an API proposal – I always like to see sample code that shows how the API might be used).

An example notional task I would like to see pseudo-code for is: Given a reference to a requirement by UID how would I access its preconditions(assumes) clause.

[glguy]

One thing I see missing from the proposal is a way to interrogate what models for a requirement exist, e.g.,
access-models() à set(MODEL)

The new proposal adds no new complexity in this case. Whether we are using subclasses of REQUIREMENT or subclasses of MODEL we'll have to have a nodegroup query to list the generic thing and its type to help us decide what we want to do. We can make a query that lists all the models and the type of the model associated with any particular piece of data. This would be similar to how we'd need to have a nodegroup that lists the subtype of a requirement to know how to access the fields of its subtypes.

TA3s will need advanced knowledge of the different models available in the dataset in order to do anything interesting with them. There should be no surprise MODEL subtypes to discover. We'll want anyone generating a new model type to carefully document the meaning of the model so that a consume of the model can productively use it while building an assurance case.

[glguy]

One thing I see missing from the proposal is a way to interrogate what models for a requirement exist, e.g.,
access-models() → set(MODEL)

This would be done through any nodegroup with a models edge in it.

Here's a nodegroup that has a runtime constraint on the model identifier (not indicated in the picture) that returns all the models associated with a particular requirement along with the type of that model so that you can request more specific information.

An alternative would be what was suggested above in writing a union query to select any of the kinds of models that you have code to handle.

[glguy]

(I see the link from each model to the requirements, but do we need a link from the requirement to its set of models?
or can this be done by a query: access-models() = all(x:Model) | x.models =

Relations live independently of the class they happen to be be described by in the SADL file. There's no query power changed by moving the relation from one file to the other. The directionality of a query like the nodegroup above comes from specifying either the REQUIREMENT or the MODEL identifier as the runtime constraint.

[glguy]

An example notional task I would like to see pseudo-code for is: Given a reference to a requirement by UID how would I access its preconditions(assumes) clause.

Here's a nodegroup that queries the Acert-extracted ifTexts for a particular requirement given a runtime constraint on the requirement's identifier. (obviously the same would work for the givenText or thenText)

I don't think it's fair to assume, in general, that all models have a distinguished assumes field, so you'd probably have to know what kind of model you were looking at to know this. I would expect that for a complex requirement that there might not be a single assumption that covers all behaviors of the requirement. A more interesting requirement in a structured language might allow for a much richer logical implication or set of assumptions. I'm willing to be convinced that no requirements are this complex, however.

[glguy]

Dave wrote:

Howard has a great point here, which we knew was lurking: how does one query the data model efficiently. Even in SQL, and given a pre-defined set of model types, his query is not easy:

SELECT *
FROM Requirement NATURAL JOIN EARS_model
UNION
SELECT *
FROM Requirement NATURAL JOIN Acert_Requirement_model

In a graph database, the query would find the Requirement first and then follow all paths with inbound edges labeled "models".

How does this get expressed in the RACK query language?

We can write a query that pulls in optional matches from multiple subtypes in one shot. Here's the simplest example.

[ttrine]

Made a pass over this to check that I understand how it implements the functionality Eric described. All looks good to my eye.

[AbhaMoitra]

I reviewed most of the files and they look fine. The files I did not review are Thing.json and store_data.csv. Can you add a new line at the end of the files STR.sadl and Thing.json.