Review of Evaluator Truth Tables (of 5 September)

[nitmws]

This is a review of the Truth Tables of an ODRL Evaluator as shown in https://www.w3.org/2016/poe/wiki/Evaluator - version of 5 September:

What is stated exactly by Not-/Active - see #245 - has not been clarified yet, my assumption from how it is used in the Truth Tables:

• Active = can be used for further evaluation or other ODRL processing
• Not-Active = cannot be used ...

For the three sub-classes of Rule this translates into

• Permission: if Active the action of the Permission may be exercised
• Prohibition: I can't draw a direct conclusion because of the very few Prohibitions in the examples. See comment on example 24
• Duty: the state Active is required as prerequisite for stating a Duty has been Not-/Fulfilled, this depends on taking the action of the Duty. By that three states of Duty have to be considered by a Rule referring to it: Fulfilled, Not-Fulfilled, Not-Active

I comment only on Truth Tables I have a different view on - not-commented tables are ok.

• Example 16 + 17: why does the Evaluator evaluate the target and the assignee? This is not done for any other example. And the shown conclusion is not easy to follow: to be able to evaluate the state Not-/Satisfied of the refinement in example 17 it is required to have a real entity to be matched against this refinement and this requires to match the entity first against the PartyCollection. (Similar issue with example 16) A Truth Table about such examples needs to include the state of a "is the entity a member of the Collection" evaluation.
• Example 20: An Obligation is an instance of a Duty Class and the full evaluation state of a Duty is: Not-/Fulfilled. The shown Truth Table does not take the action into account. But this should be done by the Duty Class specification: https://w3c.github.io/poe/model/#duty
• Example 21: columns with a blank header need one defining what the state in the rows below is about.
  The second column shows Not/Fulfilled states. It looks like the state of the Obligation as Duty without taking the consequence into account. Conclusion: if an Obligation (=Duty) is Not-Fulfilled (= the action has not been exercised) after this first round of evaluation a second round is entered to evaluate the consequence too.
  And: where is the final state of the Obligation - after having taken into account consequences, if required?
  • the 1st row shows Cq1=Not Active: by what evaluation has the consequence been de-activated? But: having exercised the action of a Duty does not require to evaluate any existing consequences, therefore the Cq1 value is not relevant for the evaluation of the Obligation/Duty.
  • The 3rd row shows that Cq1 is Active despite of a Not-Satisfied refinement R1 - this is wrong.
  • The 4th and the 5th row show if fact the same as row 1 - but now the refinement is taken into account. As said above: if the action of a Duty has been exercised - and this is reflected by the Fulfilled state in the 2nd column - consequences don't need to be evaluated. Therefore these two rows are misleading. (In fact the final state of Obligation must be Fulfilled for both rows.)
• Example 22: (only match the JSON-LD in the IM with the Truth Tables - regardless of issues with the semantics)
  This example shows related/referenced Rules: the Permission Rule references a Duty Rule by a duty property. The rule for evaluating constraints and refinements of a Rule is therefore: this evaluation needs to be done independently for the constraints of the Permission and the refinements of its action AND for the constraints of the Duty and the refinements of its action!
  Conclusion: C1 and R1 have a direct impact on D1 only, and only the state of D1 has an impact on P1
  Going over the Truth Tables - guessing the column with the blank header is the state of D1
  • row 1 + 2: ok
  • row 3: if R1 is Not-Satisfied, Duty D1 cannot be Active.
  • row 4: C1 is shown as Not-Satisfied. Why does R1 show no state? Because of the Rule "if any consequence or refinement is Not-Satisfied the consequence+remedy evaluation results in Not-Satisfied"? The other cells of the row are ok.
• Example 23: re the columns with blank headers: see Example 21 comments above.
  • row 2: why is P1 not Active, the duty has been finally fulfilled by Fulfill-ing the consequence
  • row 4: why does it show states for Cq1 and the next column as Cq1 is not relevant, the action of the Duty has been exercised.
  • row 5: if the action of a Duty has been exercised it is not required to fulfill a consequence - a Not-Fulfilled consequence cannot set a Permission to Not-Active in this case.
• Example 24: a Truth Table has to include if the action of a Prohibition has been exercised or not as this is the trigger of the relevance of a remedy.
  • row 3: why does exercising the disallowed action and then fulfilling the remedy set the Prohibition to Not-Active? It must be the same state as "the disallowed action has not been exercised" - but this state is not shown in the Tables and the semantics of "a Prohibition is Active" (or ... Not-Active) are not defined.
• Example 25: the headings show an issue: at first sight C1 and C2 are constraints side by side, but they aren't: C2 is a constraint on the Duty and not on the Permission. Suggestion: use "D1.C2" and "D1.R1".
  • row 3 + 7: R1 of D1 is Not-Satisfied, therefore D1 must be Not-Active

[benedictws]

Hi Michael. Thanks for the feedback.

Some of your concerns can, I think, be assuaged by being clearer about the scope of the evaluator.

The evaluator ensures that the flow of logic encapsulated in the IM has been correctly followed. It does no more than that.

It cannot decide whether a constraint has been satisfied or a duty fulfilled. The world is simply a black box to the evaluator.

What it can do though is decide whether a rule is active or not-active depending on the states reported to it: satisfied/not satisfied; fulfilled/not fulfilled.

This provides a level of interoperability between implementations beyond simply validity. But it does not insist that two implementations share the same view of the world. It has no view of the world!

What we're providing here are a set of expected results to which an implementation should conform. If it does so, then we know it's logical flow is consistent with the standard.

We then know that two different implementations that happen to agree on the state of the world will also agree which rules are active when interpreting the same policy.

That's the best we can do.

And that's why the evaluator does not check against real entities. It's only told the states of rules and constraints.

In Example 21 I wish I didn't have a blank header - the obligation and the consequence both have two states, so require two columns to describe them fully. Are they active or not-active, and, if they are active, are they fulfilled or not.

In row one the consequence is not active because the obligation has been fulfilled.

In row three the obligation has not been fulfilled, triggering the consequence. It's constraint has not been satisfied, so the consequence has not been fulfilled either.

The fourth and fifth rows differ from row one because the consequence has been triggered.

Maybe the tables need more commentary?

In Example 22, row 3, only C1 can determine whether the duty is active or not. R1 simply has to be satisfied before D1 can be fulfilled. It has no influence on the active status of D1.

In row 4, if C1 is not satisfied then D1 is not active, and we don't care about the status of R1. It has no effect.

In Example 23, row 2, P1 is not active because its duty D1 has not yet been fulfilled, even though its consequence has been. Inherent in the logic of consequence is the idea that its original duty must be fulfilled too.

That also explains rows 4 and 5: once the consequence has been trigged both it, and the original duty, must be fulfilled.

Again, more commentary needed?

Example 24 - ha! This is where these status tables really uncover the true semantics of the model. I'd say this was up for debate!

Example 25 - suggestion gladly taken. Hope the tables are a little clearer now.

Rows 3 and 7: refinements do not effect the active status of a rule. They speak only to fulfilment.

Phew! I think this shows we have a consistent semantics that implementors can test against ... except possibly for the issue raised in Example 24.