Find the pairs of norms that cannot both be obeyed

[tamnd]

Closes #35.

Question 19 asks which provisions contradict each other. The Cypher form of it compares a bearer and an action and says in its own comment that it will produce false positives, because two norms with the same bearer and action are usually distinguished by a condition it ignores. This is the pass that deletes that comment.

What the model does and what it does not do

No model is ever asked whether two norms conflict. A model asked that question answers it fluently in both directions and there is no way from the outside to tell a real contradiction from a plausible one.

So the strong model is a parser. It maps one trusted statement onto a comparable form, which is a party, an act, an object, a modality and the scope the norm applies in, and it never sees a second statement. Four rules then run in code over the pairs: an obligation against a prohibition, a permission against a prohibition, two deadlines on one duty, and two consequences for one act. Every finding carries the slots that had to agree and the slot where the pair parts, which is the minimal responsible set written out for a person. Lex superior, lex posterior and lex specialis sit beside a finding as ranking information and are never applied, because deciding which provision wins is not something to do to somebody without telling them.

Scope intersection is arithmetic and stays in code. Intervals are half open, so a version that ends the day its successor starts does not overlap it, and a norm that stands down to another instrument is pointing at it rather than fighting it.

Conditions are the hard half. Containment is sound and often silent, and a pair where neither condition set contains the other may describe one situation or may exclude itself. That gap was the entire error of the checker on the gold set, so those pairs, and only those pairs, go to a judge that is shown the two sets of circumstances and the party they are about, and nothing else. It never sees an operator, an act, a deadline, a rule or a quote, so it cannot be answering whether the two conflict. A judge that is unsure is told to say the circumstances can hold together, which means it can only ever remove a finding it is confident about, and an unreachable one costs precision instead of conflicts.

Numbers

The gold set is grown from real statements by changing exactly one thing, so the label follows from the mutation rather than from anybody's opinion. Four mutations produce a conflict and six produce a near miss, and the two condition mutations pull against each other on purpose: one plants circumstances that exclude each other and one plants circumstances that differ and can both hold, so a judge that answers everything the same way scores on one and fails the other.

180 generated pairs, 60 conflicting by construction and 120 near misses
precision 0.98, recall 0.98, f1 0.98
40 pairs went to the judge, 20 dropped as never triggered together, 0 it would not answer

Without the judge the same set scores 0.75 precision at the same recall, and all 20 false positives are the exclusive conditions mutation. The two cases still wrong are pairs where the generator grafted an employment condition onto a tax provision or onto a duty of the Government, where the model's answer is defensible and the label is not.

Asking the model directly, over the same 180 pairs, one call each:

precision 0.46, recall 0.70, f1 0.55

The shape matters more than the score. Direct prompting is good at what a person would call a contradiction on sight, 20 of 20 flipped operators and 20 of 20 clashes under compatible conditions, and poor at anything that is arithmetic. It caught 2 of 20 pairs with two different deadlines on one duty, and it called a conflict on 17 of 20 pairs that were never in force on the same day and on 19 of 20 where one norm expressly stands down to the other. That is 36 confident false positives on the two tests the checker does in four lines of code.

The real corpus reports nothing, over 671 comparable forms and 55 pairs, at a noise floor of zero. A run that reports nothing is either a clean scope or a detector that could never have fired, so the check prints what the scope gave the rules: 467 obligations, 8 prohibitions, 79 permissions, 117 rights, 43 deadlines counted from an event and no sanction anywhere. Both operator rules need a prohibition on one side and there are eight of those, and the sanction rule could not have fired in this scope whatever the norms said.

What did not work

The gold set was wrong before it was right. The first condition table called working in the country and working abroad mutually exclusive, which is true of one worker and false of an enterprise that employs hundreds, and the same for a fixed term against an open ended contract. The judge disagreed with my labels and the judge was correct, so the table was rewritten to phrases that exclude each other on one occasion whoever the party is, and the judge is now told the party.

The direct baseline scored 0.00 on its first run and that number measured this harness, not the model. The generated side of a pair has no sentence anywhere in the corpus and was carrying a note saying the test suite produced it, and half the mutations plant a condition or an interval on the original side as well, so the two quotes were not the two norms being compared. Both sides are now written back out from their fields, which also gives the baseline the norm rather than the paragraph it was extracted from, so it loses on cleaner input than the checker gets.

Gate

gofmt -l silent, go vet ./... silent, golangci-lint run at 0 issues, go test -race -cover ./... green across all 28 packages with the new package at 95.0 percent.