RAP compiles a script that violates invariants without error messages

[EstherHageraats]

I was fiddling around with RAP3, much like I think students with do. Trying out stuff.
I tried the Hawaii code, compile, create docs, create proto, change data. Change a bit of code, create proto, reinstall database, change data, fire a violation message. Works fine.
Now I want to fire a violation message from the start using a population that is in violation of the rule.
Hawaii_met_fout.txt

Compile, no problems.
Generate diagnoses doc and in the document the violation is reported.
Generate prototype, no error. Reinstall database. Still no error. It turns out it has reinstalled the correct data from the original Hawaii code.
Delete history, no change. Close and restart Safari. No change. But after some time reinstalling the database results in no data at all.

Let's try a new script, with the same code.
Compile, OK.
make diagnoses, OK.
Make prototype:
403 Forbidden
You don't have permission to access /RAP3/scripts/123456789/generated/Script_1506081797_00964632/prototype/ on this server.
Apache/2.4.10 (Debian) Server at ampersand.tarski.nl Port 80

[stefjoosten]

@EstherHageraats, I have looked into this, toyed around with it, and I cannot reproduce it for 100%. I don't fully understand what you have been doing. But perhaps I don't have to. I learned enough to guess what went wrong, so you can tell me if I'm right... :-) Here is my interpretation of your problem

Symptoms

The script Hawaii_met_fout.txt is a slight variation of the Hawaii script. Both scripts have one rule, which is an invariant. The original Hawaii script does not violate the rule, whereas the adapted script contains one violation. (The violation is that Brown is registered for a trip to Rome, but he does not qualify because he did not pass for Latin.)

The symptom is this: the script Hawaii_met_fout.txt compiles, but does not run, whereas the original script with no violations did run.

Analysis

Each script with one or more violations of an invariant rule will not run. This is normal (and desired) behaviour of Ampersand. The bug here is that RAP3 does not report it back to the user. If Ampersand would have said: This script violates an invariant rule, you would have tracked it down, fixed it, and run the result.

Diagnosis

The compiler messages are not shown for a script without compile errors that has violations in one of its invariants. We have to show these messages an block the prototype in RAP3 from being run. @hanjoosten I'd like to discuss this with you briefly.

Workaround

@EstherHageraats, you can temporarily work around this. For some reason, you found the cause, because you looked at the diagnosis and saw the violation. Chapeau!!! In such cases you need to remember that a script that violates invariants will never run. You can get rid of the violations:

1. by tracking it down and removing its cause;
2. by assigning a role to the rule that is violated. In that way it becomes a process rule (and is no longer an invariant).
3. by disabling the rule (e.g. turn it into comment).

When you do either of these, the violation is gone and your script will run.

[stefjoosten]

Analysis towards a solution

I have looked into cli.php in the function Prototype. It sets the relation protoOk when generation of the prototype returns with exit code 0. Apparently this happens even when violations remain. It makes no distinction between a script that compiles correctly with violations and a script that compiles correctly without violations.

Solution 1

I suggest that we make the Ampersand compiler generate a different exit code for a correct compilation with and without violations.
I also suggest the Ampersand compiler returns these violations, allowing the front-end to turn these into proper error messages. We must of course generate the mechanisms for the front end to report these messages properly.

Solution 2

Let us call the ampersand compiler (from cli.php in function Prototype) with switch --dev. In that way it generates code even if violations exist. Hence, RAP3 will deal with all violations and prevents the user from accessing an inconsistent data set. I am currently experimenting with this option.

@hanjoosten and @Michiel-s , what do you think?

[Michiel-s]

First of all, could we agree not to open RAP specific issues/topics/discussions in the Ampersand repo? Maybe good to (re)open this issue in the RAP repository.

Furthermore, I think solution 1 is the most logical solution. When a script with initial population includes invariant violations, it shouldn't create a fspec or prototype. As far as I know the compiler produces an error already and stops. A exit code 0 is not appropriate in this case. Signal rule violations are of course no issue.

When the exit code changes, you automatically have the feedback to the user in RAP3 application, because for all exit codes other than 0, the output of the compiler is shown to the user.

[hanjoosten]

@Michiel-s : I just added a reminder for those creating a new issue. Hopes that helps. 7f7a4bf

For this specific issue, since there is work to do in Ampersand, I suggest not to move this issue to RAP3.
It should be pretty easy to throw a proper exit code in case that a prototype could not be generated. I'll give it a shot.

[hanjoosten]

I have looked into this issue. It turns out that an exit code of 40 is thrown. I did the following experiment, using a small script with a violated invariant:

C:\Users\hjo20125\Ampersand>copy con aap.cmd
ampersand Issue700.adl -p
echo Exit Code is %errorlevel%
^Z
        1 file(s) copied.

C:\Users\hjo20125\Ampersand>aap

C:\Users\hjo20125\Ampersand>ampersand Issue700.adl -p
Violations of rule "TOT r[A*B]":
- [("Piet","Piet")]

ERROR: No prototype generated because of rule violations.

C:\Users\hjo20125\Ampersand>echo Exit Code is 40
Exit Code is 40

So we need to investigate further, to see what is going on.

[hanjoosten]

Very strange. I compiled the latest version of RAP, and there it throws an errorcode of zero, as can be observed in the log file:

[2017-12-11 21:44:08] COMPILEENGINE.DEBUG: exitcode:'0' response:'Violations of rule "TOT r[A*B]": - [("Piet","Piet")]  Generating frontend.. Frontend generated.  Finished processing your model.' [] []

[hanjoosten]

It appeared that the obsolete --dev switch is being used. That forces the prototype to be generated anyways.

[hanjoosten]

I removed the obsolete --dev switch from Ampersand. ebf13b7. This closes the work in ampersand. I will open a new issue a RAP, for the rest of the work is at RAP.

[Michiel-s]

Exit code was already implemented correctly. There was/is a typo in the variable name in RAP3 specific php file.