Efficiency incident

[stefjoosten]

Problem

When developing a prototype, I wrote the following rule:

RULE startpuntVanRoutes :
   vanEnd;vanEnd~/\berichttype;berichttype~/\naarEnd;naarEnd~
    |- (vanEnd;vanDeel~/\berichttype;ber~/\naarEnd;naarDeel~);I[Deelroute];(vanEnd/vanDeel/\naarEnd/naarDeel/\berichttype/ber)~

The prototype took forever to evaluate this rule (approx. 30 minutes).
The compiler just took 15 seconds (OK, it is not the most trivial rule around...)
The difference between these results (factor 120) is too large to be acceptable, so I categorize this as an efficiency issue.

Symptoms

Docker image of Ampersand that was used: ampersandtarski/prototype-framework:v1.16

The Ampersand term to be evaluated is:

RULE startpuntVanRoutes :
   vanEnd;vanEnd~/\berichttype;berichttype~/\naarEnd;naarEnd~
    |- (vanEnd;vanDeel~/\berichttype;ber~/\naarEnd;naarDeel~);I[Deelroute];(vanEnd/vanDeel/\naarEnd/naarDeel/\berichttype/ber)~
VIOLATION (TXT "{EX} NewStruct;Deelroute"
               ,TXT ";vanDeel;Deelroute;_NEW;Applicatie;", SRC vanEnd
               ,TXT ";ber;Deelroute;_NEW;Berichttype;", SRC berichttype
               ,TXT ";naarDeel;Deelroute;_NEW;Applicatie;", SRC naarEnd
          )

If the compiler computes the violations, it takes 15 seconds to compute the following result:

32.18 /usr/local/project/Communicatie.adl:66:1 error:
#0 32.18   There are 10 violations of RULE "startpuntVanRoutes":
#0 32.18     {EX} NewStruct;Deelroute;vanDeel;Deelroute;_NEW;Applicatie;"LMS_0665";ber;Deelroute;_NEW;Berichttype;"IncidentBericht";naarDeel;Deelroute;_NEW;Applicatie;"LMS_0011"
#0 32.18     {EX} NewStruct;Deelroute;vanDeel;Deelroute;_NEW;Applicatie;{"LMS_1057", "LMS_1056"};ber;Deelroute;_NEW;Berichttype;{"IncidentBericht", "aan- en afmelding"};naarDeel;Deelroute;_NEW;Applicatie;"LMS_0011"
#0 32.18     {EX} NewStruct;Deelroute;vanDeel;Deelroute;_NEW;Applicatie;"LMS_0018";ber;Deelroute;_NEW;Berichttype;"Voertuig-status";naarDeel;Deelroute;_NEW;Applicatie;"LMS_0865"
#0 32.18     {EX} NewStruct;Deelroute;vanDeel;Deelroute;_NEW;Applicatie;"LMS_0869";ber;Deelroute;_NEW;Berichttype;"Voertuig-status";naarDeel;Deelroute;_NEW;Applicatie;"LMS_0011"
#0 32.18     {EX} NewStruct;Deelroute;vanDeel;Deelroute;_NEW;Applicatie;"LMS_0664";ber;Deelroute;_NEW;Berichttype;"Voertuig-locatie";naarDeel;Deelroute;_NEW;Applicatie;"LMS_0223"
#0 32.19     {EX} NewStruct;Deelroute;vanDeel;Deelroute;_NEW;Applicatie;{"LMS_1055", "LMS_1054"};ber;Deelroute;_NEW;Berichttype;"IncidentBericht";naarDeel;Deelroute;_NEW;Applicatie;"LMS_0011"
#0 32.19     {EX} NewStruct;Deelroute;vanDeel;Deelroute;_NEW;Applicatie;"LMS_0665";ber;Deelroute;_NEW;Berichttype;"Voertuig-locatie";naarDeel;Deelroute;_NEW;Applicatie;"LMS_0050"
#0 32.19     {EX} NewStruct;Deelroute;vanDeel;Deelroute;_NEW;Applicatie;"LMS_0665";ber;Deelroute;_NEW;Berichttype;"Voertuig-locatie (buiten regio)";naarDeel;Deelroute;_NEW;Applicatie;"LMS_0223"
#0 32.19     {EX} NewStruct;Deelroute;vanDeel;Deelroute;_NEW;Applicatie;"LMS_0265";ber;Deelroute;_NEW;Berichttype;"Voertuig-locatie";naarDeel;Deelroute;_NEW;Applicatie;"LMS_0223"
#0 32.19     {EX} NewStruct;Deelroute;vanDeel;Deelroute;_NEW;Applicatie;"LMS_0011";ber;Deelroute;_NEW;Berichttype;"Voertuig-status";naarDeel;Deelroute;_NEW;Applicatie;"LMS_0864"
#0 32.19 2023-05-15 07:16:30.632830: [error] ExitFailure 10
--

However, if the prototype computes the same result, it takes approx. 30 minutes.
Here are some lines in the log file, just before the SQL query was run.

prototype  | [2023-05-15 06:52:40] RULEENGINE.DEBUG: Checking rule 'startpuntVanRoutes' [] {"request_id":"26418b3df5"}
prototype  | [2023-05-15 06:52:40] RULEENGINE.DEBUG: Evaluating conjunct 'conj_131' [] {"request_id":"26418b3df5"}
prototype  | [2023-05-15 06:52:40] DATABASE.DEBUG:

This is the SQL query. It is roughly 30000 characters without prettyprinting.

select distinct someDummyNameBecauseMySQLNeedsOne.src as src, someDummyNameBecauseMySQLNeedsOne.tgt as tgt from (select subIntersect0.src as src, subIntersect0.tgt as tgt from (/* ECps (EDcD vanEnd[End2End*Applicatie],EFlp (EDcD vanEnd[End2End*Applicatie])) */ /* Expression: vanEnd [End2End*Applicatie];vanEnd [End2End*Applicatie]~ */ /* Signature : [End2End*End2End] */ /* case: (ECps es), with two or more elements in es. */ /* Expression: vanEnd [End2End*Applicatie];vanEnd [End2End*Applicatie]~ */ /* Signature : [End2End*End2End] */ select fence0.src as src, fence1.tgt as tgt from (/* EDcD vanEnd[End2End*Applicatie] */ /* Expression: vanEnd [End2End*Applicatie] */ /* Signature : [End2End*Applicatie] */ select "End2End" as src, "Applicatie" as tgt from "vanEnd" where ("End2End" is not null) and ("Applicatie" is not null)) as fence0, (/* EFlp (EDcD vanEnd[End2End*Applicatie]) */ /* Expression: vanEnd [End2End*Applicatie]~ */ /* Signature : [Applicatie*End2End] */ /* case: EFlp x */ /* Expression: vanEnd [End2End*Applicatie]~ */ /* Signature : [Applicatie*End2End] */ /* EDcD vanEnd[End2End*Applicatie] */ /* Expression: vanEnd [End2End*Applicatie] */ /* Signature : [End2End*Applicatie] */ /* Expression: vanEnd [End2End*Applicatie]~ */ /* Signature : [Applicatie*End2End] */ select "Applicatie" as src, "End2End" as tgt from "vanEnd" where ("End2End" is not null) and ("Applicatie" is not null)) as fence1 where (fence0.tgt = fence1.src)) as subIntersect0, (/* ECps (EEps End2End [End2End*Koppeling],ECps (EDcD berichttype[Koppeling*Berichttype],ECps (EFlp (EDcD berichttype[Koppeling*Berichttype]),EEps End2End [Koppeling*End2End]))) */ /* Expression: berichttype [Koppeling*Berichttype];berichttype [Koppeling*Berichttype]~ */ /* Signature : [End2End*End2End] */ /* case: (ECps es), with two or more elements in es. */ /* Expression: berichttype [Koppeling*Berichttype];berichttype [Koppeling*Berichttype]~ */ /* Signature : [End2End*End2End] */ select fence0.src as src, fence3.tgt as tgt from (/* EEps End2End [End2End*Koppeling] */ /* Expression: I[End2End] */ /* Signature : [End2End*Koppeling] */ /* case: EEps c _ */ /* Expression: I[End2End] */ /* Signature : [End2End*Koppeling] */ select "End2End" as src, "End2End" as tgt from "BeheerEenheid" where "End2End" is not null) as fence0, (/* EDcD berichttype[Koppeling*Berichttype] */ /* Expression: berichttype [Koppeling*Berichttype] */ /* Signature : [Koppeling*Berichttype] */ select "Koppeling" as src, "Berichttype" as tgt from "berichttype1" where ("Koppeling" is not null) and ("Berichttype" is not null)) as fence1, (/* EFlp (EDcD berichttype[Koppeling*Berichttype]) */ /* Expression: berichttype [Koppeling*Berichttype]~ */ /* Signature : [Berichttype*Koppeling] */ /* case: EFlp x */ /* Expression: berichttype [Koppeling*Berichttype]~ */ /* Signature : [Berichttype*Koppeling] */ /* EDcD berichttype[Koppeling*Berichttype] */ /* Expression: berichttype [Koppeling*Berichttype] */ /* Signature : [Koppeling*Berichttype] */ /* Expression: berichttype [Koppeling*Berichttype]~ */ /* Signature : [Berichttype*Koppeling] */ select "Berichttype" as src, "Koppeling" as tgt from "berichttype1" where ("Koppeling" is not null) and ("Berichttype" is not null)) as fence2, (/* EEps End2End [Koppeling*End2End] */ /* Expression: I[End2End] */ /* Signature : [Koppeling*End2End] */ /* case: EEps c _ */ /* Expression: I[End2End] */ /* Signature : [Koppeling*End2End] */ select "End2End" as src, "End2End" as tgt from "BeheerEenheid" where "End2End" is not null) as fence3 where (fence0.tgt = fence1.src) and (fence1.tgt = fence2.src) and (fence2.tgt = fence3.src)) as subIntersect1, (/* EDif (ECps (EDcD naarEnd[End2End*Applicatie],EFlp (EDcD naarEnd[End2End*Applicatie])),ECps (EIsc (ECps (EDcD vanEnd[End2End*Applicatie],EFlp (EDcD vanDeel[Deelroute*Applicatie])),EIsc (ECps (EEps End2End [End2End*Koppeling],ECps (EDcD berichttype[Koppeling*Berichttype],EFlp (EDcD ber[Deelroute*Berichttype]))),ECps (EDcD naarEnd[End2End*Applicatie],EFlp (EDcD naarDeel[Deelroute*Applicatie])))),EFlp (EBrk (EIsc (ELrs (EDcD vanEnd[End2End*Applicatie],EDcD vanDeel[Deelroute*Applicatie]),EIsc (ELrs (EDcD naarEnd[End2End*Applicatie],EDcD naarDeel[Deelroute*Applicatie]),ECps (EEps End2End [End2End*Koppeling],ELrs (EDcD berichttype[Koppeling*Berichttype],EDcD ber[Deelroute*Berichttype])))))))) */ /* Expression: naarEnd [End2End*Applicatie];naarEnd [End2End*Applicatie]~-vanEnd [End2End*Applicatie];vanDeel [Deelroute*Applicatie]~/\berichttype [Koppeling*Berichttype];ber [Deelroute*Berichttype]~/\naarEnd [End2End*Applicatie];naarDeel [Deelroute*Applicatie]~;(vanEnd [End2End*Applicatie]/vanDeel [Deelroute*Applicatie]/\naarEnd [End2End*Applicatie]/naarDeel [Deelroute*Applicatie]/\berichttype [Koppeling*Berichttype]/ber [Deelroute*Berichttype])~ */ /* Signature : [End2End*End2End] */ /* case: EDif (l,r) */ /* Expression: naarEnd [End2End*Applicatie];naarEnd [End2End*Applicatie]~-vanEnd [End2End*Applicatie];vanDeel [Deelroute*Applicatie]~/\berichttype [Koppeling*Berichttype];ber [Deelroute*Berichttype]~/\naarEnd [End2End*Applicatie];naarDeel [Deelroute*Applicatie]~;(vanEnd [End2End*Applicatie]/vanDeel [Deelroute*Applicatie]/\naarEnd [End2End*Applicatie]/naarDeel [Deelroute*Applicatie]/\berichttype [Koppeling*Berichttype]/ber [Deelroute*Berichttype])~ */ /* Signature : [End2End*End2End] */ /* EIsc (ECps (EDcD naarEnd[End2End*Applicatie],EFlp (EDcD naarEnd[End2End*Applicatie])),ECpl (ECps (EIsc (ECps (EDcD vanEnd[End2End*Applicatie],EFlp (EDcD vanDeel[Deelroute*Applicatie])),EIsc (ECps (EEps End2End [End2End*Koppeling],ECps (EDcD berichttype[Koppeling*Berichttype],EFlp (EDcD ber[Deelroute*Berichttype]))),ECps (EDcD naarEnd[End2End*Applicatie],EFlp (EDcD naarDeel[Deelroute*Applicatie])))),EFlp (EBrk (EIsc (ELrs (EDcD vanEnd[End2End*Applicatie],EDcD vanDeel[Deelroute*Applicatie]),EIsc (ELrs (EDcD naarEnd[End2End*Applicatie],EDcD naarDeel[Deelroute*Applicatie]),ECps (EEps End2End [End2End*Koppeling],ELrs (EDcD berichttype[Koppeling*Berichttype],EDcD ber[Deelroute*Berichttype]))))))))) */ /* Expression: naarEnd [End2End*Applicatie];naarEnd [End2End*Applicatie]~/\ -vanEnd [End2End*Applicatie];vanDeel [Deelroute*Applicatie]~/\berichttype [Koppeling*Berichttype];ber [Deelroute*Berichttype]~/\naarEnd [End2End*Applicatie];naarDeel [Deelroute*Applicatie]~;(vanEnd [End2End*Applicatie]/vanDeel [Deelroute*Applicatie]/\naarEnd [End2End*Applicatie]/naarDeel [Deelroute*Applicatie]/\berichttype [Koppeling*Berichttype]/ber [Deelroute*Berichttype])~ */ /* Signature : [End2End*End2End] */ /* Optimized case for: <expr1> intersect with the complement of <expr2>. */ /* where */ /* <expr1> = naarEnd [End2End*Applicatie];naarEnd [End2End*Applicatie]~ (sign: [End2End*End2End]) */ /* <expr2> = vanEnd [End2End*Applicatie];vanDeel [Deelroute*Applicatie]~/\berichttype [Koppeling*Berichttype];ber [Deelroute*Berichttype]~/\naarEnd [End2End*Applicatie];naarDeel [Deelroute*Applicatie]~;(vanEnd [End2End*Applicatie]/vanDeel [Deelroute*Applicatie]/\naarEnd [End2End*Applicatie]/naarDeel [Deelroute*Applicatie]/\berichttype [Koppeling*Berichttype]/ber [Deelroute*Berichttype])~ (sign: [End2End*End2End]) */ /* Expression: naarEnd [End2End*Applicatie];naarEnd [End2End*Applicatie]~/\ -vanEnd [End2End*Applicatie];vanDeel [Deelroute*Applicatie]~/\berichttype [Koppeling*Berichttype];ber [Deelroute*Berichttype]~/\naarEnd [End2End*Applicatie];naarDeel [Deelroute*Applicatie]~;(vanEnd [End2End*Applicatie]/vanDeel [Deelroute*Applicatie]/\naarEnd [End2End*Applicatie]/naarDeel [Deelroute*Applicatie]/\berichttype [Koppeling*Berichttype]/ber [Deelroute*Berichttype])~ */ /* Signature : [End2End*End2End] */ select t1.src as src, t1.tgt as tgt from (/* ECps (EDcD naarEnd[End2End*Applicatie],EFlp (EDcD naarEnd[End2End*Applicatie])) */ /* Expression: naarEnd [End2End*Applicatie];naarEnd [End2End*Applicatie]~ */ /* Signature : [End2End*End2End] */ /* case: (ECps es), with two or more elements in es. */ /* Expression: naarEnd [End2End*Applicatie];naarEnd [End2End*Applicatie]~ */ /* Signature : [End2End*End2End] */ select fence0.src as src, fence1.tgt as tgt from (/* EDcD naarEnd[End2End*Applicatie] */ /* Expression: naarEnd [End2End*Applicatie] */ /* Signature : [End2End*Applicatie] */ select "End2End" as src, "Applicatie" as tgt from "naarEnd" where ("End2End" is not null) and ("Applicatie" is not null)) as fence0, (/* EFlp (EDcD naarEnd[End2End*Applicatie]) */ /* Expression: naarEnd [End2End*Applicatie]~ */ /* Signature : [Applicatie*End2End] */ /* case: EFlp x */ /* Expression: naarEnd [End2End*Applicatie]~ */ /* Signature : [Applicatie*End2End] */ /* EDcD naarEnd[End2End*Applicatie] */ /* Expression: naarEnd [End2End*Applicatie] */ /* Signature : [End2End*Applicatie] */ /* Expression: naarEnd [End2End*Applicatie]~ */ /* Signature : [Applicatie*End2End] */ select "Applicatie" as src, "End2End" as tgt from "naarEnd" where ("End2End" is not null) and ("Applicatie" is not null)) as fence1 where (fence0.tgt = fence1.src)) as t1 left join (/* ECps (EIsc (ECps (EDcD vanEnd[End2End*Applicatie],EFlp (EDcD vanDeel[Deelroute*Applicatie])),EIsc (ECps (EEps End2End [End2End*Koppeling],ECps (EDcD berichttype[Koppeling*Berichttype],EFlp (EDcD ber[Deelroute*Berichttype]))),ECps (EDcD naarEnd[End2End*Applicatie],EFlp (EDcD naarDeel[Deelroute*Applicatie])))),EFlp (EBrk (EIsc (ELrs (EDcD vanEnd[End2End*Applicatie],EDcD vanDeel[Deelroute*Applicatie]),EIsc (ELrs (EDcD naarEnd[End2End*Applicatie],EDcD naarDeel[Deelroute*Applicatie]),ECps (EEps End2End [End2End*Koppeling],ELrs (EDcD berichttype[Koppeling*Berichttype],EDcD ber[Deelroute*Berichttype]))))))) */ /* Expression: vanEnd [End2End*Applicatie];vanDeel [Deelroute*Applicatie]~/\berichttype [Koppeling*Berichttype];ber [Deelroute*Berichttype]~/\naarEnd [End2End*Applicatie];naarDeel [Deelroute*Applicatie]~;(vanEnd [End2End*Applicatie]/vanDeel [Deelroute*Applicatie]/\naarEnd [End2End*Applicatie]/naarDeel [Deelroute*Applicatie]/\berichttype [Koppeling*Berichttype]/ber [Deelroute*Berichttype])~ */ /* Signature : [End2End*End2End] */ /* case: (ECps es), with two or more elements in es. */ /* Expression: vanEnd [End2End*Applicatie];vanDeel [Deelroute*Applicatie]~/\berichttype [Koppeling*Berichttype];ber [Deelroute*Berichttype]~/\naarEnd [End2End*Applicatie];naarDeel [Deelroute*Applicatie]~;(vanEnd [End2End*Applicatie]/vanDeel [Deelroute*Applicatie]/\naarEnd [End2End*Applicatie]/naarDeel [Deelroute*Applicatie]/\berichttype [Koppeling*Berichttype]/ber [Deelroute*Berichttype])~ */ /* Signature : [End2End*End2End] */ select fence0.src as src, fence1.tgt as tgt from (/* EIsc (ECps (EDcD vanEnd[End2End*Applicatie],EFlp (EDcD vanDeel[Deelroute*Applicatie])),EIsc (ECps (EEps End2End [End2End*Koppeling],ECps (EDcD berichttype[Koppeling*Berichttype],EFlp (EDcD ber[Deelroute*Berichttype]))),ECps (EDcD naarEnd[End2End*Applicatie],EFlp (EDcD naarDeel[Deelroute*Applicatie])))) */ /* Expression: vanEnd [End2End*Applicatie];vanDeel [Deelroute*Applicatie]~/\berichttype [Koppeling*Berichttype];ber [Deelroute*Berichttype]~/\naarEnd [End2End*Applicatie];naarDeel [Deelroute*Applicatie]~ */ /* Signature : [End2End*Deelroute] */ /* case: EIsc{} */ /* Expression: vanEnd [End2End*Applicatie];vanDeel [Deelroute*Applicatie]~/\berichttype [Koppeling*Berichttype];ber [Deelroute*Berichttype]~/\naarEnd [End2End*Applicatie];naarDeel [Deelroute*Applicatie]~ */ /* Signature : [End2End*Deelroute] */ select someDummyNameBecauseMySQLNeedsOne.src as src, someDummyNameBecauseMySQLNeedsOne.tgt as tgt from (/* `intersect` does not work in MySQL, so this statement is generated: */ select subIntersect0.src as src, subIntersect0.tgt as tgt from (/* ECps (EDcD vanEnd[End2End*Applicatie],EFlp (EDcD vanDeel[Deelroute*Applicatie])) */ /* Expression: vanEnd [End2End*Applicatie];vanDeel [Deelroute*Applicatie]~ */ /* Signature : [End2End*Deelroute] */ /* case: (ECps es), with two or more elements in es. */ /* Expression: vanEnd [End2End*Applicatie];vanDeel [Deelroute*Applicatie]~ */ /* Signature : [End2End*Deelroute] */ select fence0.src as src, fence1.tgt as tgt from (/* EDcD vanEnd[End2End*Applicatie] */ /* Expression: vanEnd [End2End*Applicatie] */ /* Signature : [End2End*Applicatie] */ select "End2End" as src, "Applicatie" as tgt from "vanEnd" where ("End2End" is not null) and ("Applicatie" is not null)) as fence0, (/* EFlp (EDcD vanDeel[Deelroute*Applicatie]) */ /* Expression: vanDeel [Deelroute*Applicatie]~ */ /* Signature : [Applicatie*Deelroute] */ /* case: EFlp x */ /* Expression: vanDeel [Deelroute*Applicatie]~ */ /* Signature : [Applicatie*Deelroute] */ /* EDcD vanDeel[Deelroute*Applicatie] */ /* Expression: vanDeel [Deelroute*Applicatie] */ /* Signature : [Deelroute*Applicatie] */ /* Expression: vanDeel [Deelroute*Applicatie]~ */ /* Signature : [Applicatie*Deelroute] */ select "vanDeel" as src, "Deelroute" as tgt from "Deelroute" where ("Deelroute" is not null) and ("vanDeel" is not null)) as fence1 where (fence0.tgt = fence1.src)) as subIntersect0, (/* ECps (EEps End2End [End2End*Koppeling],ECps (EDcD berichttype[Koppeling*Berichttype],EFlp (EDcD ber[Deelroute*Berichttype]))) */ /* Expression: berichttype [Koppeling*Berichttype];ber [Deelroute*Berichttype]~ */ /* Signature : [End2End*Deelroute] */ /* case: (ECps es), with two or more elements in es. */ /* Expression: berichttype [Koppeling*Berichttype];ber [Deelroute*Berichttype]~ */ /* Signature : [End2End*Deelroute] */ select fence0.src as src, fence2.tgt as tgt from (/* EEps End2End [End2End*Koppeling] */ /* Expression: I[End2End] */ /* Signature : [End2End*Koppeling] */ /* case: EEps c _ */ /* Expression: I[End2End] */ /* Signature : [End2End*Koppeling] */ select "End2End" as src, "End2End" as tgt from "BeheerEenheid" where "End2End" is not null) as fence0, (/* EDcD berichttype[Koppeling*Berichttype] */ /* Expression: berichttype [Koppeling*Berichttype] */ /* Signature : [Koppeling*Berichttype] */ select "Koppeling" as src, "Berichttype" as tgt from "berichttype1" where ("Koppeling" is not null) and ("Berichttype" is not null)) as fence1, (/* EFlp (EDcD ber[Deelroute*Berichttype]) */ /* Expression: ber [Deelroute*Berichttype]~ */ /* Signature : [Berichttype*Deelroute] */ /* case: EFlp x */ /* Expression: ber [Deelroute*Berichttype]~ */ /* Signature : [Berichttype*Deelroute] */ /* EDcD ber[Deelroute*Berichttype] */ /* Expression: ber [Deelroute*Berichttype] */ /* Signature : [Deelroute*Berichttype] */ /* Expression: ber [Deelroute*Berichttype]~ */ /* Signature : [Berichttype*Deelroute] */ select "ber" as src, "Deelroute" as tgt from "Deelroute" where ("Deelroute" is not null) and ("ber" is not null)) as fence2 where (fence0.tgt = fence1.src) and (fence1.tgt = fence2.src)) as subIntersect1, (/* ECps (EDcD naarEnd[End2End*Applicatie],EFlp (EDcD naarDeel[Deelroute*Applicatie])) */ /* Expression: naarEnd [End2End*Applicatie];naarDeel [Deelroute*Applicatie]~ */ /* Signature : [End2End*Deelroute] */ /* case: (ECps es), with two or more elements in es. */ /* Expression: naarEnd [End2End*Applicatie];naarDeel [Deelroute*Applicatie]~ */ /* Signature : [End2End*Deelroute] */ select fence0.src as src, fence1.tgt as tgt from (/* EDcD naarEnd[End2End*Applicatie] */ /* Expression: naarEnd [End2End*Applicatie] */ /* Signature : [End2End*Applicatie] */ select "End2End" as src, "Applicatie" as tgt from "naarEnd" where ("End2End" is not null) and ("Applicatie" is not null)) as fence0, (/* EFlp (EDcD naarDeel[Deelroute*Applicatie]) */ /* Expression: naarDeel [Deelroute*Applicatie]~ */ /* Signature : [Applicatie*Deelroute] */ /* case: EFlp x */ /* Expression: naarDeel [Deelroute*Applicatie]~ */ /* Signature : [Applicatie*Deelroute] */ /* EDcD naarDeel[Deelroute*Applicatie] */ /* Expression: naarDeel [Deelroute*Applicatie] */ /* Signature : [Deelroute*Applicatie] */ /* Expression: naarDeel [Deelroute*Applicatie]~ */ /* Signature : [Applicatie*Deelroute] */ select "naarDeel" as src, "Deelroute" as tgt from "Deelroute" where ("Deelroute" is not null) and ("naarDeel" is not null)) as fence1 where (fence0.tgt = fence1.src)) as subIntersect2 where (subIntersect1.src = subIntersect0.src) and (subIntersect1.tgt = subIntersect0.tgt) and (subIntersect2.src = subIntersect0.src) and (subIntersect2.tgt = subIntersect0.tgt)) as someDummyNameBecauseMySQLNeedsOne) as fence0, (/* EFlp (EBrk (EIsc (ELrs (EDcD vanEnd[End2End*Applicatie],EDcD vanDeel[Deelroute*Applicatie]),EIsc (ELrs (EDcD naarEnd[End2End*Applicatie],EDcD naarDeel[Deelroute*Applicatie]),ECps (EEps End2End [End2End*Koppeling],ELrs (EDcD berichttype[Koppeling*Berichttype],EDcD ber[Deelroute*Berichttype])))))) */ /* Expression: (vanEnd [End2End*Applicatie]/vanDeel [Deelroute*Applicatie]/\naarEnd [End2End*Applicatie]/naarDeel [Deelroute*Applicatie]/\berichttype [Koppeling*Berichttype]/ber [Deelroute*Berichttype])~ */ /* Signature : [Deelroute*End2End] */ /* case: EFlp x */ /* Expression: (vanEnd [End2End*Applicatie]/vanDeel [Deelroute*Applicatie]/\naarEnd [End2End*Applicatie]/naarDeel [Deelroute*Applicatie]/\berichttype [Koppeling*Berichttype]/ber [Deelroute*Berichttype])~ */ /* Signature : [Deelroute*End2End] */ /* EBrk (EIsc (ELrs (EDcD vanEnd[End2End*Applicatie],EDcD vanDeel[Deelroute*Applicatie]),EIsc (ELrs (EDcD naarEnd[End2End*Applicatie],EDcD naarDeel[Deelroute*Applicatie]),ECps (EEps End2End [End2End*Koppeling],ELrs (EDcD berichttype[Koppeling*Berichttype],EDcD ber[Deelroute*Berichttype]))))) */ /* Expression: (vanEnd [End2End*Applicatie]/vanDeel [Deelroute*Applicatie]/\naarEnd [End2End*Applicatie]/naarDeel [Deelroute*Applicatie]/\berichttype [Koppeling*Berichttype]/ber [Deelroute*Berichttype]) */ /* Signature : [End2End*Deelroute] */ /* Expression: (vanEnd [End2End*Applicatie]/vanDeel [Deelroute*Applicatie]/\naarEnd [End2End*Applicatie]/naarDeel [Deelroute*Applicatie]/\berichttype [Koppeling*Berichttype]/ber [Deelroute*Berichttype])~ */ /* Signature : [Deelroute*End2End] */ /* EIsc (ELrs (EDcD vanEnd[End2End*Applicatie],EDcD vanDeel[Deelroute*Applicatie]),EIsc (ELrs (EDcD naarEnd[End2End*Applicatie],EDcD naarDeel[Deelroute*Applicatie]),ECps (EEps End2End [End2End*Koppeling],ELrs (EDcD berichttype[Koppeling*Berichttype],EDcD ber[Deelroute*Berichttype])))) */ /* Expression: vanEnd [End2End*Applicatie]/vanDeel [Deelroute*Applicatie]/\naarEnd [End2End*Applicatie]/naarDeel [Deelroute*Applicatie]/\berichttype [Koppeling*Berichttype]/ber [Deelroute*Berichttype] */ /* Signature : [End2End*Deelroute] */ /* Expression: (vanEnd [End2End*Applicatie]/vanDeel [Deelroute*Applicatie]/\naarEnd [End2End*Applicatie]/naarDeel [Deelroute*Applicatie]/\berichttype [Koppeling*Berichttype]/ber [Deelroute*Berichttype])~ */ /* Signature : [Deelroute*End2End] */ /* case: EIsc{} */ /* Expression: vanEnd [End2End*Applicatie]/vanDeel [Deelroute*Applicatie]/\naarEnd [End2End*Applicatie]/naarDeel [Deelroute*Applicatie]/\berichttype [Koppeling*Berichttype]/ber [Deelroute*Berichttype] */ /* Signature : [End2End*Deelroute] */ /* Expression: (vanEnd [End2End*Applicatie]/vanDeel [Deelroute*Applicatie]/\naarEnd [End2End*Applicatie]/naarDeel [Deelroute*Applicatie]/\berichttype [Koppeling*Berichttype]/ber [Deelroute*Berichttype])~ */ /* Signature : [Deelroute*End2End] */ select someDummyNameBecauseMySQLNeedsOne.tgt as src, someDummyNameBecauseMySQLNeedsOne.src as tgt from (/* `intersect` does not work in MySQL, so this statement is generated: */ select subIntersect0.src as src, subIntersect0.tgt as tgt from (/* ELrs (EDcD vanEnd[End2End*Applicatie],EDcD vanDeel[Deelroute*Applicatie]) */ /* Expression: vanEnd [End2End*Applicatie]/vanDeel [Deelroute*Applicatie] */ /* Signature : [End2End*Deelroute] */ /* case: ELrs (l,r) */ /* Expression: vanEnd [End2End*Applicatie]/vanDeel [Deelroute*Applicatie] */ /* Signature : [End2End*Deelroute] */ /* EFlp (ERrs (EFlp (EDcD vanDeel[Deelroute*Applicatie]),EFlp (EDcD vanEnd[End2End*Applicatie]))) */ /* Expression: vanDeel [Deelroute*Applicatie]~\vanEnd [End2End*Applicatie]~~ */ /* Signature : [End2End*Deelroute] */ /* case: EFlp x */ /* Expression: vanDeel [Deelroute*Applicatie]~\vanEnd [End2End*Applicatie]~~ */ /* Signature : [End2End*Deelroute] */ /* ERrs (EFlp (EDcD vanDeel[Deelroute*Applicatie]),EFlp (EDcD vanEnd[End2End*Applicatie])) */ /* Expression: vanDeel [Deelroute*Applicatie]~\vanEnd [End2End*Applicatie]~ */ /* Signature : [Deelroute*End2End] */ /* Expression: vanDeel [Deelroute*Applicatie]~\vanEnd [End2End*Applicatie]~~ */ /* Signature : [End2End*Deelroute] */ /* case: ERrs (l,r) */ /* Expression: vanDeel [Deelroute*Applicatie]~\vanEnd [End2End*Applicatie]~ */ /* Signature : [Deelroute*End2End] */ /* Expression: vanDeel [Deelroute*Applicatie]~\vanEnd [End2End*Applicatie]~~ */ /* Signature : [End2End*Deelroute] */ select RResRight."End2End" as src, RResLeft."Deelroute" as tgt from "Deelroute" as RResLeft, "BeheerEenheid" as RResRight where /* Left hand side: vanDeel [Deelroute*Applicatie]~ */ NOT exists (select * from (/* EFlp (EDcD vanDeel[Deelroute*Applicatie]) */ /* Expression: vanDeel [Deelroute*Applicatie]~ */ /* Signature : [Applicatie*Deelroute] */ /* case: EFlp x */ /* Expression: vanDeel [Deelroute*Applicatie]~ */ /* Signature : [Applicatie*Deelroute] */ /* EDcD vanDeel[Deelroute*Applicatie] */ /* Expression: vanDeel [Deelroute*Applicatie] */ /* Signature : [Deelroute*Applicatie] */ /* Expression: vanDeel [Deelroute*Applicatie]~ */ /* Signature : [Applicatie*Deelroute] */ select "vanDeel" as src, "Deelroute" as tgt from "Deelroute" where ("Deelroute" is not null) and ("vanDeel" is not null)) as lhs where (RResLeft."Deelroute" = lhs.tgt) and (/* Right hand side: vanEnd [End2End*Applicatie]~ */ NOT exists (select * from (/* EFlp (EDcD vanEnd[End2End*Applicatie]) */ /* Expression: vanEnd [End2End*Applicatie]~ */ /* Signature : [Applicatie*End2End] */ /* case: EFlp x */ /* Expression: vanEnd [End2End*Applicatie]~ */ /* Signature : [Applicatie*End2End] */ /* EDcD vanEnd[End2End*Applicatie] */ /* Expression: vanEnd [End2End*Applicatie] */ /* Signature : [End2End*Applicatie] */ /* Expression: vanEnd [End2End*Applicatie]~ */ /* Signature : [Applicatie*End2End] */ select "Applicatie" as src, "End2End" as tgt from "vanEnd" where ("End2End" is not null) and ("Applicatie" is not null)) as rhs where (rhs.src = lhs.src) and (rhs.tgt = RResRight."End2End"))))) as subIntersect0, (/* ELrs (EDcD naarEnd[End2End*Applicatie],EDcD naarDeel[Deelroute*Applicatie]) */ /* Expression: naarEnd [End2End*Applicatie]/naarDeel [Deelroute*Applicatie] */ /* Signature : [End2End*Deelroute] */ /* case: ELrs (l,r) */ /* Expression: naarEnd [End2End*Applicatie]/naarDeel [Deelroute*Applicatie] */ /* Signature : [End2End*Deelroute] */ /* EFlp (ERrs (EFlp (EDcD naarDeel[Deelroute*Applicatie]),EFlp (EDcD naarEnd[End2End*Applicatie]))) */ /* Expression: naarDeel [Deelroute*Applicatie]~\naarEnd [End2End*Applicatie]~~ */ /* Signature : [End2End*Deelroute] */ /* case: EFlp x */ /* Expression: naarDeel [Deelroute*Applicatie]~\naarEnd [End2End*Applicatie]~~ */ /* Signature : [End2End*Deelroute] */ /* ERrs (EFlp (EDcD naarDeel[Deelroute*Applicatie]),EFlp (EDcD naarEnd[End2End*Applicatie])) */ /* Expression: naarDeel [Deelroute*Applicatie]~\naarEnd [End2End*Applicatie]~ */ /* Signature : [Deelroute*End2End] */ /* Expression: naarDeel [Deelroute*Applicatie]~\naarEnd [End2End*Applicatie]~~ */ /* Signature : [End2End*Deelroute] */ /* case: ERrs (l,r) */ /* Expression: naarDeel [Deelroute*Applicatie]~\naarEnd [End2End*Applicatie]~ */ /* Signature : [Deelroute*End2End] */ /* Expression: naarDeel [Deelroute*Applicatie]~\naarEnd [End2End*Applicatie]~~ */ /* Signature : [End2End*Deelroute] */ select RResRight."End2End" as src, RResLeft."Deelroute" as tgt from "Deelroute" as RResLeft, "BeheerEenheid" as RResRight where /* Left hand side: naarDeel [Deelroute*Applicatie]~ */ NOT exists (select * from (/* EFlp (EDcD naarDeel[Deelroute*Applicatie]) */ /* Expression: naarDeel [Deelroute*Applicatie]~ */ /* Signature : [Applicatie*Deelroute] */ /* case: EFlp x */ /* Expression: naarDeel [Deelroute*Applicatie]~ */ /* Signature : [Applicatie*Deelroute] */ /* EDcD naarDeel[Deelroute*Applicatie] */ /* Expression: naarDeel [Deelroute*Applicatie] */ /* Signature : [Deelroute*Applicatie] */ /* Expression: naarDeel [Deelroute*Applicatie]~ */ /* Signature : [Applicatie*Deelroute] */ select "naarDeel" as src, "Deelroute" as tgt from "Deelroute" where ("Deelroute" is not null) and ("naarDeel" is not null)) as lhs where (RResLeft."Deelroute" = lhs.tgt) and (/* Right hand side: naarEnd [End2End*Applicatie]~ */ NOT exists (select * from (/* EFlp (EDcD naarEnd[End2End*Applicatie]) */ /* Expression: naarEnd [End2End*Applicatie]~ */ /* Signature : [Applicatie*End2End] */ /* case: EFlp x */ /* Expression: naarEnd [End2End*Applicatie]~ */ /* Signature : [Applicatie*End2End] */ /* EDcD naarEnd[End2End*Applicatie] */ /* Expression: naarEnd [End2End*Applicatie] */ /* Signature : [End2End*Applicatie] */ /* Expression: naarEnd [End2End*Applicatie]~ */ /* Signature : [Applicatie*End2End] */ select "Applicatie" as src, "End2End" as tgt from "naarEnd" where ("End2End" is not null) and ("Applicatie" is not null)) as rhs where (rhs.src = lhs.src) and (rhs.tgt = RResRight."End2End"))))) as subIntersect1, (/* ECps (EEps End2End [End2End*Koppeling],ELrs (EDcD berichttype[Koppeling*Berichttype],EDcD ber[Deelroute*Berichttype])) */ /* Expression: berichttype [Koppeling*Berichttype]/ber [Deelroute*Berichttype] */ /* Signature : [End2End*Deelroute] */ /* case: (ECps es), with two or more elements in es. */ /* Expression: berichttype [Koppeling*Berichttype]/ber [Deelroute*Berichttype] */ /* Signature : [End2End*Deelroute] */ select fence0.src as src, fence1.tgt as tgt from (/* EEps End2End [End2End*Koppeling] */ /* Expression: I[End2End] */ /* Signature : [End2End*Koppeling] */ /* case: EEps c _ */ /* Expression: I[End2End] */ /* Signature : [End2End*Koppeling] */ select "End2End" as src, "End2End" as tgt from "BeheerEenheid" where "End2End" is not null) as fence0, (/* ELrs (EDcD berichttype[Koppeling*Berichttype],EDcD ber[Deelroute*Berichttype]) */ /* Expression: berichttype [Koppeling*Berichttype]/ber [Deelroute*Berichttype] */ /* Signature : [Koppeling*Deelroute] */ /* case: ELrs (l,r) */ /* Expression: berichttype [Koppeling*Berichttype]/ber [Deelroute*Berichttype] */ /* Signature : [Koppeling*Deelroute] */ /* EFlp (ERrs (EFlp (EDcD ber[Deelroute*Berichttype]),EFlp (EDcD berichttype[Koppeling*Berichttype]))) */ /* Expression: ber [Deelroute*Berichttype]~\berichttype [Koppeling*Berichttype]~~ */ /* Signature : [Koppeling*Deelroute] */ /* case: EFlp x */ /* Expression: ber [Deelroute*Berichttype]~\berichttype [Koppeling*Berichttype]~~ */ /* Signature : [Koppeling*Deelroute] */ /* ERrs (EFlp (EDcD ber[Deelroute*Berichttype]),EFlp (EDcD berichttype[Koppeling*Berichttype])) */ /* Expression: ber [Deelroute*Berichttype]~\berichttype [Koppeling*Berichttype]~ */ /* Signature : [Deelroute*Koppeling] */ /* Expression: ber [Deelroute*Berichttype]~\berichttype [Koppeling*Berichttype]~~ */ /* Signature : [Koppeling*Deelroute] */ /* case: ERrs (l,r) */ /* Expression: ber [Deelroute*Berichttype]~\berichttype [Koppeling*Berichttype]~ */ /* Signature : [Deelroute*Koppeling] */ /* Expression: ber [Deelroute*Berichttype]~\berichttype [Koppeling*Berichttype]~~ */ /* Signature : [Koppeling*Deelroute] */ select RResRight."Koppeling" as src, RResLeft."Deelroute" as tgt from "Deelroute" as RResLeft, "BeheerEenheid" as RResRight where /* Left hand side: ber [Deelroute*Berichttype]~ */ NOT exists (select * from (/* EFlp (EDcD ber[Deelroute*Berichttype]) */ /* Expression: ber [Deelroute*Berichttype]~ */ /* Signature : [Berichttype*Deelroute] */ /* case: EFlp x */ /* Expression: ber [Deelroute*Berichttype]~ */ /* Signature : [Berichttype*Deelroute] */ /* EDcD ber[Deelroute*Berichttype] */ /* Expression: ber [Deelroute*Berichttype] */ /* Signature : [Deelroute*Berichttype] */ /* Expression: ber [Deelroute*Berichttype]~ */ /* Signature : [Berichttype*Deelroute] */ select "ber" as src, "Deelroute" as tgt from "Deelroute" where ("Deelroute" is not null) and ("ber" is not null)) as lhs where (RResLeft."Deelroute" = lhs.tgt) and (/* Right hand side: berichttype [Koppeling*Berichttype]~ */ NOT exists (select * from (/* EFlp (EDcD berichttype[Koppeling*Berichttype]) */ /* Expression: berichttype [Koppeling*Berichttype]~ */ /* Signature : [Berichttype*Koppeling] */ /* case: EFlp x */ /* Expression: berichttype [Koppeling*Berichttype]~ */ /* Signature : [Berichttype*Koppeling] */ /* EDcD berichttype[Koppeling*Berichttype] */ /* Expression: berichttype [Koppeling*Berichttype] */ /* Signature : [Koppeling*Berichttype] */ /* Expression: berichttype [Koppeling*Berichttype]~ */ /* Signature : [Berichttype*Koppeling] */ select "Berichttype" as src, "Koppeling" as tgt from "berichttype1" where ("Koppeling" is not null) and ("Berichttype" is not null)) as rhs where (rhs.src = lhs.src) and (rhs.tgt = RResRight."Koppeling"))))) as fence1 where (fence0.tgt = fence1.src)) as subIntersect2 where (subIntersect1.src = subIntersect0.src) and (subIntersect1.tgt = subIntersect0.tgt) and (subIntersect2.src = subIntersect0.src) and (subIntersect2.tgt = subIntersect0.tgt)) as someDummyNameBecauseMySQLNeedsOne) as fence1 where (fence0.tgt = fence1.src)) as t2 on (t1.src = t2.src) and (t1.tgt = t2.tgt) where (t2.src is null) or (t2.tgt is null)) as subIntersect2 where (subIntersect1.src = subIntersect0.src) and (subIntersect1.tgt = subIntersect0.tgt) and (subIntersect2.src = subIntersect0.src) and (subIntersect2.tgt = subIntersect0.tgt)) as someDummyNameBecauseMySQLNeedsOne []

Help to reproduce

Here is a small script, meant to reproduce the SQL query and experiment with it.
I reckon you can remove relation t and its occurrences to make it even smaller without spoiling the essence of the experiment, but I'm not 100% sure (also: The SQL optimization might influence the results).

CONTEXT "Issue1409"

RELATION l[X*A]
RELATION r[X*A]
RELATION t[X*Type]
RELATION van[K*A] [UNI, TOT]
RELATION naar[K*A] [UNI, TOT]
RELATION t[K*Type] [UNI, TOT]
RELATION l[Y*A] [UNI, TOT]
RELATION r[Y*A] [UNI, TOT]
RELATION t[Y*Type] [UNI, TOT]

ROLE ExecEngine MAINTAINS "startpuntVanRoutes", deelroutes

RULE startpuntVanRoutes :
   I[X] /\ l;l~ /\ r;r~ /\ t;t~
    |- (l;l~ /\ r;r~ /\ t;t~) ; I[Y] ; (l/l /\ r/r /\ t/t)~
VIOLATION (TXT "{EX} NewStruct;Y"
               ,TXT ";l;Y;_NEW;A;", SRC l
               ,TXT ";r;Y;_NEW;A;", SRC r
               ,TXT ";t;Y;_NEW;Type;", SRC t
          )

RULE deelroutes : (l[Y*A];van~/\t;t~);I[K];naar-r |-
    (r;r~/\t;t~);I[Y];l
VIOLATION (TXT "{EX} NewStruct;Y"
               ,TXT ";t;Y;_NEW;Type;", SRC t
               ,TXT ";l;Y;_NEW;A;", TGT I[A]
               ,TXT ";r;Y;_NEW;A;", SRC r
          )

ENDCONTEXT

I tried this script on rap.cs.ou.nl with Issue1409.xlsx as input. That should give sufficient queues for experimentation, to reproduce the efficiency problem.