Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NB unmatched variables and equations in PowerGrids #11556

Closed
casella opened this issue Nov 11, 2023 · 18 comments · Fixed by #11827
Closed

NB unmatched variables and equations in PowerGrids #11556

casella opened this issue Nov 11, 2023 · 18 comments · Fixed by #11827
Assignees
Labels
COMP/OMC/New Backend Issues and pull requests related to the new backend.

Comments

@casella
Copy link
Contributor

casella commented Nov 11, 2023

Description

Please check PowerGrids.Electrical.Test.OneBusOneLoad. This is the simplest phasor-based model of the library. It has 83 variables and equations.

The NB currently fails during the matching phase with:

Error: Internal error NBResolveSingularities.noIndexReduction failed.
(8) Unmatched Variables
*************************
        [ALGB] (1) redeclare Real infiniteBus.port.iGen.re (start = -infiniteBus.port.iStart.re, nominal = infiniteBus.port.IBase)
         slice: {}
        [ALGB] (1) Real infiniteBus.terminal.v.im (start = infiniteBus.port.vStart.im, nominal = 1e4)
         slice: {}
        [ALGB] (1) redeclare Real infiniteBus.port.iGen.im (start = -infiniteBus.port.iStart.im, nominal = infiniteBus.port.IBase)
         slice: {}
        [ALGB] (1) Real infiniteBus.terminal.v.re (start = infiniteBus.port.vStart.re, nominal = 1e4)
         slice: {}
        [ALGB] (1) Real infiniteBus.port.VPu = (1e4 * infiniteBus.port.VPu) / 1e4 (start = infiniteBus.port.VStart / infiniteBus.port.VBase)
         slice: {}
        [ALGB] (1) redeclare Real infiniteBus.v.im = infiniteBus.port.v.im (nominal = infiniteBus.port.VBase)
         slice: {}
        [ALGB] (1) input Real infiniteBus.port.v.re (start = infiniteBus.port.vStart.re, nominal = infiniteBus.port.VBase)
         slice: {}
        [ALGB] (1) redeclare Real infiniteBus.v.re = infiniteBus.port.v.re (nominal = infiniteBus.port.VBase)
         slice: {}

(3) Unmatched Equations
*************************
        [RECD] (2) infiniteBus.v = infiniteBus.port.v ($RES_BND_54)
         slice: {}
        [RECD] (2) infiniteBus.i = infiniteBus.port.i ($RES_BND_55)
         slice: {}
        [RECD] (2) infiniteBus.port.v = infiniteBus.terminal.v ($RES_SIM_7)
         slice: {}

@kabdelhak can you please try to figure out why the matching fails? Once this model works, we may have ScalableTestGrids running with the NB.

@casella casella added the COMP/OMC/New Backend Issues and pull requests related to the new backend. label Nov 11, 2023
@kabdelhak
Copy link
Contributor

kabdelhak commented Nov 13, 2023

It seems like input variables are still considered unknown. Will have a look!

Or maybe this is still correct. It is not a top-level input, so it is unknown right?

@casella
Copy link
Contributor Author

casella commented Nov 13, 2023

The flat Modelica output of that model has this declaration

  Real infiniteBus.port.v.re(quantity = "ElectricPotential", unit = "V", displayUnit = "kV", start = infiniteBus.port.vStart.re, nominal = infiniteBus.port.VBase) "Real part of complex number";

I can't see any input prefix there, nor anywhere else in the model.

@perost
Copy link
Member

perost commented Nov 13, 2023

For the old backend the frontend modifies the variable direction here when generating the DAE structure. Potentially we should move that to an earlier phase so it applies to both backends.

@casella
Copy link
Contributor Author

casella commented Nov 13, 2023

I agree.

The issue in PowerGrids.Electrical.Test.OneBusOneLoad is probably stemming from the InfiniteBus model. The BaseBus model has two inputs (e and Z), but they are bound to a variable when extending it in InfiniteBus.

I guess simply removing the input prefix for non-top-level inputs is enough to handle these cases.

@perost, can you please go ahead with that? This may be the last pending issue before we can actually run PowerGrids with the NB.

In fact, as soon as we can do that, we should also solve #10034, otherwise the dense solver won't be able to handle models with more than a few dozen buses, but that's a runtime issue.

@casella
Copy link
Contributor Author

casella commented Nov 13, 2023

@perost if you can try it today and we are lucky, I could actually mention running PowerGrids in the report for RTE, which would be great. There's time until tomorrow to do that 😅 .

You can use the model mentioned at the beginning of the ticket as a test case.

@perost
Copy link
Member

perost commented Nov 13, 2023

@perost if you can try it today and we are lucky, I could actually mention running PowerGrids in the report for RTE, which would be great. There's time until tomorrow to do that 😅 .

You can use the model mentioned at the beginning of the ticket as a test case.

Implemented in #11562, but it seems we're not lucky because it doesn't seem to change much. The NB still thinks that infiniteBus.port.v.re is an input for some reason, even though infiniteBus.port.v in the flat model no longer is. I assume it's getting the direction from somewhere other than the Variable in the flat model.

I also tried removing the direction from the children of the Variable:s, but it also made no difference. I didn't include that in my PR since I'm not sure if it matters or not, maybe @kabdelhak has some opinion on that.

But regardless it seems this is now an issue with the NB that we need to look into rather than a frontend issue.

@casella
Copy link
Contributor Author

casella commented Nov 13, 2023

@kabdelhak please check the following MWE:

package NonTopLevelInputs
  model M1
    input Real e;
  end M1;

  model M2
    parameter Real p = 1;
    extends M1(e = p);
  end M2;

  model S
  M2 m2;
    annotation(
      __OpenModelica_commandLineOptions = "--newBackend");
  end S;

  model M1c
    input Complex e;
  end M1c;

  model M2c
    parameter Complex p = Complex(1);
    extends M1c(e = p);
  end M2c;

  model Sc
  M2c m2;
    annotation(
      __OpenModelica_commandLineOptions = "--newBackend");
  end Sc;

  model M1cv
    Complex e;
  end M1cv;

  model M2cv
    parameter Complex p = Complex(1);
    extends M1cv(e = p);
  end M2cv;

  model Scv
    M2cv m2;
    annotation(
      __OpenModelica_commandLineOptions = "--newBackend");
  end Scv;
  
end NonTopLevelInputs;

Model S has a non-top-level Real input and works fine.

Model Sc has a non-top-level Complex input and fails with the usual matching error:

Internal error NBResolveSingularities.noIndexReduction failed.
(2) Unmatched Variables
*************************
        [ALGB] (1) input Real m2.e.im = m2.p.im
         slice: {}
        [ALGB] (1) input Real m2.e.re = m2.p.re
         slice: {} (1) Unmatched Equations
*************************
        [RECD] (2) m2.e = m2.p ($RES_BND_0)
         slice: {} 

However, also model Scv, which has no inputs anywhere, fails exactly in the same way

Internal error NBResolveSingularities.noIndexReduction failed.
(2) Unmatched Variables
*************************
        [ALGB] (1) Real m2.e.im = m2.p.im
         slice: {}
        [ALGB] (1) Real m2.e.re = m2.p.re
         slice: {} (1) Unmatched Equations
*************************
        [RECD] (2) m2.e = m2.p ($RES_BND_0)
         slice: {} 

So, I guess the problem is not the non-top-level inputs, but rather the [ALGB] variables not being matched correctly with the [RECD] variables.

If I flatten Scv with --newBackend, I get:

class NonTopLevelInputs.Scv
  parameter Complex m2.p = Complex(1.0, 0.0);
  Complex m2.e = m2.p;
end NonTopLevelInputs.Scv;

so there is probably some pending issue with how the NB handles record equations. For Complex numbers, it is always convenient to split such equations in their real and imaginary counterparts.

@kabdelhak can you please have a look?

kabdelhak added a commit to kabdelhak/OpenModelica that referenced this issue Nov 16, 2023
 - make correct adjacency matrix entries for records
 - split record equations to their attribute equations once converted to statements (code gen cant handle records on the rhs apparently)
 - add testsuite example
 - fixes minimal example of OpenModelica#11556
casella pushed a commit that referenced this issue Nov 16, 2023
* [NB] update record handling

 - make correct adjacency matrix entries for records
 - split record equations to their attribute equations once converted to statements (code gen cant handle records on the rhs apparently)
 - add testsuite example
 - fixes minimal example of #11556

* [testuite] update test file
@casella
Copy link
Contributor Author

casella commented Nov 17, 2023

The PowerGrids.Electrical.Test.OneBusOneLoad model now fails with

Error: Internal error NBResolveSingularities.noIndexReduction failed.
(2) Unmatched Variables
*************************
        [ALGB] (1) redeclare Real infiniteBus.port.iGen.re (start = -infiniteBus.port.iStart.re, nominal = infiniteBus.port.IBase)
         slice: {}
        [ALGB] (1) redeclare Real infiniteBus.port.iGen.im (start = -infiniteBus.port.iStart.im, nominal = infiniteBus.port.IBase)
         slice: {}

(0) Unmatched Equations
*************************

Initially we had 8 unmatched scalar variables and 6 unmatched scalar equations, now we are left with just two unmatched scalar variables, so there were actually two separate issues in this ticket. #11585 fixed the first one, I hope the other one is the last before victory 😄

@casella
Copy link
Contributor Author

casella commented Dec 29, 2023

Please also check PowerGrids.Examples.ENTSOE.PowerFlow, which gives:

(2) Unmatched Variables
*************************
        [ALGB] (1) Real TGEN.portA.IPu = TGEN.portA.I / TGEN.portA.IBase (start = TGEN.portA.IStart / TGEN.portA.IBase)
         slice: {}
        [ALGB] (1) Real GEN.terminalAC.v.re (start = GEN.port.vStart.re, nominal = 1e4)
         slice: {}

(2) Unmatched Equations
*************************
        [SCAL] (1) 1.7320508075688772 * GRID.port.v.re = GRID.U * cos(GRID.UPhase) ($RES_SIM_243)
         slice: {}
        [SCAL] (1) 1.7320508075688772 * GRID.port.v.im = GRID.U * sin(GRID.UPhase) ($RES_SIM_244)
         slice: {}

@kabdelhak
Copy link
Contributor

kabdelhak commented Jan 18, 2024

The original problem is fixed with #11827.

I did not verify the results yet, but it runs. Despite that it will still produce following message due to consistency checks not being implemented yet:

"Warning: NBResolveSingularities.balanceInitialization reports an overdetermined initialization!
Checking for consistency is not yet supported, following equations had to be removed:
        [SCAL] (1) loadPQ.QStart = loadPQ.QRefConst ($RES_BND_106)
         slice: {};
        [SCAL] (1) loadPQ.PStart = loadPQ.PRefConst ($RES_BND_105)
         slice: {};
        [SCAL] (1) infiniteBus.port.UPhaseStart = infiniteBus.UPhaseStart ($RES_BND_98)
         slice: {};
        [SCAL] (1) infiniteBus.port.QStart = infiniteBus.QStart ($RES_BND_97)
         slice: {};
        [SCAL] (1) loadPQ.port.vStart.re = 5773.502691896258 * cos(loadPQ.port.UPhaseStart) ($RES_SIM_131)
         slice: {};
        [SCAL] (1) loadPQ.port.vStart.im = 5773.502691896258 * sin(loadPQ.port.UPhaseStart) ($RES_SIM_132)
         slice: {};
        [SCAL] (1) infiniteBus.port.vStart.re = 5773.502691896258 * cos(infiniteBus.port.UPhaseStart) ($RES_SIM_135)
         slice: {};
        [SCAL] (1) infiniteBus.port.vStart.im = 5773.502691896258 * sin(infiniteBus.port.UPhaseStart) ($RES_SIM_136)
         slice: {};

@casella
Copy link
Contributor Author

casella commented Jan 19, 2024

The original problem is fixed with #11827.

Great news! 😃

I did not verify the results yet, but it runs.

Yeah, I should generate reference results with the old backend and upload them somewhere. Not enough time to do it.

Despite that it will still produce following message due to consistency checks not being implemented yet:

"Warning: NBResolveSingularities.balanceInitialization reports an overdetermined initialization!
Checking for consistency is not yet supported, following equations had to be removed:
        [SCAL] (1) loadPQ.QStart = loadPQ.QRefConst ($RES_BND_106)
         slice: {};
        [SCAL] (1) loadPQ.PStart = loadPQ.PRefConst ($RES_BND_105)
         slice: {};
        [SCAL] (1) infiniteBus.port.UPhaseStart = infiniteBus.UPhaseStart ($RES_BND_98)
         slice: {};
        [SCAL] (1) infiniteBus.port.QStart = infiniteBus.QStart ($RES_BND_97)
         slice: {};
        [SCAL] (1) loadPQ.port.vStart.re = 5773.502691896258 * cos(loadPQ.port.UPhaseStart) ($RES_SIM_131)
         slice: {};
        [SCAL] (1) loadPQ.port.vStart.im = 5773.502691896258 * sin(loadPQ.port.UPhaseStart) ($RES_SIM_132)
         slice: {};
        [SCAL] (1) infiniteBus.port.vStart.re = 5773.502691896258 * cos(infiniteBus.port.UPhaseStart) ($RES_SIM_135)
         slice: {};
        [SCAL] (1) infiniteBus.port.vStart.im = 5773.502691896258 * sin(infiniteBus.port.UPhaseStart) ($RES_SIM_136)
         slice: {};

That's weird, these models should not have any overdetermined initialization, nor high index. In fact, if you check the report with the OB, there is no warning of redundant initial equations needing to be removed. I guess there is still some equation counting issue in the initial equations or something.

@casella
Copy link
Contributor Author

casella commented Jan 19, 2024

BTW, Adrian fixed the regression reports, so now we have the one regarding the newInst-newBackend job of the 16th: https://libraries.openmodelica.org/branches/history/newInst-newBackend/2024-01-07%2020:12:41..2024-01-16%2017:34:57.html

The two improvements were: PowerSystemStepLoad tests now simulate, and TransmissionLineEquations tests now compile (but then fail at time = 0).

I'll start another newInst-newBackend job right away, so we check the effects of your latest commit.

@casella
Copy link
Contributor Author

casella commented Jan 20, 2024

Here is the last regression report. Some progress, but nothing dramatic.

@kabdelhak
Copy link
Contributor

The PR didn't go through, i pushed the last changes and it should be merged to main in an hour. We can do another library test to check. But we can also wait for more changes, planning on several small fixes this week.

@kabdelhak
Copy link
Contributor

That's weird, these models should not have any overdetermined initialization, nor high index. In fact, if you check the report with the OB, there is no warning of redundant initial equations needing to be removed. I guess there is still some equation counting issue in the initial equations or something.

My guess is that i might be creating more initial equations than necessary. Either due to record bindings which might be created for the records themselves and their elements which leads to basically duplicated equations, or maybe i don't realize some parameters have fixed = false and are supposed to be solved by initial equations instead.

Either way, i will have a look.

kabdelhak added a commit that referenced this issue Jan 22, 2024
* [NB] update alias module to not use record elements

[NSimCode] draft: inline record residuals

* [NB] add variable record children when adding records

* [NB] inline record residual equations

 - fixes #11556

* [testsuite] udapte tests

 - alias module does not apply to record elements anymore
@kabdelhak
Copy link
Contributor

Everything is merged, my PR automatically closed this issue, feel free to reopen if its still relevant.

Using -d=initialization shows following binding equations being created:

Created Parameter Binding Equations (28):
*******************************************
        [SCAL] (1) systemPowerGrids.omegaRef = systemPowerGrids.omegaNom ($RES_BND_124)
        [SCAL] (1) systemPowerGrids.omegaRefInternal = systemPowerGrids.omegaNom ($RES_BND_123)
        [SCAL] (1) loadPQ.QRef = loadPQ.QRefConst ($RES_BND_122)
        [SCAL] (1) loadPQ.PRef = loadPQ.PRefConst ($RES_BND_121)
        [SCAL] (1) loadPQ.port.IGenPu = 0.0 ($RES_BND_120)
        [SCAL] (1) infiniteBus.port.IGenPu = 0.0 ($RES_BND_119)
        [SCAL] (1) infiniteBus.port.IPhase = 0.0 ($RES_BND_118)
        [SCAL] (1) infiniteBus.port.UPhase = 0.0 ($RES_BND_117)
        [SCAL] (1) systemPowerGrids.omegaNom = 6.283185307179586 * systemPowerGrids.fNom ($RES_BND_116)
        [SCAL] (1) loadPQ.port.iStart.re = (3.0 * loadPQ.port.QStart * loadPQ.port.vStart.im + 3.0 * loadPQ.port.PStart * loadPQ.port.vStart.re) / (9.0 * loadPQ.port.vStart.im * loadPQ.port.vStart.im + 9.0 * loadPQ.port.vStart.re * loadPQ.port.vStart.re) ($RES_BND_115)
        [SCAL] (1) loadPQ.port.iStart.im = -(3.0 * loadPQ.port.vStart.re * loadPQ.port.QStart - 3.0 * loadPQ.port.PStart * loadPQ.port.vStart.im) / (9.0 * loadPQ.port.vStart.re * loadPQ.port.vStart.re + 9.0 * loadPQ.port.vStart.im * loadPQ.port.vStart.im) ($RES_BND_114)
        [SCAL] (1) loadPQ.port.vStart.re = 5773.502691896258 * cos(loadPQ.port.UPhaseStart) ($RES_BND_113)
        [SCAL] (1) loadPQ.port.vStart.im = 5773.502691896258 * sin(loadPQ.port.UPhaseStart) ($RES_BND_112)
        [SCAL] (1) loadPQ.port.IStart = (0.3333333333333333 * sqrt(loadPQ.port.PStart ^ 2.0 + loadPQ.port.QStart ^ 2.0)) / loadPQ.port.VStart ($RES_BND_111)
        [SCAL] (1) loadPQ.port.IBase = 3.3333333333333335e6 / loadPQ.port.VBase ($RES_BND_110)
        [SCAL] (1) loadPQ.port.UPhaseStart = loadPQ.UPhaseStart ($RES_BND_109)
        [SCAL] (1) loadPQ.port.QStart = loadPQ.QStart ($RES_BND_108)
        [SCAL] (1) loadPQ.port.PStart = loadPQ.PStart ($RES_BND_107)
        [SCAL] (1) loadPQ.QStart = loadPQ.QRefConst ($RES_BND_106)
        [SCAL] (1) loadPQ.PStart = loadPQ.PRefConst ($RES_BND_105)
        [SCAL] (1) infiniteBus.port.iStart.re = (3.0 * infiniteBus.port.QStart * infiniteBus.port.vStart.im + 3e7 * infiniteBus.port.vStart.re) / (9.0 * infiniteBus.port.vStart.im * infiniteBus.port.vStart.im + 9.0 * infiniteBus.port.vStart.re * infiniteBus.port.vStart.re) ($RES_BND_104)
        [SCAL] (1) infiniteBus.port.iStart.im = -(3.0 * infiniteBus.port.vStart.re * infiniteBus.port.QStart - 3e7 * infiniteBus.port.vStart.im) / (9.0 * infiniteBus.port.vStart.re * infiniteBus.port.vStart.re + 9.0 * infiniteBus.port.vStart.im * infiniteBus.port.vStart.im) ($RES_BND_103)
        [SCAL] (1) infiniteBus.port.vStart.re = 5773.502691896258 * cos(infiniteBus.port.UPhaseStart) ($RES_BND_102)
        [SCAL] (1) infiniteBus.port.vStart.im = 5773.502691896258 * sin(infiniteBus.port.UPhaseStart) ($RES_BND_101)
        [SCAL] (1) infiniteBus.port.IStart = (0.3333333333333333 * sqrt(1e14 + infiniteBus.port.QStart ^ 2.0)) / infiniteBus.port.VStart ($RES_BND_100)
        [SCAL] (1) infiniteBus.port.IBase = 3.3333333333333335e6 / infiniteBus.port.VBase ($RES_BND_99)
        [SCAL] (1) infiniteBus.port.UPhaseStart = infiniteBus.UPhaseStart ($RES_BND_98)
        [SCAL] (1) infiniteBus.port.QStart = infiniteBus.QStart ($RES_BND_97)

Created Record Parameter Binding Equations (4):
*************************************************
        [RECD] (2) loadPQ.port.iStart = Modelica.ComplexMath.conj(Complex.'/'(Complex.'constructor'.fromReal(loadPQ.port.PStart, loadPQ.port.QStart), Complex.'*'.multiply(Complex(3.0, 0.0), loadPQ.port.vStart))) ($RES_BND_128)
        [RECD] (2) loadPQ.port.vStart = Modelica.ComplexMath.fromPolar(5773.502691896258, loadPQ.port.UPhaseStart) ($RES_BND_127)
        [RECD] (2) infiniteBus.port.iStart = Modelica.ComplexMath.conj(Complex.'/'(Complex.'constructor'.fromReal(1e7, infiniteBus.port.QStart), Complex.'*'.multiply(Complex(3.0, 0.0), infiniteBus.port.vStart))) ($RES_BND_126)
        [RECD] (2) infiniteBus.port.vStart = Modelica.ComplexMath.fromPolar(5773.502691896258, infiniteBus.port.UPhaseStart) ($RES_BND_125)

We can see that there are multiple binding equations for the same thing. Record elements:

        [SCAL] (1) loadPQ.port.iStart.re = (3.0 * loadPQ.port.QStart * loadPQ.port.vStart.im + 3.0 * loadPQ.port.PStart * loadPQ.port.vStart.re) / (9.0 * loadPQ.port.vStart.im * loadPQ.port.vStart.im + 9.0 * loadPQ.port.vStart.re * loadPQ.port.vStart.re) ($RES_BND_115)
        [SCAL] (1) loadPQ.port.iStart.im = -(3.0 * loadPQ.port.vStart.re * loadPQ.port.QStart - 3.0 * loadPQ.port.PStart * loadPQ.port.vStart.im) / (9.0 * loadPQ.port.vStart.re * loadPQ.port.vStart.re + 9.0 * loadPQ.port.vStart.im * loadPQ.port.vStart.im) ($RES_BND_114)

And full record:

        [RECD] (2) loadPQ.port.iStart = Modelica.ComplexMath.conj(Complex.'/'(Complex.'constructor'.fromReal(loadPQ.port.PStart, loadPQ.port.QStart), Complex.'*'.multiply(Complex(3.0, 0.0), loadPQ.port.vStart))) ($RES_BND_128)

Which seems like they could be the same equations. I will look and see why there are duplicate bindings, probably expanding the records does that.

@casella
Copy link
Contributor Author

casella commented Jan 22, 2024

I just started a newInst-newBackend job to test it on the whole library, as soon as the job is finished, the PowerGrids report will be here.

@casella
Copy link
Contributor Author

casella commented Jan 22, 2024

@kabdelhak, I also created #11852 for the duplication issue, so we keep track of it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
COMP/OMC/New Backend Issues and pull requests related to the new backend.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants