Approach for updating domain-specific model equivalents

[lvanfretti]

@tinrabuzin @MaximeBaudette I would like to address an important issue regarding the approach when changing models that have been through a sw-to-sw validation.

When modifying some "weird" blocks, or something of the model, as you are doing now, we have to be very careful not to affect models related to a domain-specific tool.

If the model has been validated against PSS/E and Eurostag, and modifications are made, we would have to do the validation against the domain specific tool again or compare with the previous implementation.

Otherwise, we can't argue that we can produce similar results than domain specific tools, and this can have a negative impact on user adoption. Remember that one key goal to make this library successful is that the users can have confidence that they will get the same (or better) results than their tool of choice. If we do not take care of this now, we will be faced with more resistance to change from the power system community, and the library will have the same fate as others in the past.

We need to establish a procedure to do this.

I open this issue so that you guys can discuss with @petitrenaudseb @dietmarw @AIAitesla on how to approach this. One suggestion is that we document this through the issue tracker: 1 model = 1 issue.

@jbheyberger and @geofjamg please add your comments about this also.

I am assigning @tinrabuzin and @MaximeBaudette to follow up on this.

[tinrabuzin]

@lvanfretti

There is another approach. If we validate the behaviour of the transfer functions with respect to the old ones then the behaviour of the changed models should not change.

Another issue is the fact that some of the validated models were using the blocks which did not behave properly. E.g., I found that the old lead-lag filter, which had anti windup limits, did not follow the correct limits.

[lvanfretti]

What I would like to see is that there is a procedure to follow when doing changes, the particularities are up to you to decide on how to do. Most important is that these changes are well justified, documented and validated.

I'm not trying to do things complicated, I just want to avoid potential criticisms and biases from the traditional power system user.

[MaximeBaudette]

@lvanfretti
My idea about the subject is that with use the Examples in the library to put all the validation test systems. We should also have some components that reads a file containing the curves from the domain specific tool, so it can easily be run and compared.

[tinrabuzin]

I was thinking about including the sw-to-sw validation along with the compliance testing that I've done. I just have to check how the PSS/E interface with python works so maybe we can automate this. I just have to dedicate some time to it.

[lvanfretti]

@tinrabuzin This is a very good approach.

We have extensive documentation and examples on the PSS/E Python API.
Ask @MaximeBaudette to give you the material and Giuseppe, who has a lot of experience with the Python API.

Eurostag also has a python API.
Maybe @petitrenaudseb from RTE can help you set this up for Eurostag.

Luigi

[lvanfretti]

I am closing this item as we will not continue to support iPSL within my team.