Regression Testing MSL 3.2.2-beta.3

[modelica-trac-importer]

Reported by leo.gall on 3 Mar 2016 15:12 UTC
This ticket collects result deviations of MSL 3.2.2-beta.3.
Library officers please inspect.

Reference:

• MSL v3.2.1+build.4
• Dymola 2014 FD01

To be tested:

• MSL v3.2.2+build.0-beta.3
• Dymola 2017 Beta 1

Methods for regression testing: Own mos-scripting + MA compare.exe (pre-release version 2.0.0.1) for comparison.

Number of test cases (compared to v3.2.2-beta.2):

• Modelica: 366 (unchanged)
• ModelicaTest: 518 compared to 543 (6 added due to new stop time annotations, 31 moved to ModelicaTestOverdetermined.mo)
  For new test cases, results have been generated, which can be used as a baseline for comparing to other tools.

Blocks

ModelicaTest.Blocks.Mean

• compare report
• already explained for beta.2 in ticket:1924#comment:4

Electrical.Machines

Modelica.Electrical.Analog.Examples.AmplifierWithOpAmpDetailed

• compare report
• already explained for beta.2 in ticket:1924#comment:6

Mechanics

Modelica.Mechanics.MultiBody.Examples.Systems.RobotR3.fullRobot

• compare.exe works now, thanks to fixed pre-release-version 2.0.0.1.
• compare report
• already explained for beta.2 in ticket:1924#comment:9

Modelica.Mechanics.Rotational.Examples.HeatLosses

• compare.exe works now, thanks to fixed pre-release-version 2.0.0.1.
• compare report
• already explained for beta.2 in ticket:1924#comment:7

Modelica.MultiBody.Joints.JointUSP2

• compare report
• already explained for beta.2 in ticket:1924#comment:10

ModelicaTest.MultiBody.Forces.SpringDamperSeries

• No stop time annotation. Please add experiment(StopTime=1.0)

ModelicaTest.MultiBody.Joints.CheckConstaintTorqueUniversalJoint

• No stop time annotation. Please add experiment(StopTime=1.0)

Fluid

Modelica.Fluid.Examples.Tanks.TanksWithOverflow

• New deviation in beta.3
• medium model has been changed in 1aca4c0, therefore deviation was probably expected?
• compare report

ModelicaTest.Fluid.Dissipation.Verifications.PressureLoss.Bend.dp_edgedOverall_DPMFLOW

• No stop time annotation. Please add experiment(StopTime=1.0)

Access to full comparison reports

Access comparison reports of all test cases (HTML generated by MA/ITI compare.exe v2.0.0.1):
Modelica
ModelicaTest

Basic documentation of the testing process is available here:
https://trac.modelica.org/Modelica/wiki/HowTo/MAP-LIB/ModelicaStandardLibrary/RegressionTesting

Access to result files

MA file server (including mat files):
[sftp://modelica.org:20022/files/RegressionTesting/ReferenceResults/MSL/tags/v3.2.2%2Bbuild.0-beta.3/]
Attention: upload is now started and will take some hours. I'll comment this ticket a soon as upload is finished.

Readme for accessing file server:
https://svn.modelica.org/projects/RegressionTesting/AccessInfo/read-access/README.html

SVN (csv files, r9071)
https://svn.modelica.org/projects/RegressionTesting/ReferenceResults/MSL/trunk
Attention: This does not yet contain results from beta.3. Should be updated later, based on the released version.

If you have any questions replicating the test results, please don't hesitate to contact [mailto:leo.gall@ltx.de me]!

---

Migrated-From: https://trac.modelica.org/Modelica/ticket/1949

[modelica-trac-importer]

Comment by otter on 3 Mar 2016 15:43 UTC
StopTime annotations added in c215591:

• ModelicaTest.MultiBody.Forces.SpringDamperSeries
• ModelicaTest.MultiBody.Joints.CheckConstaintTorqueUniversalJoint
• ModelicaTest.Fluid.Dissipation.Verifications.PressureLoss.Bend.dp_edgedOverall_DPMFLOW

Modelica.Fluid.Examples.Tanks.TanksWithOverflow:
medium model has been changed in 1aca4c0 and therefore there is a deviation

To summarize, all regressions are explained and fine.

[modelica-trac-importer]

Comment by leo.gall on 3 Mar 2016 16:57 UTC
Upload of results is now finished.
​sftp://modelica.org:20022/files/RegressionTesting/ReferenceResults/MSL/tags/v3.2.2%2Bbuild.0-beta.3/

[modelica-trac-importer]

Comment by fcasella on 3 Mar 2016 17:42 UTC
Replying to [comment:1 otter]:

medium model has been changed in 1aca4c0 and therefore there is a deviation

Confirmed.

[modelica-trac-importer]

Comment by otter on 5 Mar 2016 08:10 UTC
Closing, since regression testing was successful