Bug: Older AutoKoopman is More Accurate

[EthanJamesLew]

I have been trying to run the Koopman falsification code on a different machine and have noticed quite different results. The main branch is giving different results than the version I have (which I installed straight after the fix you provided regarding the time step). I am attaching two figures highlighting this, with the result after one iteration learning the Vanderpool model. The old version learns a very accurate representation whereas the current main does not. I haven’t been able to locate the source of error yet so figured it was best to flag it with you.

[EthanJamesLew]

@Abdu-Hekal

[EthanJamesLew]

@Abdu-Hekal to provide a timeline: I will look at this tomorrow evening. I am quite swamped in other things atm. Sorry for the delay.

[Abdu-Hekal]

No worries, working on the older version for now!

[EthanJamesLew]

CC @KochdumperNiklas

[EthanJamesLew]

@Abdu-Hekal can you help me reproduce your issue? I have attempted to test autokoopman for the case you show, notebook here: https://colab.research.google.com/drive/18UPHoEZ0kTxknfUcwqNJarFcr_cOBtb8?usp=sharing

[Abdu-Hekal]

@EthanJamesLew I have updated the notebook to reproduce the issue. This issue seems to happen when there is a lesser number of trajectories (I have changed training data to 4 trajectories).

[EthanJamesLew]

@Abdu-Hekal Awesome! I see this

However, the new training data got deleted when starting a new colab runtime. May you please email me the data?

[Abdu-Hekal]

Sure thing! Are these the ones you are after? From: Ethan Lew ***@***.***> Date: Wednesday, 7 June 2023 at 9:00 PM To: EthanJamesLew/AutoKoopman ***@***.***> Cc: Abdelrahman Waleed Elsayed Aly Hekal (PGR) ***@***.***>, Mention ***@***.***> Subject: Re: [EthanJamesLew/AutoKoopman] Bug: Older AutoKoopman is More Accurate (Issue #70) ⚠ External sender. Take care when opening links or attachments. Do not provide your login details. @Abdu-Hekal<https://github.com/Abdu-Hekal> Awesome! I see this [Image removed by sender. download (1)]<https://user-images.githubusercontent.com/15683738/244183475-dd285be2-8958-40da-8046-c46708c9f71e.png> However, the new training data got deleted when starting a new colab runtime. May you please email me the data? — Reply to this email directly, view it on GitHub<#70 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AVZPXM24WI5VCXJ2MQFS7NTXKDMWBANCNFSM6AAAAAAYJOE2UU>. You are receiving this because you were mentioned.Message ID: ***@***.***>

[EthanJamesLew]

We resolved this and the falsification now uses the latest autokoopman