failing tests #8
Comments
|
not sure, I'll look into it. What's odd is it is only failing on some machines... |
|
maybe the |
|
resolved. test and main were out of sync slightly. |
|
@mayork, @1ozturkbe, should we use |
|
No I don't think so. This was just poorly formulated on my part. Fixed inD8. |
|
tests are failing again. |
|
I haven't pushed to this in a while...odd...on vacation now I'll take a look in a few days. |
|
status on this? Sporadic test failures are continuing. |
|
@mayork and/or @1ozturkbe if one of you could prioritize this I'd appreciate it, I really want us to find it unacceptable when tests are failing. Makes us less likely to break things when improving GPkit. |
|
sorry had some family things come up and have had to take an unexpected trip to SC. I think I've narrowed it down to to two possible causes...the fix for once of which is just making a better testing framework within the turbofan repo, which I've started working on. Should be pushed soon. |
|
i made a series of updates that I think should have resolved this issue but it looks like there's now an import error within the model testing code on windows7x64... @bqpd what is going on here? The relevant output is at https://acdl.mit.edu/csi/job/gpkit_ResearchModel_turbofan_Push/buildnode=windows7x64,optimizer=mosek/125/console |
|
re-running that test..seems to be hanging?
…On Mon, Jul 3, 2017 at 2:14 PM, mayork ***@***.***> wrote:
i made a series of updates that I think should have resolved this issue
but it looks like there's now an import error within the model testing code
on windows7x64... @bqpd <https://github.com/bqpd> what is going on here?
The relevant output is at https://acdl.mit.edu/csi/job/
gpkit_ResearchModel_turbofan_Push/buildnode=windows7x64,
optimizer=mosek/125/console
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#8 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/ABagGJ9DErxnJQ-zt2WoUnuUix9GbfFoks5sKVnMgaJpZM4NacV4>
.
|
|
it hung and eventually solved...i just made another push and got the exact same error |
|
the new push seems to be failing on all moseks
…On Mon, Jul 3, 2017 at 2:36 PM, mayork ***@***.***> wrote:
it hung and eventually solved...i just made another push and got the exact
same error
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#8 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/ABagGLP27FVAwAEwaRxh6M8T2k2XTgTTks5sKV7JgaJpZM4NacV4>
.
|
|
resolved that i'll see what happens on the current run |
|
hmm it's all passing now |
|
hmm well got 2 successful builds in a row. I'm going to close this for now, not sure what the import error was but it seems to have resolved itself. I think the turbofan tests should be much more stable now. I removed the possibility of having main and tests run with different substitutions. |
|
Still failing |
|
@bqpd what are your thoughts on this. I'm not sure what to do because I cannot recreate any sort of failing behavior on my machine. |
|
@mayork, this never fails on your machine? |
|
Nope! I almost think there's some issue with testing large SPs |
|
hmm...across multiple platforms? I'll check if I can reproduce this failure & get back to you. |
|
For what's it worth I think the mosek failure is almost exclusively mosek_cli. Not sure why, seems like it shouldn't make a difference. |
|
That's generally a sign of knife-edge... |
|
@bqpd I don't think it's knife edge...but....could the tex file rounding cause some issues for the signommial equalities? That is by far the least reliable constraint and is in both the engine and SP aircraft, the two places we are seeing this issue... |
|
I guess 'knife edge' is a loaded word; I mean there's some effect where the program either converges to a good solution or to NEAR_OPTIMAL/UNKNOWN, and I think that the balance between those depends on very small effects (such as the text file rounding). Signomial Equalities seem more likely to have extreme approximations, so maybe? |
|
@1ozturkbe @bqpd did migrating the repo cause issues with the testing? I pushed to this repo and tests did not run. |
|
I forgot to invite @acdl-jenkins to be an admin in the new org...once their invite is accepted, the webhooks should start working again. |
|
@bqpd, I need to update the jenkins website it self to point to the new URL for this repo |
|
ah, I thought I did that (for turbofan) |
|
yeah I think you did, I just went through and updated everything else. We need a pull request in this repo to test it out though. |
|
Looks like I can manually trigger the push test at least, but we need someone to push to the repo to be sure. |
|
@mayork @1ozturkbe can we please make this more reliable? gpkitmodels is setup on jenkins so every time we push this has to pass and it fails 2/3 of time and is very annoying. |
|
It's because mosek cli has some numerical instability. You can pull the code and run it 1000 times with mosek and it will work every time but the way mosek cli creates and passes mosek files causes sporadic failures. I think this is a testing architecture issue.
… On Oct 14, 2017, at 22:11, Michael Burton ***@***.***> wrote:
@mayork @1ozturkbe can we please make this more reliable? gpkitmodels is setup on jenkins so every time we push this has to pass and it fails 2/3 of time and is very annoying.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.
|
|
Are the files lacking precision for the cli printed with insufficient precision? Can you increase the digits in the floating point numbers? |
|
can you skip that solver? |
|
(you can definitely just add that to skipsolvers in TESTCONFIG) |
|
Marshall yes to you and Ned I'll just do that
… On Oct 14, 2017, at 23:15, bqpd ***@***.***> wrote:
(you can definitely just add that to skipsolvers in TESTCONFIG)
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.
|
@mayork, do you know why this is failing tests? Can we get them passing reliably?
The text was updated successfully, but these errors were encountered: