sig abort after kiwi::InternalSolverError

[simonjamain]

Hello,
I am using kiwi on a project to move temporal element with flexible time intervals between them code here (https://github.com/OSSIA/i-score/tree/dev).

When manipulating, I often get a crash right after kiwi throw an InternalSolverError.

outuput
terminate called after throwing an instance of 'kiwi::InternalSolverError'
  what():  Dual optimize failed.
done debugging.

0   raise           0x7ffff546b5f8  
1   abort           0x7ffff546ca7a  
2   __gnu_cxx::__verbose_terminate_handler  /usr/lib/libstdc++.so.6 95  0x7ffff5d7fb3d  
3   __cxxabiv1::__terminate /usr/lib/libstdc++.so.6 47  0x7ffff5d7d996  
4   __cxa_call_terminate    /usr/lib/libstdc++.so.6 54  0x7ffff5d7c989  
5   __cxxabiv1::__gxx_personality_v0    /usr/lib/libstdc++.so.6 676 0x7ffff5d7d2e5  
6   _Unwind_RaiseException_Phase2   /usr/lib/libgcc_s.so.1  62  0x7ffff57ebed3  
7   _Unwind_Resume  /usr/lib/libgcc_s.so.1  230 0x7ffff57ec3f7  
8   kiwi::impl::SolverImpl::dualOptimize    solverimpl.h    605 0x7fffdb3af380  
9   kiwi::impl::SolverImpl::DualOptimizeGuard::~DualOptimizeGuard   solverimpl.h    58  0x7fffdb3acf7d  
10  kiwi::impl::SolverImpl::suggestValue    solverimpl.h    284 0x7fffdb3adf82  
11  kiwi::Solver::suggestValue  solver.h    120 0x7fffdb3afcdb  
12  CSPDisplacementPolicy::computeDisplacement  CSPDisplacementPolicy.cpp   61  0x7fffdb3dee32  
13  Scenario::Command::MoveEvent<CSPDisplacementPolicy>::update MoveEvent.hpp   106 0x7fffdb3dc588  
14  Scenario::Command::MoveEvent<CSPDisplacementPolicy>::MoveEvent(Path<ScenarioModel>&&, id_base_t<EventModel, boost::optional<int>> const&, TimeValue_T<double> const&, ExpandMode)   MoveEvent.hpp   82  0x7fffdb3dc348  
15  MoveEventCSPFactory::make(Path<ScenarioModel>&&, id_base_t<EventModel, boost::optional<int>> const&, TimeValue_T<double> const&, ExpandMode)    MoveEventCSPFactory.cpp 11  0x7fffdb3dc07c  
16  MoveEventMeta::MoveEventMeta(Path<ScenarioModel>&&, id_base_t<EventModel, boost::optional<int>> const&, TimeValue_T<double> const&, ExpandMode) MoveEventMeta.cpp   11  0x7fffdd215ac3  
17  std::make_unique<MoveEventMeta, Path<ScenarioModel>, id_base_t<EventModel, boost::optional<int>>&, TimeValue_T<double>&, ExpandMode const&>(Path<ScenarioModel>&&, id_base_t<EventModel, boost::optional<int>>&, TimeValue_T<double>&, ExpandMode const&)   unique_ptr.h    765 0x7fffdd2b51fe  
18  SingleOngoingCommandDispatcher<MoveEventMeta>::submitCommand<Path<ScenarioModel>, id_base_t<EventModel, boost::optional<int>>&, TimeValue_T<double>&, ExpandMode const&>(Path<ScenarioModel>&&, id_base_t<EventModel, boost::optional<int>>&, TimeValue_T<double>&, ExpandMode const&)  SingleOngoingCommandDispatcher.hpp  30  0x7fffdd2b49a7  
19  MoveEventState::MoveEventState(ScenarioStateMachine const&, Path<ScenarioModel> const&, iscore::CommandStack&, iscore::ObjectLocker&, QState *)::{lambda()#1}::operator()() const   MoveStates.cpp  124 0x7fffdd2b1fef  
... <plus>

I am confused, do you think is is comming from my side?

regards,
Simon.

ps: sorry if bad english

[sccolbert]

Well, the crash is expected since the exception is unhandled. However, that exception should never actually be thrown. They are only thrown when the code hits a condition which should never occur. So, this is almost certainly a bug in Kiwi.

Can you whittle down your code into a short reproducible example? That will be much easier for me to debug.

[simonjamain]

okay i'll try to do that, but it may be hard, I use kiwi in a quite complex environnement. I'll keep you in touch, thanks for your answer.

[jcelerier]

Hello,

with regards to this bug, even if the exception is handled from user code, it will not work if the file that includes kiwi.h is compiled in C++11 mode.

This is because an exception is thrown from DualOptimizeGuard::~DualOptimizeGuard, and, since C++11, destructors are marked noexcept(true) by default, which will cause std::terminate to be called as soon as an exception is thrown :
http://en.cppreference.com/w/cpp/language/destructor

Except that if no exception specification is explicitly provided, the exception specification is considered to be one that would be used by the implicitly-declared destructor (see below). In most cases, this is noexcept(true). Thus a throwing destructor must be explicitly declared noexcept(false).

[sccolbert]

Closing this, since there's not much I can do without a reproducible example. Feel free to re-open it if you come up with one.

[jklymak]

I was getting this error too. It was also for a complicated system - one that worked for modest numbers of constraints, but then would crash when more contraints were added.

My uneducated guess as to what is happening is that an "UnresolvableConstraint" error should be thrown, but the system is too complicated for that logic to catch, so the solver crashes. I (think) I can provide code that does this if it helps, but its not a trivial example. (I did attempt to craft a MNWE, but kiwi kept catching the error before updateVariables was called).

However, I found the "fix" was to be more careful about setting constraint weights. I changed most of my constraints to 'strong' instead of the default 'required' (but left a few important ones as 'required', and so far I'm not getting crashes and I'm getting the right results for the solver.

[sccolbert]

Reopening since we have another user affected by this.

@jklymak Thanks for chiming in. I'll definitely take a look at a reproducible example, even if it's complicated.

[jklymak]

@sccolbert

OK, great. You can have a look at https://github.com/jklymak/kiwierror

I narrowed down the error to one line, so that helps, I hope. See the Readme.md, but as I said, the problem is fixed if I set the strength to 'strong', whereas I hadn't really read your instructions and was just using the default constraint, which is 'required'.

Note you need matplotlib. I tried a MNWE w/o and couldn't trigger the error. Not blaming matplotlib, but obviously I failed to replicate the complications.

For some context, I'm trying to use kiwi to constrain layout of matplotlib axes. See (an obsolete) https://github.com/jklymak/mplconstrainedlayout

Thanks a lot!

[MatthieuDartiailh]

Looking at the dump of the solver before adding the last constraint, I noticed that there is one infeasible constraint that does not exist when adding earlier constraints so it may be related. I will keep digging.
Playing around more, I actually do not see an infeasible constraints before adding the last one (I do not know what was going on). However I found some other interesting thing:

• the bug does not always show up at the same stage, I see crashes at symbol 387, 687, 1383.
• in some cases there is no error !
• each time I see a failure the entering row contains only dummy symbols
  I guess my next step will to actually read the cassowary paper ...

[MatthieuDartiailh]

Based on my (limited) understanding of the algorithm, it seems to me that if an all dummy row enters the infeasible rows it should be discarded in the dualOptimize step since this constraint is redundant and entered the infeasible rows list only because the near zero constant happens to be negative.

What seems to me like a proper fix would be to add an allDummies check in https://github.com/nucleic/kiwi/blob/master/kiwi/solverimpl.h#L608. This means that we will look for an entering symbol only if the infeasible row is non dummy and furthermore this means that each time we run a dualOptimize we will clean the infeasible rows. I believe it is best to run the check this late since the row may have been added to the infeasible rows but may have become a dummy row after some pivot operations.

@sccolbert I would really appreciate your feedback here.

[MatthieuDartiailh]

Actually we should also check the constant for near zero. And if we only got dummies but non-zero constant I do not know what we should do !

[MatthieuDartiailh]

@jklymak I believe I fixed the issue as part of #56. Could you give this branch a try and see if it behaves better in matplolib ?

[MatthieuDartiailh]

This should be fixed in 1.1.0

[1313e]

This error can still be thrown with v1.2.0 when using MPL constrained layouts it seems.
I just encountered this error for the first time in 3 years making the same figure 1000s of times, so it seems that it is incredibly rare.
I therefore do not have a minimal reproducible example.

[jklymak]

Ew, sorry about that.

FWIW, Matplotlib is probably going to rewrite constrained_layout, and hopefully this won't be possible any longer.

[MatthieuDartiailh]

@jklymak do you know more about matplotlib plans. I have wanted to have a look at the implementation for a long time bit never got around to actually doing it.

[jklymak]

Sure, its this PR: matplotlib/matplotlib#17494

The main change is that there are far fewer constraints (they are by columns and rows of gridspec rather than for each axes in the gridspec) and that what is constrained is the margins around axes, so that the axes always fill the figure. Before we had the problem that zero was an acceptable solution of the constraints (i.e. a=b=2c has a trivial solution) and we had no outer constraint to fix that. It worked, but it was fragile.

[MatthieuDartiailh]

Out of curiosity are you planning to have any in-figure constraints (for example to place the colorbar next to a plot) separate from the rest (basically having a bit more hierarchy than the old code) ? I will try to have a look at the PR.

[jklymak]

@MatthieuDartiailh Not quite sure what you mean. If it weren't for colorbars, constrained_layout would be pretty easy 😉 - so most of the work was getting colorbars to work well. Is there a case that doesn't work? Its still feature matched to what's described in https://matplotlib.org/tutorials/intermediate/constrainedlayout_guide.html though colorbar placement has changed in the new code.

[MatthieuDartiailh]

Sorry about the noise, I am so used to attaching a colorbar to one graph and thinking of that as one unit in the grid that I forgot about other usecases. I really to check more the documents your wrote.