Set AugLagrangian tolerance based on ElemType

[rcurtin]

Finally, after far too long, I took some time to look into #142. I was able to replicate the problem with the Johnson844LovaszThetaFMatSDP test.

The augmented Lagrangian optimizer (AugLagrangian) keeps a penalty parameter sigma internally. This is stored as a double---but Johnson844LovaszThetaFMatSDP is a test that specifically uses arma::fmat as an objective matrix type (so, e.g., float is the element type).

AugLagrangian was also hard-coded to terminate on an absolute objective difference of 1e-10---but for this particular test, the objective function converges to -14... and due to the precision afforded by 1e-14, it's not actually possible to have two different floats very near -14 that are less than 1e-10 apart! So, I modified the tolerance to be based on the value of numeric_limits<ElemType>::epsilon().

Next, I also set a limit on how large sigma can grow. The optimization will terminate when sigma is greater than half the size that can be represented by ElemType. This issue (when paired with the other one) is what caused these errors:

Parameter 4 to routine SLASCL was incorrect
Parameter 5 to routine SLASCL was incorrect

Those were caused by a too-large sigma.

Anyway, to the best of my belief this fixes the issue. The only thing I am not sure of is why this failure only seemed to occur on an i386 system, instead of also on an x86_64 system. But, I don't know that I'll dig deeper this time---if it works, it works. :)

[zoq]

Wow, nice catch.

[mlpack-bot]

Second approval provided automatically after 24 hours. 👍

[rcurtin]

Thanks @conradsnicta, those are some nice resources. I think maybe that is what was going on here. 👍