worn in the case of non-convergence

[Eh2406]

At yesterday's meeting with @semcogli and @janowicz discuss how deeply odd our fitted mnl models wore. Specifically, we have variables that have to be positive that are being fit as negative. In the meeting we considered the possibility of collinearity, or spurious correlations, or bad sampling processes, or other complex possibilities. After the meating I thought that the simplest explanation would be if the optimization algorithm had not successfully fitted the models in the first place.

Looking at the code I see that we did not check it optimize.fmin_l_bfgs_b had succeeded. So I added a warning that reports some diagnostic information in case of non-convergence.

When I ran the tests I noted that test_alternative_specific_coeffs reliably thru a non-convergence warning 'ABNORMAL_TERMINATION_IN_LNSRCH'. According to the internet, that diagnostic code is usually caused by programer error. I also noted the RuntimeWarning: overflow encountered in exp. After some research I found a standard technique for avoiding overflow in softmax calculations, applying it solved both warnings. The resulting output did not otherwise change.

TLDR:

• Warn if we may be returning incorrect output.
• Fix a numerical instability / Overflow.
• No evidence that it solves any of the bigger problems.

[coveralls]

Coverage increased (+0.008%) to 94.345% when pulling 9794d9e on SEMCOG:stability into 2497039 on UDST:master.

[Eh2406]

cc @smmaurer looks like this code was copied to https://github.com/UDST/choicemodels/blob/master/choicemodels/mnl.py#L652
When we get this to a good state mabey this pr should be done there as well? Is there other places it has been copied?

[Eh2406]

Ping?

[smmaurer]

Hi @Eh2406, this looks great! That is super helpful to fix the runtime overflow, and good idea to report if the convergence fails. I'm going to merge this.

You're right that the code was copied to choicemodels -- i think the urbansim version wasn't quite encapsulated enough for us to just call it. And as far as i know that's the only place. Do you want to open a PR there too? (Not trying to generate more work for you, just give you full credit for the fixes. I can also update it myself.)

[Eh2406]

Thanks! I will make a pr over there soon.

[Eh2406]

I was just ready to make a pr from: SEMCOG/choicemodels@7d7022f

But looks like you did better testing in: https://github.com/UDST/choicemodels/tree/fix-runtime-overflow

So I will let you do it!