LogNormalFitter has strange properties

[CamDavidsonPilon]

I've spent far too much time trying to debug the LNF model. Trouble is, it's unstable for some inputs. Example:

1. As the following script shows, the model is only stable for ~0.06 < sigma < ~3.5. Anything outside this range will cause the dreaded Desired error not necessarily achieved due to precision loss in minimize.

from lifelines import LogNormalFitter
from lifelines.utils import ConvergenceError

MU = np.linspace(-10, 10, 5)
SIGMA_ = np.linspace(0.0001, 6, 25)
R = np.zeros((5, 25))

for i, mu_ in enumerate(MU):
    for j, sigma_ in enumerate(SIGMA_):
        try:
            N = 20000
            print(mu_)
            print(sigma_)

            X, C = np.exp(sigma_ * np.random.randn(N) + mu_), np.exp(np.random.randn(N) + mu_)
            E = X <= C
            T = np.minimum(X, C)

            LogNormalFitter().fit(T, E)
            R[i, j] = 1
        except ConvergenceError:
            R[i, j] = 0

plt.matshow(R)
plt.xticks(np.arange(25), SIGMA_)
plt.yticks(np.arange(5), MU)

2. AFAIK, the log-likelihood and the gradients are computed correctly, though it would be useful to have a second set of eyes on them. Even scipy's check_gradients seems to confirm:

print(check_grad(_negative_log_likelihood, gradient_function, [0, 0], log(T), E))

3. When there is no censorship, the model converges for all values...

4. Adding a penalizer doesn't seem to help.

5. When I power transform the durations by the inverse standard deviation of the log(T), this seems to help convergence - however I can't get back the original parameters.

6. Not specifying the gradient function, jac seems to help! That is:

minimize(_negative_log_likelihood, init, args=(log(T), E), method='BFGS')

converges, but

minimize(_negative_log_likelihood, init, args=(log(T), E), method='BFGS', jac=gradient_function)

seems to fail

[CamDavidsonPilon]

I removed the scaling in the exp domain, and this seemed to really help. Only very small values of sigma will cause problems.