REF/ENH: LikelihoodModel.fit optimization, make hessian optional #1943

Closed
josef-pkt opened this Issue Aug 31, 2014 · 6 comments

Projects

None yet

1 participant

@josef-pkt
Member

We need a fit keyword to turn of the hessian calculation if it is not needed.

see #1940 (comment)

In MixedLM hessian is currently not used. While this needs to be refactored, it's not necessary to calculate the Hessian in the super().fit

Another possible case:
super().fit might only see the transformed parameterization for the optimization and not the untransformed as it is given to the results and the user.

example: NegativeBinomial I don't see how this works, or why it works:
The Hessian in super().fit is calculated with the setting for self._transparams in effect.
The hessian from the optimization is not overwritten in the model specific fit.

So the hessian should be for the transformed parameterization?
But I thought we have the unit tests for it.
robust cov_params are not affected because I call hessian after self._transparams has been reset to false. (I needed a distribution specific pattern exactly because of this.)

@josef-pkt josef-pkt added this to the 0.6 milestone Aug 31, 2014
@josef-pkt
Member

temporary bug and prio-high label, until this is checked.

maybe my reading is wrong, maybe there's a bug

check unit tests:
parameters are transformed for estimation if:
if self.loglike_method.startswith('nb') and method not in ['newton', 'ncg']

which means default newton is not affected, We need to test with bfgs.

@josef-pkt josef-pkt added a commit to josef-pkt/statsmodels that referenced this issue Sep 2, 2014
@josef-pkt josef-pkt ENH: base.LikelihoodModel.fit add skip_hessian option see #1943 35d8ffa
@josef-pkt
Member

NegativeBinomial doesn't have a bug.
I get the same bse whether I use newton or bfgs. params is adjusted but cov_params/hessian is not adjusted.

the Hessian evaluates the derivative at the transformed value, but it is the derivative wrt. the untransformed parameter. AFAICS

But then, why did I run into problems when I added the robust covariances and cov_type?

@josef-pkt
Member

OK, to answer the last part: score is the derivative with respect to the transformed params, but not the hessian.
The problem is that super().fit evaluates the score wrt. the transformed params, but those are not relevant for the results. Except they are used if super also handles robust sandwiches while the _transparams is in effect.

NegativeBinomial is fine except for inconsistent definition of score and hessian wrt transparams.
That's why we cannot use hessian based optimization methods with transparams= True.

@josef-pkt josef-pkt removed the prio-high label Sep 18, 2014
@josef-pkt
Member

no more prio-high for this

@josef-pkt josef-pkt added a commit to josef-pkt/statsmodels that referenced this issue Sep 18, 2014
@josef-pkt josef-pkt ENH: base.LikelihoodModel.fit add skip_hessian option see #1943 36c1048
@josef-pkt
Member

I opened #1967, so I can close this.

skip_hessian has been merged in #1952

@josef-pkt
Member

as followup: there are still some duplicate calculations especially when sandwich robust covariances are calculated

@josef-pkt josef-pkt closed this Sep 18, 2014
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment