TST/ENH StandardizeTransform, reparameterize TestProbitCG #1699

Merged
merged 3 commits into from May 27, 2014

Projects

None yet

3 participants

@josef-pkt
Member

see #1690 and #1648 for convergence problems in TestProbitCG

This PR add a stateful StandardizeTransform class.
and adjusts the test to use the standardized model to get start_params for the original model.

see #1131 for expanding this to other cases that are difficult to optimize.

@coveralls

Coverage Status

Coverage increased (+0.03%) when pulling bd1abf4 on josef-pkt:reparameterize into 0b689a2 on statsmodels:master.

@josef-pkt
Member

strange python 2.6 on TravisCI still needs 59 fcalls for convergence when we are supposed to be at optimum already (with different parameterization, different scaling of gradient/score)

$ export PYTHON=2.6
$ export NUMPY="1.6.2=py26_4"
$ export SCIPY="0.11.0=np16py26_3"
@coveralls

Coverage Status

Coverage increased (+0.03%) when pulling ed457a7 on josef-pkt:reparameterize into 0b689a2 on statsmodels:master.

@josef-pkt josef-pkt merged commit ca701e7 into statsmodels:master May 27, 2014

1 check passed

continuous-integration/travis-ci The Travis CI build passed
Details
@jseabold
Member

Can you add release notes for these types of enhancements?

@josef-pkt
Member

The Transformation class still needs a bit more thought, mainly the API. I wrote it now quickly to fix the test problem.

In a similar form I wrote a reparameterization for linear constraints based on the Stata manual, which is essentially the same as Kerby's in GEE, AFAICS.
For Ridge I worked with an orthogonalizing transform/reparameterization in a similar pattern.

Each version has different method names and a slightly different structure.

@jseabold
Member

I haven't looked at the actual use only understand the intention, but should it follow the StatefulTransform protocol?

http://patsy.readthedocs.org/en/latest/stateful-transforms.html#defining-a-stateful-transform

@josef-pkt
Member

It's a bit similar, in this case to the Standardize transform https://github.com/pydata/patsy/blob/master/patsy/state.py#L123
(patsy is working in chunks with online updating which I didn't remember)

However, it's also pretty different:

  • The transformation is on the entire parameter space, entire exog (all columns)
  • The transformation is not only for the exog but also for the parameters

The parameter transformation part is similar to the stationary AR transform in that it needs to take all parameters into account, and it's similar to the other parameter transformation that we use in fit (and turn on and off during estimation).
However, it combines this with a transformation of the exog array similar to patsy's stateful transform.

The usecase of Kerby, and in Stata, is to transform the model (both data and parameters) to impose linear constraints of the form R beta = q in the estimation. However, users are not really interested in some arbitrary transform, they want the parameters in the original space.

A Principal Component Regression that returns the original parameterization would be similar:
Run PCA, and run regression of y on the factors, but then transform the parameters in factor space back to the original exog and parameters.
The last part needs to keep track of the original linear transformation in the PCA to transform the parameters correspondingly.

The usecase in this issue is similar to our parameter transformation during fit to improve optimization. Only in this case I do it from outside of the model because I need a new model with transformed exog.

I don't know, but I'm a bit doubtful whether we can integrate this cleanly into the fit of non-linear models like discrete models, without creating a new auxiliary model based on the transformed exog.
I didn't check the details carefully how @kshedden is handling this in GEE. In contrast to imposing linear constraints, in the usecase here it would be purely internal to improve numerical issues..

@josef-pkt josef-pkt deleted the josef-pkt:reparameterize branch Jul 10, 2014
@josef-pkt josef-pkt added the PR label Aug 11, 2014
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment