Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TST/ENH StandardizeTransform, reparameterize TestProbitCG #1699

Merged
merged 3 commits into from May 27, 2014

Conversation

Projects
None yet
3 participants
@josef-pkt
Copy link
Member

commented May 27, 2014

see #1690 and #1648 for convergence problems in TestProbitCG

This PR add a stateful StandardizeTransform class.
and adjusts the test to use the standardized model to get start_params for the original model.

see #1131 for expanding this to other cases that are difficult to optimize.

@coveralls

This comment has been minimized.

Copy link

commented May 27, 2014

Coverage Status

Coverage increased (+0.03%) when pulling bd1abf4 on josef-pkt:reparameterize into 0b689a2 on statsmodels:master.

@josef-pkt

This comment has been minimized.

Copy link
Member Author

commented May 27, 2014

strange python 2.6 on TravisCI still needs 59 fcalls for convergence when we are supposed to be at optimum already (with different parameterization, different scaling of gradient/score)

$ export PYTHON=2.6
$ export NUMPY="1.6.2=py26_4"
$ export SCIPY="0.11.0=np16py26_3"
@coveralls

This comment has been minimized.

Copy link

commented May 27, 2014

Coverage Status

Coverage increased (+0.03%) when pulling ed457a7 on josef-pkt:reparameterize into 0b689a2 on statsmodels:master.

josef-pkt added a commit that referenced this pull request May 27, 2014

Merge pull request #1699 from josef-pkt/reparameterize
TST/ENH StandardizeTransform, reparameterize TestProbitCG closes #1648 closes #1690

@josef-pkt josef-pkt merged commit ca701e7 into statsmodels:master May 27, 2014

1 check passed

continuous-integration/travis-ci The Travis CI build passed
Details
@jseabold

This comment has been minimized.

Copy link
Member

commented May 27, 2014

Can you add release notes for these types of enhancements?

@josef-pkt

This comment has been minimized.

Copy link
Member Author

commented May 27, 2014

The Transformation class still needs a bit more thought, mainly the API. I wrote it now quickly to fix the test problem.

In a similar form I wrote a reparameterization for linear constraints based on the Stata manual, which is essentially the same as Kerby's in GEE, AFAICS.
For Ridge I worked with an orthogonalizing transform/reparameterization in a similar pattern.

Each version has different method names and a slightly different structure.

@jseabold

This comment has been minimized.

Copy link
Member

commented May 27, 2014

I haven't looked at the actual use only understand the intention, but should it follow the StatefulTransform protocol?

http://patsy.readthedocs.org/en/latest/stateful-transforms.html#defining-a-stateful-transform

@josef-pkt

This comment has been minimized.

Copy link
Member Author

commented May 27, 2014

It's a bit similar, in this case to the Standardize transform https://github.com/pydata/patsy/blob/master/patsy/state.py#L123
(patsy is working in chunks with online updating which I didn't remember)

However, it's also pretty different:

  • The transformation is on the entire parameter space, entire exog (all columns)
  • The transformation is not only for the exog but also for the parameters

The parameter transformation part is similar to the stationary AR transform in that it needs to take all parameters into account, and it's similar to the other parameter transformation that we use in fit (and turn on and off during estimation).
However, it combines this with a transformation of the exog array similar to patsy's stateful transform.

The usecase of Kerby, and in Stata, is to transform the model (both data and parameters) to impose linear constraints of the form R beta = q in the estimation. However, users are not really interested in some arbitrary transform, they want the parameters in the original space.

A Principal Component Regression that returns the original parameterization would be similar:
Run PCA, and run regression of y on the factors, but then transform the parameters in factor space back to the original exog and parameters.
The last part needs to keep track of the original linear transformation in the PCA to transform the parameters correspondingly.

The usecase in this issue is similar to our parameter transformation during fit to improve optimization. Only in this case I do it from outside of the model because I need a new model with transformed exog.

I don't know, but I'm a bit doubtful whether we can integrate this cleanly into the fit of non-linear models like discrete models, without creating a new auxiliary model based on the transformed exog.
I didn't check the details carefully how @kshedden is handling this in GEE. In contrast to imposing linear constraints, in the usecase here it would be purely internal to improve numerical issues..

@josef-pkt josef-pkt deleted the josef-pkt:reparameterize branch Jul 10, 2014

@josef-pkt josef-pkt added the PR label Aug 11, 2014

PierreBdR pushed a commit to PierreBdR/statsmodels that referenced this pull request Sep 2, 2014

Merge pull request statsmodels#1699 from josef-pkt/reparameterize
TST/ENH StandardizeTransform, reparameterize TestProbitCG closes statsmodels#1648 closes statsmodels#1690

@josef-pkt josef-pkt added this to the 0.6 milestone Apr 30, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.