see #1690 and #1648 for convergence problems in TestProbitCG
This PR add a stateful StandardizeTransform class.
and adjusts the test to use the standardized model to get start_params for the original model.
see #1131 for expanding this to other cases that are difficult to optimize.
ENH tools, add standardize transform
TST: fix TestProbitCG by reparameterization, closes #1690
Coverage increased (+0.03%) when pulling bd1abf4 on josef-pkt:reparameterize into 0b689a2 on statsmodels:master.
TST: TestProbitCG increase max function calls
strange python 2.6 on TravisCI still needs 59 fcalls for convergence when we are supposed to be at optimum already (with different parameterization, different scaling of gradient/score)
$ export PYTHON=2.6
$ export NUMPY="1.6.2=py26_4"
$ export SCIPY="0.11.0=np16py26_3"
Coverage increased (+0.03%) when pulling ed457a7 on josef-pkt:reparameterize into 0b689a2 on statsmodels:master.
Can you add release notes for these types of enhancements?
The Transformation class still needs a bit more thought, mainly the API. I wrote it now quickly to fix the test problem.
In a similar form I wrote a reparameterization for linear constraints based on the Stata manual, which is essentially the same as Kerby's in GEE, AFAICS.
For Ridge I worked with an orthogonalizing transform/reparameterization in a similar pattern.
Each version has different method names and a slightly different structure.
I haven't looked at the actual use only understand the intention, but should it follow the StatefulTransform protocol?
It's a bit similar, in this case to the Standardize transform https://github.com/pydata/patsy/blob/master/patsy/state.py#L123
(patsy is working in chunks with online updating which I didn't remember)
However, it's also pretty different:
The parameter transformation part is similar to the stationary AR transform in that it needs to take all parameters into account, and it's similar to the other parameter transformation that we use in fit (and turn on and off during estimation).
However, it combines this with a transformation of the exog array similar to patsy's stateful transform.
The usecase of Kerby, and in Stata, is to transform the model (both data and parameters) to impose linear constraints of the form R beta = q in the estimation. However, users are not really interested in some arbitrary transform, they want the parameters in the original space.
R beta = q
A Principal Component Regression that returns the original parameterization would be similar:
Run PCA, and run regression of y on the factors, but then transform the parameters in factor space back to the original exog and parameters.
The last part needs to keep track of the original linear transformation in the PCA to transform the parameters correspondingly.
The usecase in this issue is similar to our parameter transformation during fit to improve optimization. Only in this case I do it from outside of the model because I need a new model with transformed exog.
I don't know, but I'm a bit doubtful whether we can integrate this cleanly into the fit of non-linear models like discrete models, without creating a new auxiliary model based on the transformed exog.
I didn't check the details carefully how @kshedden is handling this in GEE. In contrast to imposing linear constraints, in the usecase here it would be purely internal to improve numerical issues..