Bug gee cov type 1906 rebased #1924

Merged
merged 13 commits into from Aug 23, 2014

Projects

None yet

3 participants

@josef-pkt
Member

rebased version of #1916, makes #1916 obsolete except for comments

one commit was heavily edited during merge conflict with attribute handling in subclasses.
In contrast to Kerby's previous version, I add/copy now all attributes in the subclasses through an argument to the results.__init__. Kerby assigned the attributes after creating the instance, which is too late for the attributes that are required to be available in __init__.

TODO: now subclasses can also be added to the unit test to make sure cov_params_default is correctly set. Done for the cov_params_default test, not added to generic tests in base

I renamed cov_type and the cov_xxx in a not backwards compatible way.
(I needed to fix my noteboook for this two renamings.)

I renamed resid_xxx and sensitivity_params to use postfix qualifiers, but I kept the old names as alias. I didn't adjust the code to use the new names.

Summary of PR

  • fixes cov_type handling #1906 (original PR for this #1916 )
  • fix NominalGEE to work with pandas DataFrames and Series #1931
  • fix predict #1919
  • add results wrapper: fit now return wrapped instance, with pandas for the main results #1904
  • add df_resid so that t_test works #1918
  • rename covariance_type -> cov_type
  • rename resid_xxx and sensitivity_params, with aliased old names
  • more unit tests, all changes are covered

open

  • remove alias names
  • make cov_type calculations lazy #1910
  • review arguments transfer from fit to results, after or if cov_types are calculated through results.
  • verified or regression unit tests for cov_robust_bc`
josef-pkt added some commits Aug 20, 2014
@josef-pkt josef-pkt BUG: GEE partial cleanup of cov_type, see #1906 06c5245
@josef-pkt josef-pkt BUG/TST: GEE fix predict closes #1919 and fix conf_int for cov_params…
…_default

add GEE to generic test, and adjust those (missing _results, summary2)
9fe87d8
@josef-pkt josef-pkt REF: GEE summary remove covariance_type argument (not possible)
3822520
@coveralls

Coverage Status

Coverage increased (+0.13%) when pulling 3822520 on josef-pkt:bug_gee_cov_type_1906_rebased into e373b72 on statsmodels:master.

josef-pkt added some commits Aug 21, 2014
@josef-pkt josef-pkt BUG: GEE subclasses cov_params_default doesn't get attached, unit tests 70ee68b
@josef-pkt josef-pkt TST: test_glm: remove test noise
3b5c3ac
@josef-pkt josef-pkt REF: rename covariance_type -> cov_type 449b2fc
@coveralls

Coverage Status

Coverage increased (+0.13%) when pulling 3b5c3ac on josef-pkt:bug_gee_cov_type_1906_rebased into e373b72 on statsmodels:master.

@josef-pkt josef-pkt REF: GEE rename naive_covariance -> cov_naive, same for others
bffd3a6
@coveralls

Coverage Status

Coverage increased (+0.13%) when pulling bffd3a6 on josef-pkt:bug_gee_cov_type_1906_rebased into e373b72 on statsmodels:master.

josef-pkt added some commits Aug 22, 2014
@josef-pkt josef-pkt BUG/ENH: add df_resid, df_model, fixes t_test, closes #1918 abcf156
@josef-pkt josef-pkt BUG: NominalGEE fix initialization for pandas, closes #1931 3062a80
@josef-pkt josef-pkt ENH: GEE add Wrapper, closes #1904
274c9fe
@josef-pkt
Member

this should be about finished now

latest changes:

  • add df_resid
  • fix pandas in NominalGEE
  • add wrappers

still not consistent: residuals and maybe others in results should use postfix qualifier, e.g. resid_centered, ...

josef-pkt added some commits Aug 22, 2014
@josef-pkt josef-pkt BUG: NominalGEE pandas handling, fix typo, adjust unit tests 8a3cd39
@josef-pkt josef-pkt REF: GEEResults rename to postfix names, keep old as alias for now
a59a0b7
@josef-pkt
Member

here's my partially updated notebook, running with this branch
http://nbviewer.ipython.org/gist/josef-pkt/4d2862963ba13df5e4ab

@coveralls

Coverage Status

Coverage increased (+0.16%) when pulling a59a0b7 on josef-pkt:bug_gee_cov_type_1906_rebased into e373b72 on statsmodels:master.

@josef-pkt
Member

oh fun, pandas compatibility, but in py 3.3 version only

uses pandas 0.13, I used pandas 0.12 for testing
guess: this is a consequence that the results attributes are now pandas data structures because of the wrapper. use results._results for verifying numbers
todo: check pythonxy tomorrow for testing with pandas master.

ERROR: statsmodels.genmod.tests.test_gee.TestGEE.test_compare_OLS
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/travis/miniconda/envs/statsmodels-test/lib/python3.3/site-packages/nose/case.py", line 198, in runTest
self.test(*self.arg)
File "/home/travis/miniconda/envs/statsmodels-test/lib/python3.3/site-packages/statsmodels-0.6.0-py3.3-linux-x86_64.egg/statsmodels/genmod/tests/test_gee.py", line 719, in test_compare_OLS
assert_almost_equal(naive_tvalues, ols.tvalues, decimal=10)
File "/home/travis/miniconda/envs/statsmodels-test/lib/python3.3/site-packages/numpy/testing/utils.py", line 459, in assert_almost_equal
if not (gisfinite(desired) and gisfinite(actual)):
File "/home/travis/miniconda/envs/statsmodels-test/lib/python3.3/site-packages/pandas/core/generic.py", line 676, in __nonzero__
.format(self.__class__.__name__))
ValueError: The truth value of a Series is ambiguous. Use a.empty, a.bool(), a.item(), a.any() or a.all().
@josef-pkt josef-pkt TST: avoid pandas in assert with wrapped results (failure pandas>0.12)
9d7442e
@coveralls

Coverage Status

Coverage increased (+0.16%) when pulling 9d7442e on josef-pkt:bug_gee_cov_type_1906_rebased into e373b72 on statsmodels:master.

@josef-pkt
Member

replaced by checking either values or the unwrapped results._results, so asserts are numpy only.

@josef-pkt
Member

@kshedden I would like to merge this very soon. (We are already getting late for the release, for the Debian deadlines.)

Do you want to review this and check whether the changes are fine with you?

@kshedden
Contributor

Yes, I've reviewed it. Looks good. Thank you for dealing with this.

On Sat, Aug 23, 2014 at 11:06 AM, Josef Perktold notifications@github.com
wrote:

@kshedden https://github.com/kshedden I would like to merge this very
soon. (We are already getting late for the release, for the Debian
deadlines.)

Do you want to review this and check whether the changes are fine with you?


Reply to this email directly or view it on GitHub
#1924 (comment)
.

@josef-pkt
Member

Thanks, merging

@josef-pkt josef-pkt merged commit 957a43e into statsmodels:master Aug 23, 2014

2 checks passed

continuous-integration/appveyor AppVeyor build succeeded
Details
continuous-integration/travis-ci The Travis CI build passed
Details
@josef-pkt josef-pkt deleted the josef-pkt:bug_gee_cov_type_1906_rebased branch Aug 23, 2014
@josef-pkt josef-pkt added this to the 0.6 milestone Aug 24, 2014
@josef-pkt josef-pkt removed the PR label Aug 24, 2014
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment