Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TypeError when using statsmodels.OLS with NaNs and missing = 'drop' #4417

Open
dferens opened this issue Apr 2, 2018 · 0 comments
Open

TypeError when using statsmodels.OLS with NaNs and missing = 'drop' #4417

dferens opened this issue Apr 2, 2018 · 0 comments

Comments

@dferens
Copy link

dferens commented Apr 2, 2018

How to reproduce:

from datetime import date
import numpy as np
import pandas as pd
import statsmodels.api as sm

X = pd.Series([np.NaN], pd.DatetimeIndex([date(2017,1,1)], freq='AS-JAN'))
Y = pd.Series([10], pd.DatetimeIndex([date(2017,1,1)], freq='AS-JAN'))
sm.OLS(X, sm.add_constant(Y), missing='drop')

Traceback:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/virtualenvs/test/lib/python2.7/site-packages/statsmodels/regression/linear_model.py", line 631, in __init__
    hasconst=hasconst, **kwargs)
  File "/virtualenvs/test/lib/python2.7/site-packages/statsmodels/regression/linear_model.py", line 526, in __init__
    weights=weights, hasconst=hasconst, **kwargs)
  File "/virtualenvs/test/lib/python2.7/site-packages/statsmodels/regression/linear_model.py", line 95, in __init__
    super(RegressionModel, self).__init__(endog, exog, **kwargs)
  File "/virtualenvs/test/lib/python2.7/site-packages/statsmodels/base/model.py", line 212, in __init__
    super(LikelihoodModel, self).__init__(endog, exog, **kwargs)
  File "/virtualenvs/test/lib/python2.7/site-packages/statsmodels/base/model.py", line 63, in __init__
    **kwargs)
  File "/virtualenvs/test/lib/python2.7/site-packages/statsmodels/base/model.py", line 88, in _handle_data
    data = handle_data(endog, exog, missing, hasconst, **kwargs)
  File "/virtualenvs/test/lib/python2.7/site-packages/statsmodels/base/data.py", line 630, in handle_data
    **kwargs)
  File "/virtualenvs/test/lib/python2.7/site-packages/statsmodels/base/data.py", line 65, in __init__
    **kwargs)
  File "/virtualenvs/test/lib/python2.7/site-packages/statsmodels/base/data.py", line 284, in handle_missing
    combined = dict(zip(combined_names, lmap(drop_nans, combined)))
  File "/virtualenvs/test/lib/python2.7/site-packages/statsmodels/base/data.py", line 282, in <lambda>
    drop_nans = lambda x: cls._drop_nans(x, nan_mask)
  File "/virtualenvs/test/lib/python2.7/site-packages/statsmodels/base/data.py", line 478, in _drop_nans
    return x.ix[nan_mask]
  File "/virtualenvs/test/lib/python2.7/site-packages/pandas/core/indexing.py", line 121, in __getitem__
    return self._getitem_axis(key, axis=0)
  File "/virtualenvs/test/lib/python2.7/site-packages/pandas/core/indexing.py", line 1077, in _getitem_axis
    return self._get_label(key, axis=axis)
  File "/virtualenvs/test/lib/python2.7/site-packages/pandas/core/indexing.py", line 136, in _get_label
    return self.obj._xs(label, axis=axis)
  File "/virtualenvs/test/lib/python2.7/site-packages/pandas/core/generic.py", line 2032, in xs
    loc = self.index.get_loc(key)
  File "/virtualenvs/test/lib/python2.7/site-packages/pandas/core/indexes/datetimes.py", line 1435, in get_loc
    stamp = Timestamp(key, tz=self.tz)
  File "pandas/_libs/tslib.pyx", line 402, in pandas._libs.tslib.Timestamp.__new__ (pandas/_libs/tslib.c:10051)
  File "pandas/_libs/tslib.pyx", line 1528, in pandas._libs.tslib.convert_to_tsobject (pandas/_libs/tslib.c:28851)
TypeError: Cannot convert input [False] of type <type 'numpy.bool_'> to Timestamp

In first call to pandas:

  File "/virtualenvs/test/lib/python2.7/site-packages/statsmodels/base/data.py", line 478, in _drop_nans
    return x.ix[nan_mask]

value of nan_mask is False.


version:

>>> sm.version.full_version
'0.8.0'

Tested on pandas versions 0.20.3 and 0.22.0.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant