Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

interpolate_df() changes datetime unit of date column #212

Open
capelastegui opened this issue May 16, 2024 · 0 comments
Open

interpolate_df() changes datetime unit of date column #212

capelastegui opened this issue May 16, 2024 · 0 comments

Comments

@capelastegui
Copy link
Contributor

We have seen forecasts crashing with the following error:

  File "/home/jupyter_user/workspace/nsa.notebooks-voice/virtual/lib/python3.9/site-packages/anticipy/forecast.py", line 928, in run_forecast
    return run_forecast_single(df_y,
  File "/home/jupyter_user/workspace/nsa.notebooks-voice/virtual/lib/python3.9/site-packages/anticipy/forecast.py", line 1386, in run_forecast_single
    df_data = pd.concat(l_df_data, sort=False, ignore_index=True)
  File "/home/jupyter_user/workspace/nsa.notebooks-voice/virtual/lib/python3.9/site-packages/pandas/core/reshape/concat.py", line 393, in concat
    return op.get_result()
  File "/home/jupyter_user/workspace/nsa.notebooks-voice/virtual/lib/python3.9/site-packages/pandas/core/reshape/concat.py", line 680, in get_result
    new_data = concatenate_managers(
  File "/home/jupyter_user/workspace/nsa.notebooks-voice/virtual/lib/python3.9/site-packages/pandas/core/internals/concat.py", line 189, in concatenate_managers
    values = _concatenate_join_units(join_units, copy=copy)
  File "/home/jupyter_user/workspace/nsa.notebooks-voice/virtual/lib/python3.9/site-packages/pandas/core/internals/concat.py", line 486, in _concatenate_join_units
    concat_values = concat_compat(to_concat, axis=1)
  File "/home/jupyter_user/workspace/nsa.notebooks-voice/virtual/lib/python3.9/site-packages/pandas/core/dtypes/concat.py", line 132, in concat_compat
    return cls._concat_same_type(to_concat_eas)
  File "/home/jupyter_user/workspace/nsa.notebooks-voice/virtual/lib/python3.9/site-packages/pandas/core/arrays/datetimelike.py", line 2251, in _concat_same_type
    new_obj = super()._concat_same_type(to_concat, axis)
  File "/home/jupyter_user/workspace/nsa.notebooks-voice/virtual/lib/python3.9/site-packages/pandas/core/arrays/_mixins.py", line 230, in _concat_same_type
    return super()._concat_same_type(to_concat, axis=axis)
  File "arrays.pyx", line 190, in pandas._libs.arrays.NDArrayBacked._concat_same_type
ValueError: all the input array dimensions except for the concatenation axis must match exactly, but along dimension 1, the array at index 0 has size 141 and the array at index 1 has size 616```

We have traced this back to a bug in interpolate_df() that changes the datetime unit of the date column. In this case, the input data had a date column with dtype datetime64[us], and interpolate_df() returned a date with dtype datetime64[ns]. The crash happened when concatenating both series together.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant