BUG: .rolling() returns incorrect values when ts index is not nano seconds #55173

hkhojasteh · 2023-09-16T16:33:12Z

closes BUG: .rolling() returns incorrect values when ts index is not nano seconds #55106
closes BUG: Rolling pd.date_range incorrect for unit='s' #55026
[Tests added and passed]

The bug is referred to and fixed inside the code. It needs to be generalised. @jorisvandenbossche we need to have the first feedback if we can generalise the data type with some utils or more generally. Done during the in-person sprint check.

doc/source/conf.py

pandas/core/window/rolling.py

pandas/tests/window/test_rolling.py

Co-authored-by: Joris Van den Bossche <jorisvandenbossche@gmail.com>

the df variable name fixed in the test

hkhojasteh · 2023-09-19T20:00:00Z

The test was parametrised, the rolling unit generalised and the version commit reverted :)
@jorisvandenbossche please let me know in case of any other changes

lithomas1 · 2023-09-20T14:19:15Z

Bumping off of 2.1.1.

hkhojasteh · 2023-09-22T12:43:02Z

@lithomas1 This fix is also related to other subfunctions like max. More at #55026. The PR with the related tests is available.

lithomas1 · 2023-09-22T22:36:39Z

Can you add a whatsnew note for 2.1.2?

hkhojasteh · 2023-09-30T11:14:56Z

Can you add a whatsnew note for 2.1.2?

Gladly added :)

doc/source/whatsnew/v2.1.2.rst

Co-authored-by: Matthew Roeschke <10647082+mroeschke@users.noreply.github.com>

pandas/core/window/rolling.py

pandas/tests/window/test_rolling.py

Data conversion removed Tests merged

hkhojasteh · 2023-10-03T10:33:57Z

@jorisvandenbossche I have modified the validation function based on your suggestion. Tests are merged and the previous conversion has been removed. Please check it out and let me know

pandas/core/window/rolling.py

…anch

hkhojasteh · 2023-10-24T15:04:47Z

@jorisvandenbossche I was about to ask how is the merge on this fix going on? Anything that I can help with?

lithomas1 · 2023-10-25T01:12:09Z

Tests are failing

FAILED pandas/tests/window/test_rolling.py::tests_empty_df_rolling[1s] - TypeError: cannot get datetime metadata from non-datetime type
FAILED pandas/tests/window/test_timeseries_window.py::TestRollingTS::test_invalid_window_nonfixed[2MS] - AssertionError: Regex pattern did not match.
 Regex: '\\<2 \\* MonthBegins\\> is a non-fixed frequency'
 Input: 'Value must be Timedelta, string, integer, float, timedelta or convertible, not MonthBegin'
FAILED pandas/tests/window/test_timeseries_window.py::TestRollingTS::test_invalid_window_nonfixed[freq1] - AssertionError: Regex pattern did not match.
 Regex: '\\<2 \\* MonthBegins\\> is a non-fixed frequency'
 Input: 'Value must be Timedelta, string, integer, float, timedelta or convertible, not MonthBegin'
FAILED pandas/tests/window/test_timeseries_window.py::TestRollingTS::test_rolling_on_empty - TypeError: cannot get datetime metadata from non-datetime type

…s when ts index is not nano seconds

mroeschke · 2023-10-26T02:15:15Z

Thanks @hkhojasteh and @jorisvandenbossche

… values when ts index is not nano seconds) (#55697) Backport PR #55173: BUG: .rolling() returns incorrect values when ts index is not nano seconds Co-authored-by: Hadi Abdi Khojasteh <hadiabdikhojasteh@gmail.com>

hkhojasteh added 2 commits September 16, 2023 15:52

Fix rolling microseconds for sum

3e107f3

- adding a test for rounding sum

15abeb5

jorisvandenbossche added the Sprints Sprint Pull Requests label Sep 16, 2023

jorisvandenbossche added this to the 2.1.1 milestone Sep 17, 2023

jorisvandenbossche reviewed Sep 17, 2023

View reviewed changes

doc/source/conf.py Outdated Show resolved Hide resolved

pandas/core/window/rolling.py Show resolved Hide resolved

pandas/tests/window/test_rolling.py Show resolved Hide resolved

pandas/tests/window/test_rolling.py Outdated Show resolved Hide resolved

jorisvandenbossche changed the title ~~BUG: .rolling() returns incorrect values when ts index is not nano seconds #55106~~ BUG: .rolling() returns incorrect values when ts index is not nano seconds Sep 17, 2023

jorisvandenbossche added Bug Non-Nano datetime64/timedelta64 with non-nanosecond resolution labels Sep 17, 2023

hkhojasteh and others added 4 commits September 19, 2023 20:43

Update pandas/tests/window/test_rolling.py

bc28f3f

Co-authored-by: Joris Van den Bossche <jorisvandenbossche@gmail.com>

Update test_rolling.py

dede3b6

the df variable name fixed in the test

Reverted version varialbe in doc/source/conf.py

ba9cf75

Units generalised to us/ms/s and test parameterised

0a0b938

lithomas1 modified the milestones: 2.1.1, 2.1.2 Sep 20, 2023

jorisvandenbossche mentioned this pull request Sep 20, 2023

RLS 2.1.1 #54901

Closed

hkhojasteh mentioned this pull request Sep 22, 2023

BUG: Rolling pd.date_range incorrect for unit='s' #55026 #55243

Closed

2 tasks

Rolling max tests added, related to #55026

4fd40eb

mroeschke added the Window rolling, ewma, expanding label Sep 29, 2023

mroeschke mentioned this pull request Sep 29, 2023

Bug fix where df.rolling doesn't work with certain datetime64 index t… #55325

Closed

4 tasks

hkhojasteh added 2 commits September 30, 2023 13:06

Merge branch 'pandas-dev:main' into branch

5dd1a68

whatsnew note for 2.1.2 added

43b2ccb

hkhojasteh requested a review from jorisvandenbossche October 2, 2023 11:52

mroeschke reviewed Oct 2, 2023

View reviewed changes

doc/source/whatsnew/v2.1.2.rst Outdated Show resolved Hide resolved

Update doc/source/whatsnew/v2.1.2.rst

2b6458e

Co-authored-by: Matthew Roeschke <10647082+mroeschke@users.noreply.github.com>

hkhojasteh requested a review from mroeschke October 2, 2023 17:48

UTC timezone for _index_array of rolling

710a7e1

hkhojasteh requested a review from mroeschke October 3, 2023 00:26

jorisvandenbossche reviewed Oct 3, 2023

View reviewed changes

pandas/core/window/rolling.py Outdated Show resolved Hide resolved

pandas/core/window/rolling.py Outdated Show resolved Hide resolved

jorisvandenbossche reviewed Oct 3, 2023

View reviewed changes

pandas/tests/window/test_rolling.py Outdated Show resolved Hide resolved

jorisvandenbossche reviewed Oct 3, 2023

View reviewed changes

pandas/tests/window/test_rolling.py Outdated Show resolved Hide resolved

Validating tz-aware data

a6a4d30

Data conversion removed Tests merged

hkhojasteh requested a review from jorisvandenbossche October 3, 2023 10:34

Merge branch 'main' into branch

d308082

jbrockmendel reviewed Oct 9, 2023

View reviewed changes

pandas/core/window/rolling.py Outdated Show resolved Hide resolved

hkhojasteh added 3 commits October 14, 2023 05:01

Conversion replaced by Timedelta.as_unit

f62e484

Merge branch 'branch' of https://github.com/hkhojasteh/pandas into br…

c23ad0e

…anch

Merge branch 'main' into branch

5994a23

jorisvandenbossche added 4 commits October 25, 2023 21:44

Merge remote-tracking branch 'upstream/main' into branch

772c114

fixes for failing tests

da3d7ea

update whatsnew

e3d93b0

type checking

11ceb3a

lithomas1 modified the milestones: 2.1.2, 2.1.3 Oct 25, 2023

mroeschke mentioned this pull request Oct 25, 2023

RLS: 2.1.2 #55229

Closed

mroeschke approved these changes Oct 26, 2023

View reviewed changes

mroeschke merged commit fe17818 into pandas-dev:main Oct 26, 2023

meeseeksmachine mentioned this pull request Oct 26, 2023

Backport PR #55173 on branch 2.1.x (BUG: .rolling() returns incorrect values when ts index is not nano seconds) #55697

Merged

meeseeksmachine pushed a commit to meeseeksmachine/pandas that referenced this pull request Oct 26, 2023

Backport PR pandas-dev#55173: BUG: .rolling() returns incorrect value…

3e30241

…s when ts index is not nano seconds

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

BUG: .rolling() returns incorrect values when ts index is not nano seconds #55173

BUG: .rolling() returns incorrect values when ts index is not nano seconds #55173

Uh oh!

hkhojasteh commented Sep 16, 2023 •

edited by jorisvandenbossche

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

hkhojasteh commented Sep 19, 2023

Uh oh!

lithomas1 commented Sep 20, 2023

Uh oh!

hkhojasteh commented Sep 22, 2023 •

edited

Loading

Uh oh!

lithomas1 commented Sep 22, 2023

Uh oh!

hkhojasteh commented Sep 30, 2023

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

hkhojasteh commented Oct 3, 2023 •

edited

Loading

Uh oh!

Uh oh!

hkhojasteh commented Oct 24, 2023

Uh oh!

lithomas1 commented Oct 25, 2023

Uh oh!

mroeschke commented Oct 26, 2023

Uh oh!

Uh oh!

Uh oh!

BUG: .rolling() returns incorrect values when ts index is not nano seconds #55173

BUG: .rolling() returns incorrect values when ts index is not nano seconds #55173

Uh oh!

Conversation

hkhojasteh commented Sep 16, 2023 • edited by jorisvandenbossche Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

hkhojasteh commented Sep 19, 2023

Uh oh!

lithomas1 commented Sep 20, 2023

Uh oh!

hkhojasteh commented Sep 22, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

lithomas1 commented Sep 22, 2023

Uh oh!

hkhojasteh commented Sep 30, 2023

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

hkhojasteh commented Oct 3, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

hkhojasteh commented Oct 24, 2023

Uh oh!

lithomas1 commented Oct 25, 2023

Uh oh!

mroeschke commented Oct 26, 2023

Uh oh!

Uh oh!

hkhojasteh commented Sep 16, 2023 •

edited by jorisvandenbossche

Loading

hkhojasteh commented Sep 22, 2023 •

edited

Loading

hkhojasteh commented Oct 3, 2023 •

edited

Loading