Fix bug in FiniteDateRange.is_disjoint (and union as a result) #99

josh-lynch · 2023-10-10T16:21:52Z

Primarily this change is aimed at changing the behaviour of the
FiniteDateRange.union. Previously FiniteDateRanges would not make
a union with adjacent ranges, despite their inclusive boundaries
meaning the presence of an adjacent range means they cover a
fully contiguous period of time.

This change makes adjacent FiniteDateRanges unionize by no longer
considering ranges on adjacent days disjoint.

xocto/ranges.py

josh-lynch · 2023-10-12T11:36:00Z

I also can't figure out the typing complaints here and they are driving me insane. Insights welcome.

pydanny · 2023-10-12T13:50:02Z

I also can't figure out the typing complaints here and they are driving me insane. Insights welcome.

What about being more explicit?

    def is_disjoint(self, other: Range[datetime.date]) -> bool:
        # Adjacent dates should not be considered disjoint, we extend the other
        # range to allow them to be considered adjacent.
        other_start = other.start
        if other._is_right_inclusive:
			if other_start is not None:  # replaces assertion
	            other_start -= datetime.timedelta(days=1)
        other_end = other.end
        if other._is_right_inclusive:
			if other_end is not None:  # replaces assertion
	            other_end += datetime.timedelta(days=1)
        return super().is_disjoint(
            Range(start=other_start, end=other_end, boundaries=other.boundaries)
        )

delfick · 2023-10-13T00:26:16Z

I also can't figure out the typing complaints here and they are driving me insane. Insights welcome.

var2 = var1.blah
assert var1.blah is not None

When you do this, mypy still doesn't know if var2 is not None or not because var1.blah might be a different value to var2.

so when you do

        other_start = other.start
        if other._is_right_inclusive:
            assert other.start
            other_start -= datetime.timedelta(days=1)

what you actually want is

        other_start = other.start
        if other._is_right_inclusive:
            assert other_start is not None
            other_start -= datetime.timedelta(days=1)

(I have not tested this theory, but I imagine that's what it is)

frederizia · 2023-10-13T00:29:46Z

what you actually want is
        other_start = other.start
        if other._is_right_inclusive:
            assert other_start is not None
            other_start -= datetime.timedelta(days=1)
(I have not tested this theory, but I imagine that's what it is)

Yes was just coming here to say this (I have tested it and that sorts it). Other option is doing other_start = other.start - datetime.timedelta(days=1) but that's not as nice.

GlennS · 2023-10-16T00:22:11Z

xocto/ranges.py

+        other_start = other.start
+        if other._is_right_inclusive:
+            assert other.start
+            other_start -= datetime.timedelta(days=1)


🐮 this bug is wider than dates - it also affects any Range where the values are discrete. For example: integers.

I think it could also potentially affect other boundary conditions than inclusive-inclusive, although I guess we'd have to go through and figure it out.

I wonder if a wider solution would be some kind of increment or resolution field? Either directly on the Range itself, or coming from a mapping RANGE_TYPE_RESOLUTIONS = {datetime.Date: datetime.timedelta(days=1), int: 1, ...}.

(I don't want my comment to block this PR. I'm sure date ranges are our biggest problem in Kraken and your fix as it stands would be a very useful win.)

I also think this might be a wider problem, and some kind of resolution might be a good idea (although since ranges can be for any orderable type, it might not always work).

I agree that we shouldn't block here though.

I did consider the integer problem here, but I think it's a bit different because if you consider float values then there are plenty of values that exist between 2 and 3.

GlennS · 2023-10-16T00:24:48Z

tests/test_ranges.py

+                end=datetime.date(2000, 1, 4),
+            )
+
+        def test_handles_ranges_out_of_order(self):


🐼 I don't follow - isn't this identical to the 2nd case for the test above?

Yeah I think I meant to delete this in a test refactor, my bad.

GlennS · 2023-10-16T00:26:34Z

tests/test_ranges.py

+            union = range | other
+            assert union is None
+
+        @pytest.mark.parametrize(


🐼 this test is complex enough that it should probably have ids on its cases.

GlennS

Definitely a 👍 from me - this has been bugging me for a while.

leamingrad

This looks sensible to me, but I think it might be bugged in its current form so am requesting changes.

leamingrad · 2023-10-16T08:50:27Z

xocto/ranges.py

+        # range to allow them to be considered adjacent.
+        other_start = other.start
+        if other._is_right_inclusive:
+            assert other.start


❓ Why does this assertion hold? I would expect (None, '2022-01-01'] to be a valid range.

Maybe this should be ._is_left_inclusive?

🤔 You are right, this is a bug, I will review my tests to see why they weren't failing.

Because I was testing this behaviour via union it turns out this method wasn't run in the problematic case, because the union method orders the ranges first and then was calling the is_disjoint method on the other range instead.

I have addressed now with additional tests specifically for the is_disjoint method.

leamingrad · 2023-10-16T08:51:40Z

xocto/ranges.py

+        other_start = other.start
+        if other._is_right_inclusive:
+            assert other.start
+            other_start -= datetime.timedelta(days=1)


I also think this might be a wider problem, and some kind of resolution might be a good idea (although since ranges can be for any orderable type, it might not always work).

I agree that we shouldn't block here though.

josh-lynch · 2023-10-16T12:53:07Z

Ahh @delfick thanks that makes total sense 👌

Primarily this change is aimed at changing the behaviour of the FiniteDateRange.union. Previously FiniteDateRanges would not make a union with adjacent ranges, despite their inclusive boundaries meaning the presence of an adjacent range means they cover a fully contiguous period of time. This change makes adjacent FiniteDateRanges unionize by no longer considering ranges on adjacent days disjoint.

leamingrad

LGTM

josh-lynch changed the base branch from main to ranges/bugfix-days October 10, 2023 16:22

josh-lynch changed the title ~~ranges/bugfix union~~ Fix bug in FiniteDateRange.union Oct 10, 2023

josh-lynch force-pushed the ranges/bugfix-union branch from 9ac0358 to a8cec90 Compare October 10, 2023 16:23

josh-lynch commented Oct 10, 2023

View reviewed changes

xocto/ranges.py Show resolved Hide resolved

josh-lynch requested review from leamingrad and samueljsb October 10, 2023 16:24

josh-lynch mentioned this pull request Oct 10, 2023

Fix bug in FiniteDateRange.days property #98

Merged

josh-lynch force-pushed the ranges/bugfix-union branch 7 times, most recently from 495d8a1 to cdf8acf Compare October 12, 2023 11:27

josh-lynch changed the title ~~Fix bug in FiniteDateRange.union~~ Fix bug in FiniteDateRange.is_disjoint (and union as a result) Oct 12, 2023

josh-lynch force-pushed the ranges/bugfix-union branch from cdf8acf to 6d6ab5f Compare October 12, 2023 11:29

josh-lynch requested a review from a team October 12, 2023 11:36

GlennS reviewed Oct 16, 2023

View reviewed changes

GlennS approved these changes Oct 16, 2023

View reviewed changes

leamingrad requested changes Oct 16, 2023

View reviewed changes

josh-lynch force-pushed the ranges/bugfix-union branch from 6d6ab5f to 3cdc852 Compare October 16, 2023 13:28

josh-lynch requested a review from leamingrad October 16, 2023 13:30

josh-lynch force-pushed the ranges/bugfix-union branch from 3cdc852 to f3b0bba Compare October 16, 2023 13:31

josh-lynch force-pushed the ranges/bugfix-union branch from f3b0bba to c36a375 Compare October 16, 2023 13:53

Base automatically changed from ranges/bugfix-days to main October 17, 2023 12:20

leamingrad approved these changes Oct 17, 2023

View reviewed changes

josh-lynch merged commit 97a54c4 into main Oct 17, 2023
1 check passed

josh-lynch deleted the ranges/bugfix-union branch October 17, 2023 12:53

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix bug in FiniteDateRange.is_disjoint (and union as a result) #99

Fix bug in FiniteDateRange.is_disjoint (and union as a result) #99

josh-lynch commented Oct 10, 2023 •

edited

Loading

josh-lynch commented Oct 12, 2023

pydanny commented Oct 12, 2023 •

edited

Loading

delfick commented Oct 13, 2023

frederizia commented Oct 13, 2023

GlennS Oct 16, 2023

leamingrad Oct 16, 2023

josh-lynch Oct 16, 2023

GlennS Oct 16, 2023

josh-lynch Oct 16, 2023

GlennS Oct 16, 2023

josh-lynch Oct 16, 2023

GlennS left a comment

leamingrad left a comment

leamingrad Oct 16, 2023

josh-lynch Oct 16, 2023

josh-lynch Oct 16, 2023

leamingrad Oct 16, 2023

josh-lynch commented Oct 16, 2023

leamingrad left a comment

Fix bug in FiniteDateRange.is_disjoint (and union as a result) #99

Fix bug in FiniteDateRange.is_disjoint (and union as a result) #99

Conversation

josh-lynch commented Oct 10, 2023 • edited Loading

josh-lynch commented Oct 12, 2023

pydanny commented Oct 12, 2023 • edited Loading

delfick commented Oct 13, 2023

frederizia commented Oct 13, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

GlennS left a comment

Choose a reason for hiding this comment

leamingrad left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

josh-lynch commented Oct 16, 2023

leamingrad left a comment

Choose a reason for hiding this comment

josh-lynch commented Oct 10, 2023 •

edited

Loading

pydanny commented Oct 12, 2023 •

edited

Loading