Skip to content

Fix date-only range stop dropping the final day (half-open [start, stop)) - #309

Merged
Bernhard Merkle (bmerkle) merged 2 commits into
microsoft:mainfrom
bmerkle:fix-date-range-half-open
Aug 5, 2026
Merged

Fix date-only range stop dropping the final day (half-open [start, stop))#309
Bernhard Merkle (bmerkle) merged 2 commits into
microsoft:mainfrom
bmerkle:fix-date-range-half-open

Conversation

@bmerkle

@bmerkle Bernhard Merkle (bmerkle) commented Aug 4, 2026

Copy link
Copy Markdown
Collaborator

The bug

date_range_from_datetime_range compiled a stop_date with no time to that day's midnight, so a query like "Jan 1 to Jan 5" became [Jan 1 00:00, Jan 5 00:00) and every message on Jan 5 was silently dropped.

Reproduced on main before the fix:

compiled DateRange: DateRange(2023-01-01 00:00+00:00, 2023-01-05 00:00+00:00)
index hits: ['2023-01-01T00:00:00+00:00', '2023-01-04T09:30:00+00:00']
   # dropped: 2023-01-05T00:00, 2023-01-05T09:30, 2023-01-05T23:59

This is the bug reported by Shadow_Lu (@LuShadowX) in #296 — credit for finding it goes to them. That PR fixed it by padding the (then inclusive) end to 23:59:59.999999; this PR fixes it with the half-open [start, stop) convention agreed in #198 and documented in AGENTS.md instead.

The fix

Three source edits

  1. searchlang.py — new exclusive_stop_from_date_time(); a stop_date with no time now rolls to next-day midnight. One if, plus docstrings.
  2. interfaces_core.py — DateRange.end comment inclusive → exclusive, and contains <= → <. Plus a docstring.
  3. convutils.py — docstring only, no code.

Tests: one regression test for the bug, unit tests for the new helper, two DateRange.contains tests, and test_start_and_stop updated to expect Jan 1 instead of Dec 31.

Full make is green: isort/black clean, pyright clean, 756 tests pass, build OK.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This pull request fixes an off-by-one-day bug when compiling date-only stop bounds in date_range_from_datetime_range, aligning the system around the half-open interval convention [start, stop) so the final day is not silently excluded.

Changes:

  • Introduces exclusive_stop_from_date_time() so date-only stops compile to the next day’s midnight (exclusive bound).
  • Updates DateRange semantics/docs and __contains__ to be half-open (start <= dt < end).
  • Adds/updates tests covering date-only stops, explicit-time stops, rollover cases, and half-open containment behavior.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
tests/test_searchlang_compile.py Adds regression + unit tests for exclusive-stop compilation and date-range behavior.
tests/test_interfaces.py Adds containment tests verifying DateRange is half-open and unbounded when end=None.
src/typeagent/knowpro/searchlang.py Switches stop compilation to an exclusive bound via exclusive_stop_from_date_time().
src/typeagent/knowpro/interfaces_core.py Updates DateRange documentation and containment semantics to [start, end).
src/typeagent/knowpro/convutils.py Documents that the conversation time range is for prompt display and not for index lookups.

Comment thread src/typeagent/knowpro/convutils.py Outdated
Comment thread src/typeagent/knowpro/interfaces_core.py Outdated
@gvanrossum

Copy link
Copy Markdown
Collaborator

Too much copilot slop to review. Is there a brief summary of the changes and their consequences?

…op))

A search time range whose stop_date carries no time compiled to that day's
midnight, so "Jan 1 to Jan 5" became [Jan 1 00:00, Jan 5 00:00) and every
message on Jan 5 was silently excluded. Reported in PR microsoft#296.

Both timestamp index backends already filter half-open (get_in_range:
"End is exclusive"; SQLite: start_timestamp >= ? AND start_timestamp < ?),
so the fix is to make the compiled stop the exclusive bound the storage
layer expects: for a bare date, midnight of the following day. A stop with
an explicit time is already exclusive and is kept as-is. This follows the
[start, stop) convention agreed in PR microsoft#198 and documented in AGENTS.md,
rather than padding an inclusive end to 23:59:59.999999.

Also fixes two boundary inconsistencies this uncovered:

- DateRange documented and implemented its end as inclusive
  (start <= dt <= end) while both storage backends excluded it. It is now
  documented and implemented as half-open, so the two scope-filtering
  paths agree: with a timestamp index (lookup_range) and without one
  (get_text_range_for_date_range, which uses __contains__).
- get_time_range_for_conversation builds an inclusive-looking end from the
  last message's timestamp; documented as prompt-display only, not for
  index lookups.

get_enclosing_date_range_for_text_range already used the exclusive end
ordinal's timestamp, so it needed no change.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@bmerkle

Copy link
Copy Markdown
Collaborator Author

Summary of changes are in description now shorter.
python docs strings area now shorter. Copilot slop removed

@LuShadowX

Copy link
Copy Markdown
Contributor

Bernhard Merkle (@bmerkle) I took you up on the offer from #296 — the half-open version is in #312, built on your design and with you credited on the commit. Close this one whenever suits you.

@bmerkle
Bernhard Merkle (bmerkle) merged commit 4c4e6b0 into microsoft:main Aug 5, 2026
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants