Skip to content

Add $toDate test cases from $convert tests#101

Merged
eerxuan merged 5 commits intodocumentdb:mainfrom
danielfrankcom:pr/todate-convert-tests
Apr 17, 2026
Merged

Add $toDate test cases from $convert tests#101
eerxuan merged 5 commits intodocumentdb:mainfrom
danielfrankcom:pr/todate-convert-tests

Conversation

@danielfrankcom
Copy link
Copy Markdown
Collaborator

@danielfrankcom danielfrankcom commented Apr 15, 2026

This change expands $toDate test coverage with cases uncovered while testing $convert.

The main addition is the DATETOSTRING_YEAR_RANGE_ERROR error code and tests around year parsing. The behavior around the year parsing logic is quite strange, and in some cases MongoDB returns years which are unparseable by the BSON driver.

It's not overly clear why the MongoDB range and Python range differ, and it's hard to say if the mismatch points to a bug or not. As a result this adds a new assertExceptionType helper which allows us to assert the BSON driver throws an exception based on the result. This is the current behavior when testing against MongoDB, so it seems like part of the spec even though it's unusual.

There is also a likely bug observed when testing against MongoDB: when a date string has more than 4 digits before the first -, $toDate silently produces a wrong date instead of rejecting the input. The parser reads the first 4 characters as the year, and when that value exceeds 2059, it falls back to re-parsing from the 5th character using a short year format. DocumentDB does not exhibit this behavior and correctly rejects these inputs.

coll.aggregate([{"$project": {"r": {"$toDate": "20609-01-01T00:00:00Z"}}}])
# Expected: OperationFailure(241, "Error parsing date string")
# MongoDB: [{"r": datetime(2009, 1, 1)}]
# Parser reads year 2060, falls back, re-parses "9-01-01" as year 2009

The fallback applies a 2-digit year cutoff at 2059:

  • First 4 digits ≤ 2059 (e.g. 20599): correctly rejected
  • First 4 digits > 2059 (e.g. 20609): silently misparses remainder

This affects any digit count, even a 50-digit year string is silently accepted and misparsed. Both sides of the 2059 boundary are tested, along with longer digit sequences. The tests assert the expected correct behavior (rejection), which DocumentDB already implements.

Other new test cases include:

  • String whitespace trimming (leading/trailing space, tab, newline, null byte, NBSP)
  • Date/time separator variants and timezone offset formats (+HH:MM, +HHMM, -00:00, +00, etc.)
  • Year boundary tests (years 0001, 9999, 10000)
  • Out-of-Python-range year verification via $year extraction (years -1, 0, 10000)
  • Additional invalid string and type rejection cases

$convert integration

Moves test_smoke_toDate.py from date/toDate/ to type/toDate/ to collocate it with the detailed tests and other similar operators.

Adds minimal convert_expr and convert_field_expr helpers in convert/utils/convert_common.py, and parametrizes test_toDate_basic.py and test_toDate_expressions.py over both the native $toDate operator and the equivalent $convert expression. This follows the pattern established for $toString and applied to all to* operators, since forms of $convert are aliases for these operators.

Ref: #15

Signed-off-by: Daniel Frankcom <frankcom@amazon.com>
@danielfrankcom danielfrankcom requested a review from a team as a code owner April 15, 2026 17:01
@danielfrankcom
Copy link
Copy Markdown
Collaborator Author

The 16 test failures for CI are expected. They are the 4 parameterized cases for the following: string_year_5_digits_high_prefix, string_year_5_digits_max, string_year_6_digits, string_year_50_digits.

Once #79 is merged I can mark these as expected failures for MongoDB, but I don't want to duplicate the framework effort for adding that machinery.

Signed-off-by: Daniel Frankcom <frankcom@amazon.com>
Signed-off-by: Daniel Frankcom <frankcom@amazon.com>
Signed-off-by: Daniel Frankcom <frankcom@amazon.com>
Signed-off-by: Daniel Frankcom <frankcom@amazon.com>
@eerxuan eerxuan merged commit f729c84 into documentdb:main Apr 17, 2026
5 checks passed
@danielfrankcom danielfrankcom deleted the pr/todate-convert-tests branch April 20, 2026 16:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants