Fix: convert-workload produces an unparseable date bound, and loses a day on a whole-day lte

[serhiy-bzhezytskyy]

Closes #73.

Two date-bound defects in _convert_date_to_solr_format, both independent of the
bracket fix in #69 and both still present on main at f088380.

A datetime with a space separator was passed through verbatim. yyyy-MM-dd HH:mm:ss
was missing from the format map, and an unknown format is logged and used as-is. A space
inside a range term ends the term, so Solr answers HTTP 400 and the operation cannot run
at all:

Cannot parse 'dropoff_datetime:[2015-01-01 00:00:00 TO 2016-01-01 00:00:00}':
Encountered " <RANGE_GOOP> "00:00:00 "" at line 1, column 28.

#69 gave that query a correct exclusive bracket, which does not help while the query
still fails to parse. Two nyc_taxis operations are affected.

A whole-day lte was not rounded. A bound with no time names a day, and OpenSearch
rounds it to that day's edge — lte and gt to its last millisecond, gte and lt to
its first. Solr rounds nothing, so only the two that move to the end of the day need
translating. dd/MM/yyyy was already in the map, so the value converted cleanly and
silently landed on midnight, dropping the rest of the day.

Note the bracket in that case is inclusive on main and stays inclusive here: upstream
writes lte, so ] was always right. The only thing wrong was the value.

Approach

The format map now records, per format, whether the pattern carries a time, so a
whole-day bound is recognisable after parsing rather than guessed at from its length.
Rounding then advances such a bound to the following day's first instant and makes the
bracket exclusive:

dropoff_datetime:[2015-01-01T00:00:00Z TO 2015-01-22T00:00:00Z}

...T23:59:59.999Z] would be the more literal reading, but it assumes Solr's date
precision is exactly a millisecond. Naming the next day's first instant says "the whole
of the 21st" without depending on that.

Verification

300,649 nyc_taxis documents indexed into Solr 10.0.0 and OpenSearch 3.8.0 from one
file, line number as the id on both sides. Provenance checked rather than assumed:
300,649 corpus lines against 300,649 OpenSearch documents, 10 sampled ids byte-identical,
and the Solr schema read back from the live core.

	OpenSearch	before	after
date_histogram_facet matches	17,651	16,811	17,651
daily buckets returned	21	20	21

Equal counts can hide compensating differences, so I compared document id sets rather
than totals: before, 840 missing and 0 extra; after, the sets are equal. And the buckets
the operation actually returns:

OpenSearch   21 buckets   last: ('2015-01-21', 840)
before       20 buckets   last: ('2015-01-20', 828)
after        21 buckets   last: ('2015-01-21', 840)

2015-01-21 is absent from the old output entirely, not merely short.

Separately, the converter's output for the three real upstream nyc_taxis bodies is now
byte-identical to what the hand-written Solr port ships — two independent routes reaching
the same string.

5 new unit tests; pytest tests/ passes 1115, skips 5.

apache/solr-orbit-workloads#19 corrected the same bounds in the checked-in nyc_taxis
files. This is the generator, so without it a regeneration reintroduces both.

[Copilot]

🟡 Changes recommended

There are two edge-case logic bugs in the new rounding/date-only detection that can incorrectly round or change bracket semantics when dates aren’t actually parsed or when both gt and gte are provided.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

This PR fixes date range bound translation in the OpenSearch→Solr workload converter so that converted range queries both (1) remain parseable by Solr when input datetimes use a space separator and (2) preserve OpenSearch’s whole-day rounding semantics for date-only lte/gt bounds.

Changes:

• Extend date parsing to handle yyyy-MM-dd HH:mm:ss and track whether a parsed bound is “date-only” vs an instant.
• Implement whole-day rounding for date-only lte (upper) and gt (lower) bounds by advancing to the next day’s start and adjusting bracket exclusivity accordingly.
• Add unit tests covering space-separated datetimes and whole-day rounding behavior for lte, gt, gte, and lt.

File summaries

File	Description
solrorbit/conversion/query.py	Updates date parsing/format mapping and applies whole-day rounding logic during range query translation.
tests/unit/solr/test_workload_converter.py	Adds focused unit tests validating the new date conversion and rounding behaviors.

Review details

• Files reviewed: 2/2 changed files
• Comments generated: 2
• Review effort level: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[janhoy]

Hint - we normally dont force push PR branches unless necessary for other reasons (rebase etc), as it removes the commit stack as a history you can review.