Elasticsearch DateMath support #113

niemyjski · 2025-09-22T01:41:37Z

https://www.elastic.co/docs/reference/query-languages/query-dsl/query-dsl-query-string-query#_ranges

Extends the date range parsing functionality to support wildcard characters, allowing for open-ended date ranges. Implements a new `WildcardPartParser` to handle "*" characters, representing either the minimum or maximum DateTimeOffset value based on the context. Updates the `TwoPartFormatParser` to correctly handle bracketed date ranges and wildcard characters.

Adds support for "TO" delimiter and bracket syntax to the two-part date range parser. This allows for more flexible date range input formats, including those used by Elasticsearch.

CLAassistant · 2025-09-22T01:41:43Z

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

Copilot

Pull Request Overview

This PR adds Elasticsearch DateMath support to the DateTime extensions library, specifically implementing bracket syntax ([ and {), TO delimiter support, and wildcard (*) functionality for date range queries. The implementation follows Elasticsearch's query-dsl-query-string-query range syntax patterns.

Adds wildcard part parser for handling * in date ranges (maps to DateTime.MinValue/MaxValue)
Extends TwoPartFormatParser to support Elasticsearch bracket syntax and TO delimiter
Comprehensive test coverage for all new syntax patterns including edge cases

Reviewed Changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

File	Description
`src/Exceptionless.DateTimeExtensions/FormatParsers/FormatParsers/TwoPartFormatParser.cs`	Updated regex patterns to support Elasticsearch bracket syntax (`[` `]` `{` `}`)
`src/Exceptionless.DateTimeExtensions/FormatParsers/FormatParsers/PartParsers/WildcardPartParser.cs`	New parser implementation for wildcard (`*`) support in date ranges
`tests/Exceptionless.DateTimeExtensions.Tests/FormatParsers/TwoPartFormatParserTests.cs`	Added test cases for new syntax patterns including TO delimiter, brackets, and wildcards
`tests/Exceptionless.DateTimeExtensions.Tests/FormatParsers/PartParsers/WildcardPartParserTests.cs`	Comprehensive test suite for wildcard parser functionality

...ceptionless.DateTimeExtensions/FormatParsers/FormatParsers/PartParsers/WildcardPartParser.cs

Adds a new entry to .gitignore to prevent tracking of specific IDE configuration files related to the project. This ensures a cleaner repository by excluding auto-generated files and preventing accidental commits of personal IDE settings.

Adds `RegexOptions.Compiled` to regex definitions in the `TwoPartFormatParser` to improve performance.

Introduces tests for parsing date ranges using various delimiters including dashes, "TO", and Elasticsearch bracket syntax. Also, adds tests for wildcard support in date range parsing. Addresses and updates existing wildcard parser tests.

Implements parsing of Elasticsearch-style date math expressions within date ranges, offering flexible and precise date calculations. This includes support for: - Anchors (now, explicit dates) - Operations (+, -, /) - Units (y, M, w, d, h, m, s) - Timezone handling Updates the DateTimeRange parsing to support bracket notation and wildcards.

...ceptionless.DateTimeExtensions/FormatParsers/FormatParsers/PartParsers/DateMathPartParser.cs

ejsmith

Looks good, but really wish you had kept the DateMath class separate so it could be used by itself.

...ceptionless.DateTimeExtensions/FormatParsers/FormatParsers/PartParsers/DateMathPartParser.cs

Refactors the DateMathPartParser to enhance performance and accuracy in parsing Elasticsearch date math expressions. - Optimizes explicit date parsing by using length-based format selection and pre-compiled regex. - Improves timezone handling and format ordering for better parsing. - Adds comprehensive tests to ensure correctness.

Copilot

Pull Request Overview

Copilot reviewed 8 out of 9 changed files in this pull request and generated 9 comments.

Copilot · 2025-09-22T03:21:15Z

src/Exceptionless.DateTimeExtensions/FormatParsers/FormatParsers/TwoPartFormatParser.cs

 public class TwoPartFormatParser : IFormatParser
 {
-    private static readonly Regex _beginRegex = new(@"^\s*", RegexOptions.Compiled);
+    private static readonly Regex _beginRegex = new(@"^\s*(?:[\[\{])?\s*", RegexOptions.Compiled);


The updated regexes consume opening and closing brackets/braces but do not capture which type was used; without capturing, the parser cannot distinguish inclusive [] from exclusive {} semantics (Elasticsearch treats {} as exclusive). This results in silently treating exclusive bounds as inclusive. Either (a) remove brace support until exclusivity is implemented, or (b) capture bracket type and adjust start/end boundaries accordingly (e.g., shift by smallest representable unit). Example: modify begin/end patterns to capture the delimiter type and pass it through parsing logic.

Copilot · 2025-09-22T03:21:16Z

src/Exceptionless.DateTimeExtensions/FormatParsers/FormatParsers/TwoPartFormatParser.cs

+    private static readonly Regex _beginRegex = new(@"^\s*(?:[\[\{])?\s*", RegexOptions.Compiled);
    private static readonly Regex _delimiterRegex = new(@"\G(?:\s*-\s*|\s+TO\s+)", RegexOptions.Compiled | RegexOptions.IgnoreCase);
-    private static readonly Regex _endRegex = new(@"\G\s*$", RegexOptions.Compiled);
+    private static readonly Regex _endRegex = new(@"\G\s*(?:[\]\}])?\s*$", RegexOptions.Compiled);


The updated regexes consume opening and closing brackets/braces but do not capture which type was used; without capturing, the parser cannot distinguish inclusive [] from exclusive {} semantics (Elasticsearch treats {} as exclusive). This results in silently treating exclusive bounds as inclusive. Either (a) remove brace support until exclusivity is implemented, or (b) capture bracket type and adjust start/end boundaries accordingly (e.g., shift by smallest representable unit). Example: modify begin/end patterns to capture the delimiter type and pass it through parsing logic.

Copilot · 2025-09-22T03:21:16Z

src/Exceptionless.DateTimeExtensions/FormatParsers/FormatParsers/TwoPartFormatParser.cs

 public class TwoPartFormatParser : IFormatParser
 {
-    private static readonly Regex _beginRegex = new(@"^\s*", RegexOptions.Compiled);
+    private static readonly Regex _beginRegex = new(@"^\s*(?:[\[\{])?\s*", RegexOptions.Compiled);


These patterns allow mismatches like an opening '{' with a closing ']' or a closing bracket without any opener, which can mask malformed input. Consider validating for a balanced, matching pair (and rejecting lone or mismatched delimiters) by deferring closing delimiter consumption until after successful two-part parsing and explicitly checking the original characters.

Copilot · 2025-09-22T03:21:16Z

src/Exceptionless.DateTimeExtensions/FormatParsers/FormatParsers/TwoPartFormatParser.cs

+    private static readonly Regex _beginRegex = new(@"^\s*(?:[\[\{])?\s*", RegexOptions.Compiled);
    private static readonly Regex _delimiterRegex = new(@"\G(?:\s*-\s*|\s+TO\s+)", RegexOptions.Compiled | RegexOptions.IgnoreCase);
-    private static readonly Regex _endRegex = new(@"\G\s*$", RegexOptions.Compiled);
+    private static readonly Regex _endRegex = new(@"\G\s*(?:[\]\}])?\s*$", RegexOptions.Compiled);


These patterns allow mismatches like an opening '{' with a closing ']' or a closing bracket without any opener, which can mask malformed input. Consider validating for a balanced, matching pair (and rejecting lone or mismatched delimiters) by deferring closing delimiter consumption until after successful two-part parsing and explicitly checking the original characters.

Copilot · 2025-09-22T03:21:16Z

README.md

+// Bracket notation support [start TO end]
+var bracketRange = DateTimeRange.Parse("[2023-01-01 TO 2023-12-31]", DateTime.Now);


Documentation shows only square bracket usage but the tests and parser also accept curly braces; the README does not clarify (a) whether {} are supported, or (b) inclusive vs exclusive semantics. Either document that {} are currently treated the same as [] (temporary limitation) or explain exclusivity once implemented to avoid developer confusion.

Suggested change

// Bracket notation support [start TO end]

var bracketRange = DateTimeRange.Parse("[2023-01-01 TO 2023-12-31]", DateTime.Now);

// Bracket notation support [start TO end] or {start TO end}

// Both square brackets [ ] and curly braces { } are accepted for range expressions.

// Currently, both are treated the same (inclusive bounds). Exclusive semantics for { } may be added in the future.

var bracketRange = DateTimeRange.Parse("[2023-01-01 TO 2023-12-31]", DateTime.Now);

var curlyBracketRange = DateTimeRange.Parse("{2023-01-01 TO 2023-12-31}", DateTime.Now); // Currently same as above

Copilot · 2025-09-22T03:21:17Z

...ceptionless.DateTimeExtensions/FormatParsers/FormatParsers/PartParsers/DateMathPartParser.cs

+            case 4: // Built-in: year (yyyy)
+                return TryParseWithFormat(dateStr, "yyyy", offset, false, out result);


The parser supports year-only and year-month anchors, but there are no corresponding tests exercising these formats (e.g., "2023||" or "2023-06||"). Add tests to ensure these branches remain correct and to catch regressions in length-based parsing.

Copilot · 2025-09-22T03:21:17Z

...ceptionless.DateTimeExtensions/FormatParsers/FormatParsers/PartParsers/DateMathPartParser.cs

+            case 7: // Built-in: year_month (yyyy-MM)
+                if (dateStr[4] == '-')
+                    return TryParseWithFormat(dateStr, "yyyy-MM", offset, false, out result);


The parser supports year-only and year-month anchors, but there are no corresponding tests exercising these formats (e.g., "2023||" or "2023-06||"). Add tests to ensure these branches remain correct and to catch regressions in length-based parsing.

Copilot · 2025-09-22T03:21:17Z

...ceptionless.DateTimeExtensions/FormatParsers/FormatParsers/PartParsers/DateMathPartParser.cs

+    private static readonly Regex _parser = new(
+        @"^(?<anchor>now|(?<date>\d{4}[-.]?\d{2}[-.]?\d{2}(?:[T\s](?:\d{1,2}(?::?\d{2}(?::?\d{2})?)?(?:\.\d{1,3})?)?(?:[+-]\d{2}:?\d{2}|Z)?)?)\|\|)" +
+        @"(?<operations>(?:[+\-/]\d*[yMwdhHms])*)$",
+        RegexOptions.Compiled | RegexOptions.IgnoreCase);


The regex permits multiple rounding operations or additional arithmetic after a rounding (e.g., "now/d+1h"), but tests do not cover invalid sequencing; Elasticsearch requires rounding (/unit) be the final operation. Add negative tests (expecting null) for sequences like "now/d+1h" and "/d/d" to ensure spec-aligned enforcement (or clarify in docs if divergence is intentional).

Copilot · 2025-09-22T03:21:17Z

...ceptionless.DateTimeExtensions/FormatParsers/FormatParsers/PartParsers/DateMathPartParser.cs

+    private static readonly Regex _parser = new(
+        @"^(?<anchor>now|(?<date>\d{4}[-.]?\d{2}[-.]?\d{2}(?:[T\s](?:\d{1,2}(?::?\d{2}(?::?\d{2})?)?(?:\.\d{1,3})?)?(?:[+-]\d{2}:?\d{2}|Z)?)?)\|\|)" +


The date portion allows a dotted format (e.g., 2001.02.01) via the [-.]? separators, but there are no tests covering dotted date inputs. Add tests like "2001.02.01||" (with and without operations) to validate this branch and prevent future regressions.

Adds a `DateMath` utility class for parsing Elasticsearch date math expressions, offering standalone date math functionality without range capabilities. This utility supports parsing expressions with `now`, explicit dates, operations, and time units, providing a simpler API for direct parsing operations. Includes comprehensive unit tests for various parsing scenarios, edge cases, and timezone handling.

- Improves date parsing by removing support for dotted date formats and enforcing hyphenated formats for consistency. - Adds validation to ensure rounding operations in date math expressions are only used as the final operation, aligning with specification requirements. - Enhances two-part format parser to validate matching brackets, preventing parsing errors due to unbalanced brackets.

Adds validation to ensure that brackets and braces are properly matched in the format parser. This prevents incorrect parsing when there are mismatched brackets/braces.

Extends the DateMath utility to support TimeZoneInfo, enabling accurate date parsing and calculations within specific timezones. This enhancement allows for parsing expressions using a specified timezone, ensuring that "now" calculations and dates without explicit timezone information are correctly interpreted. Dates with explicit timezone information are preserved, regardless of the TimeZoneInfo parameter.

Updates test method names to clearly indicate that they specifically test timezone parsing functionality. This improves clarity and maintainability of the test suite.

niemyjski added 2 commits September 21, 2025 20:40

Extends date range parser to support new formats

f952c8d

Adds support for "TO" delimiter and bracket syntax to the two-part date range parser. This allows for more flexible date range input formats, including those used by Elasticsearch.

niemyjski requested review from ejsmith and Copilot September 22, 2025 01:41

niemyjski self-assigned this Sep 22, 2025

niemyjski added the enhancement label Sep 22, 2025

Copilot AI reviewed Sep 22, 2025

View reviewed changes

...ceptionless.DateTimeExtensions/FormatParsers/FormatParsers/PartParsers/WildcardPartParser.cs Outdated Show resolved Hide resolved

Updates .gitignore to exclude IDE files

d4ad41f

Adds a new entry to .gitignore to prevent tracking of specific IDE configuration files related to the project. This ensures a cleaner repository by excluding auto-generated files and preventing accidental commits of personal IDE settings.

ejsmith previously approved these changes Sep 22, 2025

View reviewed changes

niemyjski added 3 commits September 21, 2025 20:51

Merge branch 'main' into feature/datemath

993eb8b

Improves regex parsing performance

46a32f7

Adds `RegexOptions.Compiled` to regex definitions in the `TwoPartFormatParser` to improve performance.

Adds two-part format parser tests

1b7ef73

Introduces tests for parsing date ranges using various delimiters including dashes, "TO", and Elasticsearch bracket syntax. Also, adds tests for wildcard support in date range parsing. Addresses and updates existing wildcard parser tests.

niemyjski dismissed ejsmith’s stale review via 1b7ef73 September 22, 2025 01:52

niemyjski added 2 commits September 21, 2025 21:38

pr feedbacj

6c4049d

niemyjski commented Sep 22, 2025

View reviewed changes

...ceptionless.DateTimeExtensions/FormatParsers/FormatParsers/PartParsers/DateMathPartParser.cs Outdated Show resolved Hide resolved

ejsmith requested changes Sep 22, 2025

View reviewed changes

...ceptionless.DateTimeExtensions/FormatParsers/FormatParsers/PartParsers/DateMathPartParser.cs Outdated Show resolved Hide resolved

niemyjski requested review from Copilot and ejsmith September 22, 2025 03:18

niemyjski marked this pull request as ready for review September 22, 2025 03:19

Copilot AI reviewed Sep 22, 2025

View reviewed changes

niemyjski added 5 commits September 21, 2025 22:47

Adds bracket matching validation to parser

a3c045e

Adds validation to ensure that brackets and braces are properly matched in the format parser. This prevents incorrect parsing when there are mismatched brackets/braces.

Renames DateMath tests for timezone parsing

b74c5ea

Updates test method names to clearly indicate that they specifically test timezone parsing functionality. This improves clarity and maintainability of the test suite.

niemyjski merged commit 3555cfe into main Sep 22, 2025
2 of 3 checks passed

niemyjski deleted the feature/datemath branch September 22, 2025 04:08

		// Bracket notation support [start TO end]
		var bracketRange = DateTimeRange.Parse("[2023-01-01 TO 2023-12-31]", DateTime.Now);

-// Bracket notation support [start TO end]
-var bracketRange = DateTimeRange.Parse("[2023-01-01 TO 2023-12-31]", DateTime.Now);
+// Bracket notation support [start TO end] or {start TO end}
+// Both square brackets [ ] and curly braces { } are accepted for range expressions.
+// Currently, both are treated the same (inclusive bounds). Exclusive semantics for { } may be added in the future.
+var bracketRange = DateTimeRange.Parse("[2023-01-01 TO 2023-12-31]", DateTime.Now);
+var curlyBracketRange = DateTimeRange.Parse("{2023-01-01 TO 2023-12-31}", DateTime.Now); // Currently same as above

		case 4: // Built-in: year (yyyy)
		return TryParseWithFormat(dateStr, "yyyy", offset, false, out result);

		private static readonly Regex _parser = new(
		@"^(?<anchor>now\|(?<date>\d{4}[-.]?\d{2}[-.]?\d{2}(?:[T\s](?:\d{1,2}(?::?\d{2}(?::?\d{2})?)?(?:\.\d{1,3})?)?(?:[+-]\d{2}:?\d{2}\|Z)?)?)\\|\\|)" +

Uh oh!

Elasticsearch DateMath support #113

Elasticsearch DateMath support #113

Uh oh!

Conversation

niemyjski commented Sep 22, 2025

Uh oh!

CLAassistant commented Sep 22, 2025

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

Uh oh!

ejsmith left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Uh oh!

Copilot AI Sep 22, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Sep 22, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Sep 22, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Sep 22, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Sep 22, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Sep 22, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Sep 22, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Sep 22, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Sep 22, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!