ESQL: Locale and timezone argument for date_parse #136548

flash1293 · 2025-10-14T16:13:38Z

POST _query
{
  "query": """
    ROW date_string="10 septembre 2025" | EVAL date = DATE_PARSE("dd MMMM yyyy", date_string, {"locale": "fr", "time_zone": "Europe/Paris" })
  """
}

returns

   date_string   |          date
-----------------+------------------------
10 septembre 2025|2025-09-09T22:00:00.000Z

In case the time zone or the locale can't be resolved, this throws an error.

...ql/src/main/java/org/elasticsearch/xpack/esql/expression/function/scalar/date/DateParse.java

…ing-settings

github-actions · 2025-10-17T09:45:04Z

ℹ️ Important: Docs version tagging

👋 Thanks for updating the docs! Just a friendly reminder that our docs are now cumulative. This means all 9.x versions are documented on the same page and published off of the main branch, instead of creating separate pages for each minor version.

We use applies_to tags to mark version-specific features and changes.

Expand for a quick overview

When to use applies_to tags:

✅ At the page level to indicate which products/deployments the content applies to (mandatory)
✅ When features change state (e.g. preview, ga) in a specific version
✅ When availability differs across deployments and environments

What NOT to do:

❌ Don't remove or replace information that applies to an older version
❌ Don't add new information that applies to a specific version without an applies_to tag
❌ Don't forget that applies_to tags can be used at the page, section, and inline level

🤔 Need help?

Check out the cumulative docs guidelines
Reach out in the #docs Slack channel

elasticsearchmachine · 2025-10-17T11:21:36Z

Pinging @elastic/es-analytical-engine (Team:Analytics)

elasticsearchmachine · 2025-10-17T11:21:43Z

Hi @flash1293, I've created a changelog YAML for you.

ivancea · 2025-10-17T11:52:26Z

...c/test/java/org/elasticsearch/xpack/esql/expression/function/scalar/date/DateParseTests.java

        assertThat(e.getMessage(), startsWith("invalid date pattern for []: Invalid format: [" + pattern + "]"));
    }

+    public void testInvalidLocale() {


Can you add those tests as TestCaseSuppliers in the parameters() method? They automatically execute a bunch of cases.
I'm not sure of how map parameters work there really, but if it works like other params,you would be able to just test that it returns null with a warning with the error, which is what the user would see

actually it's different because this is not just a per-row warning. It actually fails the query. Since they are static this isn't a recoverable error.

...rg/elasticsearch/xpack/esql/expression/function/scalar/date/DateParseSerializationTests.java

ivancea · 2025-10-17T12:02:59Z

...ql/src/main/java/org/elasticsearch/xpack/esql/expression/function/scalar/date/DateParse.java

+        super(source, fields(first, second, options));
        this.field = second != null ? second : first;
        this.format = second != null ? first : null;
+        this.options = options;


This troubles me a bit. We're only allowing the options if the 2 previous parameters are present, even knowing that the format is optional too. That means, we're not allowing something like: DATE_PARSE(date, {})

However, the SVG (docs) shows it correctly.

Now, I don't know if we did something like this before, or if we should allow it. Our function overriding detection is quite manual right now, to begin with. I would like if somebody else from the team can review this first.

The worse that could happen if we ship this is that:

We would have incorrect docs

We would probably give meaningless errors, as users would expect the map parameter to "work"

Good call... To me both allowing DATE_PARSE(date, {}) or enforcing three params for options make sense, if we clearly communicate this to the user.

Happy for someone else to make the call.

For reference, DATE_PARSE(date_string, {\"time_zone\": \"Europe/Paris\",\"locale\":\"fr\"}) yields

{ "error": { "root_cause": [ { "type": "verification_exception", "reason": "Found 1 problem\nline 1:51: second argument of [DATE_PARSE(date_string, {\"time_zone\": \"Europe/Paris\",\"locale\":\"fr\"})] must be [string], found value [{\"time_zone\": \"Europe/Paris\",\"locale\":\"fr\"}] type [unsupported]" } ], "type": "verification_exception", "reason": "Found 1 problem\nline 1:51: second argument of [DATE_PARSE(date_string, {\"time_zone\": \"Europe/Paris\",\"locale\":\"fr\"})] must be [string], found value [{\"time_zone\": \"Europe/Paris\",\"locale\":\"fr\"}] type [unsupported]" }, "status": 400 }

right now.

Your call of course, but if you feel comfortable with this behavior, we can get it in, then follow up later with supporting a more flexible calling pattern - we are lucky that the existing behavior on this PR is a subset of what could realistically be supported.

I'm fine with closing this as-is. I would try to follow up soon though, as to not have the wrong docs there.

Btw, I was discussing this a bit, and I think we could make a working check here. Something like:

if (options == null) { if (second == null) { // 1 parameter, it's the date } else { if (second instanceof MapExpression) { // Second and options params, no format } else { // First and second params, no options } } } else { // 3 params available, no doubt here }

In general, having an optional param before a required is quite weird. But this is ""historical"" already, so here we are. For this special case, I think the logic to check it shouldn't be too complex, and we can manage to do it.
If there's some weird planning error after doing it, we can check it 👀

Good idea to check with instanceof MapExpression, this seems to work fine, updated the PR

…3/elasticsearch into flash1293/date-parsing-settings

…ing-settings

flash1293 · 2025-10-22T10:05:39Z

Should be ready for another look

flash1293 · 2025-10-22T16:23:09Z

I've been trying to figure it out, but I can't make it work, I'm stuck. Happy to pair with someone. It seems like the failure test case builder is building a failure case where the third parameter is null, and it complains it's not rejected as unresolved. I think I have to add a test case for this so it's marked as valid, but I can't manage to - if I try something like

new TestCaseSupplier(
                    List.of(DataType.KEYWORD, DataType.KEYWORD, DataType.NULL),
                    () -> new TestCaseSupplier.TestCase(
                        List.of(
                            new TestCaseSupplier.TypedData(new BytesRef("yyyy-MM-dd"), DataType.KEYWORD, "pattern"),
                            new TestCaseSupplier.TypedData(new BytesRef("2023-05-05"), DataType.KEYWORD, "date"),
                            new TestCaseSupplier.TypedData(
                                null,
                                DataType.NULL,
                                "options"
                            )
                        ),
                        "DateParseEvaluator[val=Attribute[channel=1], formatter=Attribute[channel=0]]",
                        DataType.DATETIME,
                        equalTo(1683244800000L)
                    )
                )

it breaks really bad and I'm not sure why. I get a bit confused with the null as in Java null pointer vs. ESQL null, something's up there and I'm not even sure what the right behavior should be - do we want to allow date_parse(a, b, null) or not, and depending on the answer, how do I construct the right test cases?

nik9000

LGTM.

Enjoy fighting CI.

nik9000 · 2025-10-22T16:28:52Z

...ql/src/main/java/org/elasticsearch/xpack/esql/expression/function/scalar/date/DateParse.java

+                    type = "keyword",
+                    valueHint = { "standard" },
+                    description = "The locale to use when parsing the date, relevant when parsing month names or week days."
+                ) },


I wonder if there's a way we can declare this thing once and reuse it. Java annotations have such weird rules.

nik9000 · 2025-10-22T16:31:08Z

...ql/src/main/java/org/elasticsearch/xpack/esql/expression/function/scalar/date/DateParse.java

+        return list;
+    }
+
+    private Expression field() {


Worth a javadoc - it's the value to translate but the name field is ambiguous.

nik9000 · 2025-10-22T16:31:30Z

...ql/src/main/java/org/elasticsearch/xpack/esql/expression/function/scalar/date/DateParse.java

+        return second != null ? second : first;
+    }
+
+    private Expression format() {


Probably worth a javadoc too.

nik9000 · 2025-10-22T16:33:29Z

...ql/src/main/java/org/elasticsearch/xpack/esql/expression/function/scalar/date/DateParse.java

+
+    private final Expression first;
+    private final Expression second;
+    private final Expression third;


It's worth leaving a comment explaining that you chose to do this naming instead of in the ctor because the first and last parameter is optional and it's just easier to do handle it this way.

nik9000 · 2025-10-22T16:35:03Z

...sql/src/main/java/org/elasticsearch/xpack/esql/expression/function/EsqlFunctionRegistry.java

+                children.get(0),
+                children.size() > 1 ? children.get(1) : null,
+                children.size() == 3 ? children.get(2) : null
+            );


That's.... fun. I don't really like it but it gets the job done. We can refine this later.

…ing-settings

flash1293 · 2025-10-24T09:25:31Z

Test failure looks like CI instability (503), trying to rerun

Fixes elastic#132487 ``` POST _query { "query": """ ROW date_string="10 septembre 2025" | EVAL date = DATE_PARSE("dd MMMM yyyy", date_string, {"locale": "fr", "time_zone": "Europe/Paris" }) """ } ``` returns ``` date_string | date -----------------+------------------------ 10 septembre 2025|2025-09-09T22:00:00.000Z ``` In case the time zone or the locale can't be resolved, this throws an error.

date parse settings

acc9837

elasticsearchmachine added v9.3.0 external-contributor Pull request authored by a developer outside the Elasticsearch team labels Oct 14, 2025

[CI] Auto commit changes from spotless

e9ee7b8

nik9000 requested a review from ivancea October 15, 2025 13:19

ivancea reviewed Oct 15, 2025

View reviewed changes

...ql/src/main/java/org/elasticsearch/xpack/esql/expression/function/scalar/date/DateParse.java Outdated Show resolved Hide resolved

flash1293 and others added 5 commits October 16, 2025 16:18

Merge remote-tracking branch 'upstream/main' into flash1293/date-pars…

104648b

…ing-settings

work work work

0f39275

[CI] Auto commit changes from spotless

f3932ca

Merge remote-tracking branch 'upstream/main' into flash1293/date-pars…

38a58cd

…ing-settings

work 2

76dedf5

flash1293 marked this pull request as ready for review October 17, 2025 09:53

elasticsearchmachine added the needs:triage Requires assignment of a team area label label Oct 17, 2025

fix checkstyle violation

e12c332

flash1293 added >enhancement Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) :Analytics/ES|QL AKA ESQL labels Oct 17, 2025

elasticsearchmachine removed the needs:triage Requires assignment of a team area label label Oct 17, 2025

Update docs/changelog/136548.yaml

f4e4518

[CI] Auto commit changes from spotless

f3187d8

ivancea reviewed Oct 17, 2025

View reviewed changes

flash1293 added 4 commits October 17, 2025 14:26

missing space

435bd98

Merge branch 'flash1293/date-parsing-settings' of github.com:flash129…

706c558

…3/elasticsearch into flash1293/date-parsing-settings

Merge remote-tracking branch 'upstream/main' into flash1293/date-pars…

701040b

…ing-settings

Review comments

507831b

checkpoint

1c90c57

flash1293 and others added 2 commits October 22, 2025 13:24

fix tests

582761f

[CI] Auto commit changes from spotless

8687666

nik9000 approved these changes Oct 22, 2025

View reviewed changes

flash1293 and others added 3 commits October 24, 2025 10:21

Merge remote-tracking branch 'upstream/main' into flash1293/date-pars…

db21f1c

…ing-settings

fix tests

df0dca2

[CI] Auto commit changes from spotless

b81e688

flash1293 added 2 commits October 24, 2025 11:25

Merge branch 'main' into flash1293/date-parsing-settings

373b825

Merge branch 'main' into flash1293/date-parsing-settings

75cc23b

flash1293 enabled auto-merge (squash) October 24, 2025 12:33

flash1293 merged commit f40259a into elastic:main Oct 24, 2025
34 checks passed

ESQL: Locale and timezone argument for date_parse #136548

ESQL: Locale and timezone argument for date_parse #136548

Uh oh!

Conversation

flash1293 commented Oct 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

github-actions bot commented Oct 17, 2025

ℹ️ Important: Docs version tagging

When to use applies_to tags:

What NOT to do:

🤔 Need help?

Uh oh!

elasticsearchmachine commented Oct 17, 2025

Uh oh!

elasticsearchmachine commented Oct 17, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

flash1293 Oct 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

flash1293 commented Oct 22, 2025

Uh oh!

flash1293 commented Oct 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

nik9000 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

flash1293 commented Oct 24, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

flash1293 commented Oct 14, 2025 •

edited

Loading

flash1293 Oct 17, 2025 •

edited

Loading

flash1293 commented Oct 22, 2025 •

edited

Loading