Document ES|QL timezone handling in Kibana#5455
Document ES|QL timezone handling in Kibana#5455florent-leborgne wants to merge 12 commits intomainfrom
Conversation
Add a new section about the SET query directive to the ES|QL in Kibana page, covering the unmapped_fields setting with its FAIL and NULLIFY values. References the ES|QL SET directive reference page for full syntax details. Made-with: Cursor
✅ Vale Linting ResultsNo issues found on modified lines! The Vale linter checks documentation changes against the Elastic Docs style guide. To use Vale locally or report issues, refer to Elastic style guide for Vale. |
🔍 Preview links for changed docs |
Document the approximation setting with its boolean and map parameter forms, including num_rows and confidence_level parameters with suggested values. Made-with: Cursor
florent-leborgne
changed the title
Move the SET directive content from a nested H3 under "Write queries" to a standalone H2 section titled "Control query behavior with SET", placed between "Filter by time" and "Use variables and controls". This better reflects the conceptual parity with other top-level sections that control query behavior. Made-with: Cursor
The SET directive is in preview per the ES|QL reference docs and the unmapped_fields setting has preview: true in the Kibana codebase. Made-with: Cursor
Document the actual server-side defaults from the Elasticsearch codebase: 100K rows without grouping, 1M with grouping, rows minimum of 10000, confidence_level range 0.5-0.95. Also corrects the suggested confidence_level values (max is 0.95, not 0.99 as suggested in Kibana autocomplete). Made-with: Cursor
Made-with: Cursor
Cross-references the timezone handling section under Filter by time for the full explanation of how Kibana handles timezones. Made-with: Cursor
Consolidates the timezone documentation into this PR since both PRs modify the same page and cross-reference each other. The timezone section is placed at the end of "Filter by time" and covers both the Kibana advanced setting and SET time_zone override. Made-with: Cursor
|
Related PR: #5402 ℹ️ that PR will be closed and recreated in elasticsearch, so link targets would change |
Both settings are only available in snapshot builds and not supported in serverless environments. Made-with: Cursor
florent-leborgne
changed the title
- Remove approximation section: not part of SET directive (confirmed) - Fix unmapped_fields serverless tag: preview, not unavailable - Remove redundant Control query timezone subsection inside SET (content already covered in the Timezone handling section above) Made-with: Cursor
florent-leborgne
changed the title
The SET directive reference content (unmapped_fields table, examples) duplicates the ES|QL reference page without adding Kibana-specific value. The timezone handling section is the only genuinely Kibana-specific content: it documents the dateFormat:tz advanced setting behavior and the per-query SET time_zone override. Made-with: Cursor
florent-leborgne
changed the title
|
@leemthompo I rescoped this page to just be about timezone handling since the rest wasn't kibana specific, making it useless to duplicate the ES|QL syntax docs. I'll treat CPS project routing separately |
Made-with: Cursor
| 3. Set it to **Browser** to use your browser's timezone, or choose a specific timezone such as **UTC** or **America/New_York**. | ||
|
|
||
|
|
||
| **Per-query override with SET** |
There was a problem hiding this comment.
@ghudgins do we want to document at kibana the SET workaround? it won't work great as per our decision back then
There was a problem hiding this comment.
Could we update to say this property is set by kibana and will be overridden?
There was a problem hiding this comment.
It won't be overriden. Kibana will take the value returned from ES (with the timezone of SET) and apply the timezone of the advanced settings. So I am not sure this works as expected. And we had agreed it won't if you remember
There was a problem hiding this comment.
I only tested simple cases where it seemed to work so happy to take it out if needed, but because this is supported AND documented on the ES side, we need to be clear about how it works/doesn't work/isn't supported or wanted in Kibana and why
There was a problem hiding this comment.
Are you sure it works and the numbers make sense? Does the SET timezone is respected or the advanced settings one?
There was a problem hiding this comment.
So I looked into it, indeed it will work ok in Discover. But with a possible confusing outcome.
The tricky part is with date histograms such as BUCKET(@timestamp, 3 hour). ES|QL bucketed using Athens timezone boundaries — so 00:00, 03:00, 06:00 etc. are aligned to Athens midnight. If the user's dateFormat:tz (the advanced setting) is set to UTC, those bucket boundaries would display as 22:00, 01:00, 04:00 — which are correct instants but look like odd bucket boundaries.
For the above reason but also because we don't want users to create dashboards with different timezones as we don't have any UI logic about it (and we don't intend to have)
Here is the decision made back then:
- The editor won’t suggest the timezone in SET, neither the function level timezones
- Kibana apps will just support the space setting formatting and nothing else
- We won’t add labels in the dashboards indicating to the users that they use a different timezone
So now @florent-leborgne and @ghudgins having all this info in mind wdyt we should do?
There was a problem hiding this comment.
I think if we don't suggest it that's fine. we can document it maybe with a warning about conflicts with space level settings. matches the pattern with SET with CPS where the query "wins" if the user chooses to configure it
kosabogi
left a comment
kosabogi
left a comment
There was a problem hiding this comment.
LGTM with one small suggestion
| | STATS count = COUNT(*) BY BUCKET(@timestamp, 1 hour) | ||
| ``` | ||
|
|
||
| The editor does not suggest `SET time_zone` in autocomplete, but you can type it manually. You can also use it through the [Dev Tools Console](../tools/console.md) or the {{es}} API directly. |
There was a problem hiding this comment.
We could link the Elasticsearch API here: https://www.elastic.co/docs/api/doc/elasticsearch/
5 participants
Summary
Adds a Timezone handling section to the ES|QL in Kibana page, under the existing "Filter by time" section.
This is Kibana-specific content not covered in the ES|QL reference: it explains how Kibana controls the timezone used in ES|QL queries, and how to override it per-query using
SET time_zone.What it covers:
dateFormat:tz) advanced setting to all ES|QL queries in Discover, dashboards, alerting, and Maps.SET time_zone = "..."before the source command to override the advanced setting for that specific query. This must be typed manually — it is not surfaced in autocomplete by design.Verified against the Kibana codebase:
dateFormat:tzis passed to ES|QL queries viadata/common/search/expressions/esql.ts(line 185)SET time_zoneis intentionally excluded from autocomplete inkbn-esql-language/scripts/generate_settings.tsManually tested using
DATE_FORMAT("yyyy-MM-dd HH:mm z", @timestamp)to confirm the timezone applied:dateFormat:tzadvanced setting ✅time_zoneparam): returnsZ(UTC) ✅SET time_zone = "America/New_York": returnsEST/EDT, overriding the advanced setting ✅Closes #4962
Supersedes #5456 (timezone content consolidated into this PR).
Related Kibana PR:
Generative AI disclosure
Tool(s) and model(s) used: claude-4.6-sonnet-medium in Cursor