Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Discover] Fix field stats for epoch time format #148288

Merged
merged 7 commits into from Jan 5, 2023
Merged

[Discover] Fix field stats for epoch time format #148288

merged 7 commits into from Jan 5, 2023

Conversation

jughosta
Copy link
Contributor

@jughosta jughosta commented Jan 3, 2023 •
edited by kibanamachine

Closes #148140

Summary

This PR fixes time format so it can work also for epoch.

Before:
Screenshot 2023-01-03 at 12 33 17

After:
Screenshot 2023-01-03 at 12 32 49

Sample data for testing:

PUT test
PUT test/_mapping
{
    "properties": {
        "timestamp": {
            "type": "date",
            "format": "epoch_second"
        },
        "message":  {
          "type": "keyword"
        },
        "bytes": {
          "type": "long"
        }
    }
}
    
POST test/_doc/1
{
    "timestamp": 1669912088.9230318,
    "message": "Message 1",
    "bytes": 250
}
POST test/_doc/2
{
    "timestamp": 1669912088.9230319,
    "message": "Message 2",
    "bytes": 20
}

@jughosta jughosta added release_note:fix Team:DataDiscovery Discover App Team (Document Explorer, Saved Search, Surrounding documents, Graph) backport:prev-minor Backport to the previous minor version (i.e. one version back from main) Feature:UnifiedFieldList The unified field list component used by Lens & Discover labels Jan 3, 2023
@jughosta jughosta self-assigned this Jan 3, 2023
@jughosta jughosta marked this pull request as ready for review January 3, 2023 13:54
@jughosta jughosta requested a review from a team as a code owner January 3, 2023 13:54
@elasticmachine
Copy link
Contributor

Pinging @elastic/kibana-data-discovery (Team:DataDiscovery)

davismcphee
davismcphee reviewed Jan 5, 2023
View reviewed changes
Copy link
Contributor

@davismcphee davismcphee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pulled and tested locally and works as expected, but I think it would be good if we add a test for buildSearchParams to check the format, or even a simple snapshot test.

@jughosta
Copy link
Contributor Author

jughosta commented Jan 5, 2023

@davismcphee Added tests, thanks!

@kibana-ci
Copy link
Collaborator

💚 Build Succeeded

Metrics [docs]

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id before after diff
unifiedFieldList 48.3KB 48.4KB +70.0B

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @jughosta

davismcphee
davismcphee approved these changes Jan 5, 2023
View reviewed changes
Copy link
Contributor

@davismcphee davismcphee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the tests! Also the repro steps were super helpful for testing 👍 LGTM

@jughosta jughosta merged commit 1b3769c into elastic:main Jan 5, 2023
@jughosta jughosta deleted the 148140-fix-field-stats-for-epoch branch January 5, 2023 14:52
@kibanamachine kibanamachine mentioned this pull request Jan 5, 2023
Merged
kibanamachine pushed a commit to kibanamachine/kibana that referenced this pull request Jan 5, 2023
@jughosta
[Discover] Fix field stats for epoch time format (elastic#148288)
d780ffc 
Closes elastic#148140

## Summary

This PR fixes time format so it can work also for epoch.

Before:
<img width="400" alt="Screenshot 2023-01-03 at 12 33 17"
src="https://user-images.githubusercontent.com/1415710/210350243-b3ae1d1a-4c76-46e7-930f-0ca9835dc121.png">

After:
<img width="400" alt="Screenshot 2023-01-03 at 12 32 49"
src="https://user-images.githubusercontent.com/1415710/210350274-af80f558-6a32-4da6-9cb9-857a8cb3de70.png">

Sample data for testing:
```
PUT test
PUT test/_mapping
{
    "properties": {
        "timestamp": {
            "type": "date",
            "format": "epoch_second"
        },
        "message":  {
          "type": "keyword"
        },
        "bytes": {
          "type": "long"
        }
    }
}

POST test/_doc/1
{
    "timestamp": 1669912088.9230318,
    "message": "Message 1",
    "bytes": 250
}
POST test/_doc/2
{
    "timestamp": 1669912088.9230319,
    "message": "Message 2",
    "bytes": 20
}
```

(cherry picked from commit 1b3769c)
@kibanamachine
Copy link
Contributor

💚 All backports created successfully

Status Branch Result
✅ 8.6

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

kibanamachine added a commit that referenced this pull request Jan 5, 2023
@kibanamachine @jughosta
[8.6] [Discover] Fix field stats for epoch time format (#148288) (#14…
57f91bf 
…8457)

# Backport

This will backport the following commits from `main` to `8.6`:
- [[Discover] Fix field stats for epoch time format
(#148288)](#148288)

<!--- Backport version: 8.9.7 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Julia
Rechkunova","email":"julia.rechkunova@elastic.co"},"sourceCommit":{"committedDate":"2023-01-05T14:52:29Z","message":"[Discover]
Fix field stats for epoch time format (#148288)\n\nCloses
#148140
Summary\r\n\r\nThis PR fixes time format so it can work also for
epoch.\r\n\r\nBefore:\r\n<img width=\"400\" alt=\"Screenshot 2023-01-03
at 12 33
17\"\r\nsrc=\"https://user-images.githubusercontent.com/1415710/210350243-b3ae1d1a-4c76-46e7-930f-0ca9835dc121.png\">\r\n\r\nAfter:\r\n<img
width=\"400\" alt=\"Screenshot 2023-01-03 at 12 32
49\"\r\nsrc=\"https://user-images.githubusercontent.com/1415710/210350274-af80f558-6a32-4da6-9cb9-857a8cb3de70.png\">\r\n\r\nSample
data for testing:\r\n```\r\nPUT test\r\nPUT test/_mapping\r\n{\r\n
\"properties\": {\r\n \"timestamp\": {\r\n \"type\": \"date\",\r\n
\"format\": \"epoch_second\"\r\n },\r\n \"message\": {\r\n \"type\":
\"keyword\"\r\n },\r\n \"bytes\": {\r\n \"type\": \"long\"\r\n }\r\n
}\r\n}\r\n \r\nPOST test/_doc/1\r\n{\r\n \"timestamp\":
1669912088.9230318,\r\n \"message\": \"Message 1\",\r\n \"bytes\":
250\r\n}\r\nPOST test/_doc/2\r\n{\r\n \"timestamp\":
1669912088.9230319,\r\n \"message\": \"Message 2\",\r\n \"bytes\":
20\r\n}\r\n```","sha":"1b3769c86b683a3e1762f23b6f503479957044c6","branchLabelMapping":{"^v8.7.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:fix","Team:DataDiscovery","backport:prev-minor","Feature:UnifiedFieldList","v8.7.0"],"number":148288,"url":"#148288
Fix field stats for epoch time format (#148288)\n\nCloses
#148140
Summary\r\n\r\nThis PR fixes time format so it can work also for
epoch.\r\n\r\nBefore:\r\n<img width=\"400\" alt=\"Screenshot 2023-01-03
at 12 33
17\"\r\nsrc=\"https://user-images.githubusercontent.com/1415710/210350243-b3ae1d1a-4c76-46e7-930f-0ca9835dc121.png\">\r\n\r\nAfter:\r\n<img
width=\"400\" alt=\"Screenshot 2023-01-03 at 12 32
49\"\r\nsrc=\"https://user-images.githubusercontent.com/1415710/210350274-af80f558-6a32-4da6-9cb9-857a8cb3de70.png\">\r\n\r\nSample
data for testing:\r\n```\r\nPUT test\r\nPUT test/_mapping\r\n{\r\n
\"properties\": {\r\n \"timestamp\": {\r\n \"type\": \"date\",\r\n
\"format\": \"epoch_second\"\r\n },\r\n \"message\": {\r\n \"type\":
\"keyword\"\r\n },\r\n \"bytes\": {\r\n \"type\": \"long\"\r\n }\r\n
}\r\n}\r\n \r\nPOST test/_doc/1\r\n{\r\n \"timestamp\":
1669912088.9230318,\r\n \"message\": \"Message 1\",\r\n \"bytes\":
250\r\n}\r\nPOST test/_doc/2\r\n{\r\n \"timestamp\":
1669912088.9230319,\r\n \"message\": \"Message 2\",\r\n \"bytes\":
20\r\n}\r\n```","sha":"1b3769c86b683a3e1762f23b6f503479957044c6"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"main","label":"v8.7.0","labelRegex":"^v8.7.0$","isSourceBranch":true,"state":"MERGED","url":"#148288
Fix field stats for epoch time format (#148288)\n\nCloses
#148140
Summary\r\n\r\nThis PR fixes time format so it can work also for
epoch.\r\n\r\nBefore:\r\n<img width=\"400\" alt=\"Screenshot 2023-01-03
at 12 33
17\"\r\nsrc=\"https://user-images.githubusercontent.com/1415710/210350243-b3ae1d1a-4c76-46e7-930f-0ca9835dc121.png\">\r\n\r\nAfter:\r\n<img
width=\"400\" alt=\"Screenshot 2023-01-03 at 12 32
49\"\r\nsrc=\"https://user-images.githubusercontent.com/1415710/210350274-af80f558-6a32-4da6-9cb9-857a8cb3de70.png\">\r\n\r\nSample
data for testing:\r\n```\r\nPUT test\r\nPUT test/_mapping\r\n{\r\n
\"properties\": {\r\n \"timestamp\": {\r\n \"type\": \"date\",\r\n
\"format\": \"epoch_second\"\r\n },\r\n \"message\": {\r\n \"type\":
\"keyword\"\r\n },\r\n \"bytes\": {\r\n \"type\": \"long\"\r\n }\r\n
}\r\n}\r\n \r\nPOST test/_doc/1\r\n{\r\n \"timestamp\":
1669912088.9230318,\r\n \"message\": \"Message 1\",\r\n \"bytes\":
250\r\n}\r\nPOST test/_doc/2\r\n{\r\n \"timestamp\":
1669912088.9230319,\r\n \"message\": \"Message 2\",\r\n \"bytes\":
20\r\n}\r\n```","sha":"1b3769c86b683a3e1762f23b6f503479957044c6"}}]}]
BACKPORT-->

Co-authored-by: Julia Rechkunova <julia.rechkunova@elastic.co>
@mistic mistic added v8.6.1 and removed v8.6.0 labels Jan 10, 2023
@mistic
Copy link
Member

mistic commented Jan 10, 2023

The backport was not included on v8.6.0 at it happened after the build candidate. Changing the label to v8.6.1.

@jughosta jughosta mentioned this pull request Jan 10, 2023
Closed
@davismcphee davismcphee mentioned this pull request Jan 12, 2023
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@davismcphee davismcphee davismcphee approved these changes

Assignees

@jughosta jughosta

Labels
backport:prev-minor Backport to the previous minor version (i.e. one version back from main) Feature:UnifiedFieldList The unified field list component used by Lens & Discover release_note:fix Team:DataDiscovery Discover App Team (Document Explorer, Saved Search, Surrounding documents, Graph) v8.6.1 v8.7.0
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[Discover] Field statistics request fails for epoch_second timestamp mappings
6 participants
@jughosta @elasticmachine @kibana-ci @kibanamachine @mistic @davismcphee