Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Checkpoint | support unix and unix_ms ts for time field #4781

Merged
merged 3 commits into from
Dec 14, 2022
Merged

Checkpoint | support unix and unix_ms ts for time field #4781

merged 3 commits into from
Dec 14, 2022

Conversation

kcreddy
Copy link
Contributor

@kcreddy kcreddy commented Dec 7, 2022 •
edited
Loading

What does this PR do?

Adds support for having checkpoint.time field as UNIX_MS.

Just adding UNIX_MS format to existing formats in date processor here doesn't work as there is no check done on DateProcessor whether the value is seconds or milliseconds. This makes the current date processor continues to match UNIX format even for millisecond epoch values.

Alternate approach to check the number of digits of checkpoint.time is taken to resolve this issue.

Checklist

  • I have reviewed tips for building integrations and this pull request is aligned with them.
  • I have verified that all data streams collect metrics or logs.
  • I have added an entry to my package's changelog.yml file.
  • I have verified that Kibana version constraints are current according to guidelines.

@kcreddy kcreddy self-assigned this Dec 7, 2022
@kcreddy kcreddy added bug Something isn't working, use only for issues Integration:checkpoint Check Point labels Dec 7, 2022
@kcreddy kcreddy changed the title support unix and unix_ms ts for time field Checkpoint | support unix and unix_ms ts for time field Dec 7, 2022
@elasticmachine
Copy link

elasticmachine commented Dec 7, 2022 •
edited
Loading

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview preview

Expand to view the summary

Build stats

  • Start Time: 2022-12-08T17:56:55.604+0000

  • Duration: 16 min 2 sec

Test stats 🧪

Test Results
Failed 0
Passed 12
Skipped 0
Total 12

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

  • /test : Re-trigger the build.

@elasticmachine
Copy link

elasticmachine commented Dec 7, 2022 •
edited
Loading

🌐 Coverage report

Name Metrics % (covered/total) Diff
Packages 100.0% (1/1) 💚
Files 100.0% (1/1) 💚
Classes 100.0% (1/1) 💚
Methods 100.0% (15/15) 💚 13.333
Lines 83.534% (695/832) 👍 29.397
Conditionals 100.0% (0/0) 💚

@kcreddy kcreddy marked this pull request as ready for review December 8, 2022 15:30
@kcreddy kcreddy requested a review from a team as a code owner December 8, 2022 15:31
@elasticmachine
Copy link

Pinging @elastic/security-external-integrations (Team:Security-External Integrations)

P1llus
P1llus approved these changes Dec 8, 2022
View reviewed changes
Copy link
Member

@P1llus P1llus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Once the question is answered or fixed then we can merge

packages/checkpoint/data_stream/firewall/elasticsearch/ingest_pipeline/default.yml Outdated
- convert:
field: checkpoint.time
type: long
ignore_failure: true
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we really want ignore failure here? Is there no other if condition we can use?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Certainly, let me change 👍🏼

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed the convert and date processors and written some painless to parse all 3 date formats. Let me know if that looks good.

@kcreddy kcreddy merged commit 64c4e2d into elastic:main Dec 14, 2022
@elasticmachine
Copy link

Package checkpoint - 1.9.1 containing this change is available at https://epr.elastic.co/search?package=checkpoint

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@P1llus P1llus P1llus approved these changes

Assignees

@kcreddy kcreddy

Labels
bug Something isn't working, use only for issues Integration:checkpoint Check Point
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@kcreddy @elasticmachine @P1llus