Elasticsearch datetime format #28456

MosheEichler · 2023-07-24T19:26:18Z

Status

In Progress
Ready
In Hold - (Reason for hold)

Related Issues

fixes: link to the issue

Description

Handle string format on Elasticsearch.

Must have

Tests
Documentation

tkatzir

Looks good. Commented.

Packs/Elasticsearch/Integrations/Elasticsearch_v2/Elasticsearch_v2.py

Packs/Elasticsearch/Integrations/Elasticsearch_v2/Elasticsearch_v2.yml

ShirleyDenkberg · 2023-07-31T07:14:13Z

@tkatzir @bziser Doc review completed.

…ormat' into XSUP-26290/elastic_search_date_format

bziser

looks good

Packs/Elasticsearch/Integrations/Elasticsearch_v2/Elasticsearch_v2.py

tkatzir

Looks very good!
Some comments.

Packs/Elasticsearch/Integrations/Elasticsearch_v2/Elasticsearch_v2.py

Packs/Elasticsearch/ReleaseNotes/1_3_13.md

ShirleyDenkberg · 2023-08-07T06:07:48Z

@tkatzir @bziser Doc review completed.

tkatzir

Looks good!
Approved, with some minor comments.

tkatzir · 2023-08-09T07:03:24Z

Packs/Elasticsearch/Integrations/Elasticsearch_v2/Elasticsearch_v2.py

@@ -454,7 +491,7 @@ def test_func(proxies):

                else:
                    # if it is unknown error - get the message from the error itself
-                    return_error("Failed to connect. The following error occurred: {}".format(str(e)))
+                    return_error(f"Failed to connect. The following error occurred: {str(e)}")


Suggested change

return_error(f"Failed to connect. The following error occurred: {str(e)}")

return_error(f"Failed to connect. The following error occurred: {e}")

tkatzir · 2023-08-09T07:04:23Z

Packs/Elasticsearch/Integrations/Elasticsearch_v2/Elasticsearch_v2.py

@@ -717,7 +755,7 @@ def fetch_incidents(proxies):
            incidents, last_fetch = results_to_incidents_datetime(response, last_fetch or FETCH_TIME)
            demisto.setLastRun({'time': str(last_fetch)})

-        demisto.info('extract {} incidents'.format(len(incidents)))
+        demisto.info(f'extract {len(incidents)} incidents')


Suggested change

demisto.info(f'extract {len(incidents)} incidents')

demisto.info(f'extracted {len(incidents)} incidents')

tkatzir · 2023-08-09T07:04:46Z

Packs/Elasticsearch/Integrations/Elasticsearch_v2/Elasticsearch_v2.py

@@ -848,7 +886,7 @@ def main():
            return_error('Failed executing {}. Seems that the client does not support the server\'s distribution, '
                         'Please try using the Open Search client in the instance configuration.'
                         '\nError message: {}'.format(demisto.command(), str(e)), error=e)
-        return_error("Failed executing {}.\nError message: {}".format(demisto.command(), str(e)), error=e)
+        return_error(f"Failed executing {demisto.command()}.\nError message: {str(e)}", error=e)


Suggested change

return_error(f"Failed executing {demisto.command()}.\nError message: {str(e)}", error=e)

return_error(f"Failed executing {demisto.command()}.\nError message: {e}", error=e)

tkatzir · 2023-08-09T07:09:21Z

Packs/Elasticsearch/Integrations/Elasticsearch_v2/Elasticsearch_v2_test.py

+    ('YYYY-MM-DD HH:mm:ss', 'YYYY-MM-DD HH:mm:ss'),
+    ('yyyy-MM-dd HH:mm:ss', 'YYYY-MM-DD HH:mm:ss'),
+    ('yyyy-MM-DD HH:mm:ss', 'YYYY-MM-DD HH:mm:ss'),
+    ('YYYY-MM-dd HH:mm:ss', 'YYYY-MM-DD HH:mm:ss'),


You may need to import DEFAULT_DATETIME_FORMAT.

Suggested change

('YYYY-MM-DD HH:mm:ss', 'YYYY-MM-DD HH:mm:ss'),

('yyyy-MM-dd HH:mm:ss', 'YYYY-MM-DD HH:mm:ss'),

('yyyy-MM-DD HH:mm:ss', 'YYYY-MM-DD HH:mm:ss'),

('YYYY-MM-dd HH:mm:ss', 'YYYY-MM-DD HH:mm:ss'),

('YYYY-MM-DD HH:mm:ss', Elasticsearch_v2.DEFAULT_DATETIME_FORMAT),

('yyyy-MM-dd HH:mm:ss', Elasticsearch_v2.DEFAULT_DATETIME_FORMAT),

('yyyy-MM-DD HH:mm:ss', Elasticsearch_v2.DEFAULT_DATETIME_FORMAT),

('YYYY-MM-dd HH:mm:ss', Elasticsearch_v2.DEFAULT_DATETIME_FORMAT),

tkatzir · 2023-08-09T07:11:17Z

Packs/Elasticsearch/Integrations/Elasticsearch_v2/Elasticsearch_v2_test.py

+    ('123456', 'Simple-Date', 'YYYY-MM-DD HH:mm:ss', 123456),
+    (dateparser.parse('July 1, 2023'), 'Simple-Date', 'YYYY-MM-DD', '2023-07-01'),
+    (dateparser.parse('July 1, 2023'), 'Simple-Date', 'YYYY-MM-DD HH:mm:ss', '2023-07-01 00:00:00'),


Suggested change

('123456', 'Simple-Date', 'YYYY-MM-DD HH:mm:ss', 123456),

(dateparser.parse('July 1, 2023'), 'Simple-Date', 'YYYY-MM-DD', '2023-07-01'),

(dateparser.parse('July 1, 2023'), 'Simple-Date', 'YYYY-MM-DD HH:mm:ss', '2023-07-01 00:00:00'),

('123456', 'Simple-Date', Elasticsearch_v2.DEFAULT_DATETIME_FORMAT 123456),

(dateparser.parse('July 1, 2023'), 'Simple-Date', 'YYYY-MM-DD', '2023-07-01'),

(dateparser.parse('July 1, 2023'), 'Simple-Date', Elasticsearch_v2.DEFAULT_DATETIME_FORMAT, '2023-07-01 00:00:00'),

* Elasticsearch datetime format * formats * formats * time format * UT * fix UT * fix ut * RN * docker * pre-commit fixes * cr fixes * description * docs fixes * typo * changes to ISO format * fix UT * pre-commit fixes * get format filed type * pre-commit fixes * get datetime filed format * add UT * fix UT * docs * fixes * cr fixes * rn * improved the imports * CR fixes * build fixes * cr fixes * date time * CR fixes * cr fixes * docker

MosheEichler added 2 commits July 24, 2023 22:25

Elasticsearch datetime format

548ff0f

Merge branch 'master' into XSUP-26290/elastic_search_date_format

2149e45

MosheEichler self-assigned this Jul 24, 2023

MosheEichler added 10 commits July 30, 2023 13:13

formats

9dfc24e

formats

43551fc

time format

1b633d7

UT

f6c34d8

fix UT

e2a14d6

fix ut

39a3a5b

Merge branch 'master' into XSUP-26290/elastic_search_date_format

eb44b7a

RN

c61a9e5

docker

58d4362

pre-commit fixes

aad7fba

MosheEichler assigned ShirleyDenkberg Jul 31, 2023

MosheEichler requested review from tkatzir and bziser July 31, 2023 06:48

tkatzir reviewed Jul 31, 2023

View reviewed changes

Packs/Elasticsearch/Integrations/Elasticsearch_v2/Elasticsearch_v2.py Outdated Show resolved Hide resolved

Packs/Elasticsearch/Integrations/Elasticsearch_v2/Elasticsearch_v2.py Outdated Show resolved Hide resolved

MosheEichler and others added 3 commits July 31, 2023 09:56

cr fixes

9cfbc97

Merge branch 'master' into XSUP-26290/elastic_search_date_format

0384c36

description

5f1e012

ShirleyDenkberg reviewed Jul 31, 2023

View reviewed changes

Packs/Elasticsearch/Integrations/Elasticsearch_v2/Elasticsearch_v2.yml Outdated Show resolved Hide resolved

ShirleyDenkberg added the docs-approved label Jul 31, 2023

MosheEichler and others added 7 commits July 31, 2023 10:18

Merge branch 'master' into XSUP-26290/elastic_search_date_format

9afb501

Merge remote-tracking branch 'origin/XSUP-26290/elastic_search_date_f…

bf8c26a

…ormat' into XSUP-26290/elastic_search_date_format

docs fixes

7653f9e

typo

dad2942

changes to ISO format

1693391

Merge branch 'master' into XSUP-26290/elastic_search_date_format

2d984e5

fix UT

c0bc95a

MosheEichler added 3 commits August 6, 2023 16:15

Merge branch 'master' into XSUP-26290/elastic_search_date_format

0f4deeb

cr fixes

eb122e3

rn

4ab2dfe

MosheEichler requested a review from tkatzir August 6, 2023 13:30

bziser reviewed Aug 6, 2023

View reviewed changes

MosheEichler added 3 commits August 6, 2023 16:32

improved the imports

9acb794

CR fixes

6e311e5

Merge branch 'master' into XSUP-26290/elastic_search_date_format

fddd44e

tkatzir reviewed Aug 6, 2023

View reviewed changes

MosheEichler added 3 commits August 6, 2023 18:25

build fixes

14a5fbd

cr fixes

e73ae8f

date time

38ab4e9

MosheEichler requested review from tkatzir, bziser and ShirleyDenkberg August 6, 2023 15:39

MosheEichler added 2 commits August 6, 2023 22:57

CR fixes

3db11d5

Merge branch 'master' into XSUP-26290/elastic_search_date_format

c66bc11

ShirleyDenkberg reviewed Aug 7, 2023

View reviewed changes

Packs/Elasticsearch/ReleaseNotes/1_3_13.md Show resolved Hide resolved

Merge branch 'master' into XSUP-26290/elastic_search_date_format

c843f11

tkatzir approved these changes Aug 9, 2023

View reviewed changes

bziser approved these changes Aug 9, 2023

View reviewed changes

MosheEichler and others added 4 commits August 9, 2023 13:40

cr fixes

0290f75

Merge branch 'master' into XSUP-26290/elastic_search_date_format

79a4ae4

docker

5e69ca4

Merge branch 'master' into XSUP-26290/elastic_search_date_format

654716e

MosheEichler merged commit 47ec79e into master Aug 9, 2023
13 of 14 checks passed

MosheEichler deleted the XSUP-26290/elastic_search_date_format branch August 9, 2023 12:51

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Elasticsearch datetime format #28456

Elasticsearch datetime format #28456

MosheEichler commented Jul 24, 2023

tkatzir left a comment

ShirleyDenkberg commented Jul 31, 2023

bziser left a comment

tkatzir left a comment

ShirleyDenkberg commented Aug 7, 2023

tkatzir left a comment

tkatzir Aug 9, 2023

tkatzir Aug 9, 2023

tkatzir Aug 9, 2023

tkatzir Aug 9, 2023

tkatzir Aug 9, 2023

	return_error(f"Failed to connect. The following error occurred: {str(e)}")
	return_error(f"Failed to connect. The following error occurred: {e}")

	demisto.info(f'extract {len(incidents)} incidents')
	demisto.info(f'extracted {len(incidents)} incidents')

	return_error(f"Failed executing {demisto.command()}.\nError message: {str(e)}", error=e)
	return_error(f"Failed executing {demisto.command()}.\nError message: {e}", error=e)

Elasticsearch datetime format #28456

Elasticsearch datetime format #28456

Conversation

MosheEichler commented Jul 24, 2023

Status

Related Issues

Description

Must have

tkatzir left a comment

Choose a reason for hiding this comment

ShirleyDenkberg commented Jul 31, 2023

bziser left a comment

Choose a reason for hiding this comment

tkatzir left a comment

Choose a reason for hiding this comment

ShirleyDenkberg commented Aug 7, 2023

tkatzir left a comment

Choose a reason for hiding this comment

tkatzir Aug 9, 2023

Choose a reason for hiding this comment

tkatzir Aug 9, 2023

Choose a reason for hiding this comment

tkatzir Aug 9, 2023

Choose a reason for hiding this comment

tkatzir Aug 9, 2023

Choose a reason for hiding this comment

tkatzir Aug 9, 2023

Choose a reason for hiding this comment