feat: add timestamp_to_datetime_format_expr optional rule paramameter

[thpiron]

Description

No breaking changes.
The goal of this PR is to provide a way to transform a timestamp returned by Elasticsearch before parsing it into a datetime.
This come from an issue we have when using a custom timestamp field that can contains nanoseconds. For now, the parsing of the returned timestamp raise a Value error, as there is 3 additional digit that can't be parse in python (datetime.datetime.strptime format doesn't support nanoseconds). And I found no workaround (other that changing the timestamp format in Elasticsearch of course).
It seems logic that as we can choose a way to modify a timestamp before sending it to Elastisearch, we could also do the same when retrieving one.

So this PR simply add a rule parameter called timestamp_to_datetime_format_expr that does the same thing as timestamp_format_expr in the _ts_to_dt_with_format function instead of _dt_to_ts_with_format.
I didn't want to use the same rule parameter to avoid any breaking changes, and corner case I could not anticipate.

Checklist

• I have reviewed the contributing guidelines.
• I have included unit tests for my changes or additions.
• I have successfully run make test-docker with my changes.
• I have manually tested all relevant modes of the change in this PR.
• I have updated the documentation.
• I have updated the changelog.

Questions or Comments

Got some docker layer cache issues running make test-docker, will create another small PR to fix that.

[jertel]

Looks like a useful change. Would be nice to get the unit tests included with this PR though. What specific error are you seeing when running make test-docker?

[thpiron]

I don't have any error when running make test-docker, every test passes. But that's not surprising as no test case exists for custom timestamp yet. At least I didn't find any.

No really sure where and how I should create those tests.
If you think it's worth it maybe I could create some in tests/base_test.py by taking as example tests like test_some_hits_unix?

[jertel]

I suggest adding a new test to loaders_test.py and perhaps use test_kibana_discover_to_timedelta as a template.

[thpiron]

Yes it's a good example thanks! I'll add those tests asap.

[thpiron]

Added tests for timestamp_type custom, timestamp_to_datetime_format_expr and timestamp_format_expr.

[jertel]

Looks good! Thanks for back-filling those related tests.