fix: ensure datetime uses nano seconds

[YuriZmytrakov]

Description:

Change datetime field type from date to date_nanos to ensure millisecond precision in datetime filter instead of truncation, for example 024999Z becomes 024Z. Also enforce 1970-2262 date boundaries required by the date_nanos resolution.

PR Checklist:

• Code is formatted and linted (run pre-commit run --all-files)
• Tests pass (run make test)
• Documentation has been updated to reflect changes, if applicable
• Changes are added to the changelog

[jonhealy1]

This looks really good but I think this should be an optional feature. Can we enable it with an env var? Users could theoretically have data that precedes 1970 I think. We could even turn on nanosecond support by default but give users the option to use the old system, if they choose.

[YuriZmytrakov]

This looks really good but I think this should be an optional feature. Can we enable it with an env var? Users could theoretically have data that precedes 1970 I think. We could even turn on nanosecond support by default but give users the option to use the old system, if they choose.

I added the USE_DATETIME_NANOS environment variable, which enables nanosecond precision. When it’s disabled, the system falls back to microsecond precision.

[jonhealy1]

Nice work! Thanks