Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HIVE-27199: Read TIMESTAMP WITH LOCAL TIME ZONE columns from text files using custom formats #4170

Closed
wants to merge 5 commits into from

Commits on Mar 29, 2023

  1. Configuration menu
    Copy the full SHA
    737b926 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    4433bc9 View commit details
    Browse the repository at this point in the history
  3. Make the default TZL parse formatter accept more time delimiters and …

    …enrich tests
    
    Before the changes only space was allowed between date and time parts.
    2022-05-29 12:23:43
    
    After the changes following literals become valid (were not before)
    2022-05-29T 12:23:43
    2022-05-29T12:23:43
    2022-05-2912:23:43
    
    Regarding the TZL parser there still a few inconsistencies:
    
    Valid: 2022-05-29T 12:23:43
    Invalid: 2022-05-29 T12:23:43
    Invalid: 2022-05-29 T 12:23:43
    
    General observation is that TZL parser is much more lenient than the TZ parser; not sure we want that.
    zabetak committed Mar 29, 2023
    Configuration menu
    Copy the full SHA
    94aea94 View commit details
    Browse the repository at this point in the history
  4. Make TZL parser stricter allowing current format + strict ZoneDateTime

    The change fixes some inconsistencies which were introduced by the previous change:
    2016-05-03T 12:26:34 -> NULL
    2016-05-0312:26:34 -> NULL
    
    but introduces some new:
    
    # Case I
    Valid TZ: 2016-05-03T12:26:34
    Invalid TZL: 2016-05-03T12:26:34
    
    # Case II
    Valid TZL: 2016-05-03 12:26:34
    Invalid TZL: 2016-05-03T12:26:34
    zabetak committed Mar 29, 2023
    Configuration menu
    Copy the full SHA
    d2e9038 View commit details
    Browse the repository at this point in the history

Commits on Mar 30, 2023

  1. HIVE-27199: Read TIMESTAMP WITH LOCAL TIME ZONE columns from text fil…

    …es using custom formats
    
    1. Support parsing TimestampTZ using the TimestampParser, which accepts
    multiple DateTimeFormatters.
    2. Pass timestamp.formats in Lazy inspector handling
    TIMESTAMP WITH LOCAL TIME ZONE and instantiate a TimestampParser.
    3. Refactor TimestampTZUtil to allow passing different
    DateTimeFormatters.
    4. Add tests covering timestamps with 3 different formats (built-in,
    plus 2 more not covered by the default).
    zabetak committed Mar 30, 2023
    Configuration menu
    Copy the full SHA
    c155247 View commit details
    Browse the repository at this point in the history