Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Standardization ignores timezone metadata for timestamp type in case the source is already timestamp #1836

Closed
benedeki opened this issue Jun 30, 2021 · 1 comment · Fixed by #1841
Closed

Standardization ignores timezone metadata for timestamp type in case the source is already timestamp #1836

benedeki opened this issue Jun 30, 2021 · 1 comment · Fixed by #1841
Assignees
@benedeki
Labels
bug Something isn't working priority: high Critical to the health of the project Standardization Standardization Job affected

Comments

@benedeki
Copy link
Collaborator

Describe the bug

When the input of Standardization is strongly type (e.g. Parquet) the metadata of timezone for timestamp type is ignored in the case the input is already of timestamp type.
This causes extra issues in case when defaultTimestampTimeZone (other as UTC) is defined in application configuration, because that is take into account.

To Reproduce

  1. Configure defaultTimestampTimeZone in application.conf to a non-UTC value
  2. Have a source data in strongly type format (parquet) containing a field of type timestamp
  3. Create a schema and dataset for the source data, add metadata timestamp with value UTC to the timestamp field
  4. Run the standardization of the dataset
  5. Check the data
  6. The timestamp field got shifted by the difference between defaultTimestampTimeZone and UTC

Expected behaviour

timezone metadata are taken into account. The timestamp from the To reproduce paragraph is not shifted.
@benedeki benedeki added bug Something isn't working Standardization Standardization Job affected priority: high Critical to the health of the project labels Jun 30, 2021
@benedeki benedeki self-assigned this Jun 30, 2021
benedeki added a commit that referenced this issue Jul 1, 2021
@benedeki
#1836: Standardization ignores timezone metadata for timestamp type i…
8a0d71f 
…n case the source is already timestamp

* Timezone in metadata is taken into account even if the source is already timestamp or date
benedeki added a commit that referenced this issue Jul 1, 2021
@benedeki
#1836: Standardization ignores timezone metadata for timestamp type i…
f718c47 
…n case the source is already timestamp

* Timezone in metadata is taken into account even if the source is already timestamp or date
@dk1844 dk1844 linked a pull request Jul 9, 2021 that will close this issue
#1836: Standardization ignores timezone metadata for timestamp type in case the source is already timestamp #1841
Merged
benedeki added a commit that referenced this issue Jul 10, 2021
@benedeki
#1836: When saving a dataset with undefined recommended property show…
5cf6553 
… a warning dialog

* Show dialogs only on submit not in case of validation on the fly
* Show property warning only if all Mandatory has passed validation
@dk1844 dk1844 added PR:tested Only for PR - PR was tested by a tester (person) and removed PR:tested Only for PR - PR was tested by a tester (person) labels Jul 12, 2021
@benedeki
Copy link
Collaborator Author

Release notes:
Time zone in metadata is taken into account even if the source data is already of timestamp type.

benedeki added a commit that referenced this issue Jul 12, 2021
@benedeki
#1836: Standardization ignores timezone metadata for timestamp type i…
a81c00c 
…n case the source is already timestamp (#1841)

#1836: Standardization ignores timezone metadata for timestamp type in case the source is already timestamp
* Timezone in metadata is taken into account even if the source is already timestamp or date
* Added explanatory comment to the test
benedeki added a commit that referenced this issue Jul 12, 2021
@benedeki
#1836: Standardization ignores timezone metadata for timestamp type i…
768ce11 
…n case the source is already timestamp (2.21) (#1840)

#1836: Standardization ignores timezone metadata for timestamp type in case the source is already timestamp
* Timezone in metadata is taken into account even if the source is already timestamp or date
* Added explanatory comment to the test
dk1844 added a commit that referenced this issue Aug 26, 2021
@dk1844
#1836 treeTable binding problem
6418109
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@benedeki benedeki

Labels
bug Something isn't working priority: high Critical to the health of the project Standardization Standardization Job affected
Projects
None yet
Milestone
No milestone
2 participants
@dk1844 @benedeki