Skip to content

AVRO-3078: Add local-timestamp-millis and local-timestamp-micros logical types to C##1628

Merged
martin-g merged 4 commits intoapache:masterfrom
zcsizmadia:avro-3078-implement-local-timestamp-millis
May 3, 2022
Merged

AVRO-3078: Add local-timestamp-millis and local-timestamp-micros logical types to C##1628
martin-g merged 4 commits intoapache:masterfrom
zcsizmadia:avro-3078-implement-local-timestamp-millis

Conversation

@zcsizmadia
Copy link
Contributor

Make sure you have checked all steps below.

Jira

Tests

  • My PR adds the following unit tests OR does not need testing for this extremely good reason:

Commits

  • My commits all reference Jira issues in their subject lines. In addition, my commits follow the guidelines from "How to write a good git commit message":
    1. Subject is separated from body by a blank line
    2. Subject is limited to 50 characters (not including Jira issue reference)
    3. Subject does not end with a period
    4. Subject uses the imperative mood ("add", not "adding")
    5. Body wraps at 72 characters
    6. Body explains "what" and "why", not "how"

Documentation

  • In case of new functionality, my PR adds documentation that describes how to use it.
    • All the public functions and the classes in the PR contain Javadoc that explain what it does

@github-actions github-actions bot added the C# label Mar 31, 2022
@martin-g martin-g merged commit c2bd724 into apache:master May 3, 2022
martin-g pushed a commit that referenced this pull request May 3, 2022
…cal types to C# (#1628)

* Add local-timestamp-millis and micros

* Add more UTC based local-timestamp tests

* Fix whitespace

Co-authored-by: Zoltan Csizmadia <CsizmadiaZ@valassis.com>
(cherry picked from commit c2bd724)
@martin-g
Copy link
Member

martin-g commented May 3, 2022

Thank you, @zcsizmadia !

@zcsizmadia zcsizmadia deleted the avro-3078-implement-local-timestamp-millis branch May 3, 2022 21:42
Comment on lines +57 to +59
public override object ConvertToLogicalValue(object baseValue, LogicalSchema schema)
{
return UnixEpochDateTime.AddMilliseconds((long)baseValue).ToLocalTime();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it correct to convert from UTC to local time here? From https://avro.apache.org/docs/1.11.0/spec.html#Local+timestamp+(millisecond+precision), I get the impression that 0 should mean 1 January 1970 00:00:00.000 local time, rather than 1 January 1970 00:00:00.000 UTC converted to local time. I'd really expect the result to have DateTimeKind.Unspecified because it's local time but not necessarily local to the computer in which this library is run.

In the Java implementation, LocalTimestampMillisConversion.fromLong first converts 0 to 1 January 1970 00:00:00.000 UTC and then drops the UTC time zone information, leaving 1 January 1970 00:00:00.000 without a time zone.

public LocalDateTime fromLong(Long millisFromEpoch, Schema schema, LogicalType type) {
Instant instant = timestampMillisConversion.fromLong(millisFromEpoch, schema, type);
return LocalDateTime.ofInstant(instant, ZoneOffset.UTC);
}

Related test:

@Test
public void testReadLocalTimestampMillis() throws IOException {
LogicalType timestamp = LogicalTypes.localTimestampMillis();
Schema longSchema = Schema.create(Schema.Type.LONG);
Schema timestampSchema = timestamp.addToSchema(Schema.create(Schema.Type.LONG));
LocalDateTime i1 = LocalDateTime.of(1986, 06, 26, 12, 07, 11, 42000000);
LocalDateTime i2 = LocalDateTime.ofInstant(Instant.ofEpochMilli(0), ZoneOffset.UTC);
List<LocalDateTime> expected = Arrays.asList(i1, i2);
Conversion<LocalDateTime> conversion = new TimeConversions.LocalTimestampMillisConversion();
// use the conversion directly instead of relying on the write side
Long i1long = conversion.toLong(i1, longSchema, timestamp);
Long i2long = 0L;
File test = write(longSchema, i1long, i2long);
Assert.assertEquals("Should convert long to LocalDateTime", expected,
read(GENERIC.createDatumReader(timestampSchema), test));
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants