Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error converting yyyyDDD-formatted String to Timestamp #2499

francescopellegrini opened this issue Feb 28, 2019 · 0 comments


Copy link

commented Feb 28, 2019

STRINGTOTIMESTAMP has issues parsing yyyyDDD-formatted Strings.

See the following test:

package io.confluent.ksql.util.timestamp;

import io.confluent.ksql.GenericRow;
import io.confluent.ksql.util.KsqlConstants;
import org.apache.kafka.clients.consumer.ConsumerRecord;
import org.junit.Test;

import java.text.ParseException;
import java.text.SimpleDateFormat;
import java.util.Collections;
import java.util.HashMap;
import java.util.Map;

import static org.hamcrest.CoreMatchers.equalTo;
import static org.hamcrest.MatcherAssert.assertThat;

public class StringTimestampDayOfYearExtractorTest {

  private final StringTimestampExtractor timestampExtractor = new StringTimestampExtractor();
  private final String format = "yyyyDDD";

  public void shouldExtractTimestampFromStringWIthFormat() throws ParseException {
    final Map props = new HashMap() {{
      put(KsqlConstants.STRING_TIMESTAMP_FORMAT, format);
      put(KsqlConstants.KSQL_TIMESTAMP_COLUMN_INDEX, 0);
    final String stringTime = "2010321";
    final long expectedTime = new SimpleDateFormat(format).parse(stringTime).getTime();
    final long actualTime = timestampExtractor.extract(new ConsumerRecord("topic",
        new GenericRow(Collections.singletonList(stringTime))), 1);
    assertThat(actualTime, equalTo(expectedTime));

StringToTimestampParser creates a DateTimeFormatterBuilder instance and sets some default values for parsing:

new DateTimeFormatterBuilder()
        .parseDefaulting(ChronoField.YEAR_OF_ERA, 1970)
        .parseDefaulting(ChronoField.MONTH_OF_YEAR, 1)
        .parseDefaulting(ChronoField.DAY_OF_MONTH, 1)
        .parseDefaulting(ChronoField.HOUR_OF_DAY, 0)

The default values clash with the parsed ones, resulting in an error (and thus a null value in KSQL).

A DateTimeFormatterBuilder with no default set actually works, but I'm not sure this change has no side effect...

What can I do?

@agavra agavra referenced this issue Mar 5, 2019
0 of 2 tasks complete

@agavra agavra closed this in 0931cad Mar 13, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
None yet
1 participant
You can’t perform that action at this time.