Error parsing NULL timestamp value in non scalar measure value

[rajeiyer]

What happened:
In a table with both scalar and non-scalar (multi measure) measure values, query select DemoMetricMultiMeasure6 from "AtlasPDXDevTimestreamDatabase"."dev-mes-multi-measure-test" where measure_name = 'DemoMetric11' succeeds but query select DemoMetricMultiMeasure6 from "AtlasPDXDevTimestreamDatabase"."dev-mes-multi-measure-test" fails with exception

logger=plugin.grafana-timestream-datasource t=2022-04-15T21:18:31.05+0000 lvl=dbug msg="panic: runtime error: invalid memory address or nil pointer dereference"
logger=plugin.grafana-timestream-datasource t=2022-04-15T21:18:31.05+0000 lvl=dbug msg="[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0xc032fe]"
logger=plugin.grafana-timestream-datasource t=2022-04-15T21:18:31.05+0000 lvl=dbug msg=
logger=plugin.grafana-timestream-datasource t=2022-04-15T21:18:31.05+0000 lvl=dbug msg="goroutine 53 [running]:"
logger=plugin.grafana-timestream-datasource t=2022-04-15T21:18:31.05+0000 lvl=dbug msg=github.com/grafana/timestream-datasource/pkg/timestream.datumParserTimestamp(0x19?)
logger=plugin.grafana-timestream-datasource t=2022-04-15T21:18:31.05+0000 lvl=dbug msg="\t/drone/src/pkg/timestream/parsers.go:214 +0x1e"
logger=plugin.grafana-timestream-datasource t=2022-04-15T21:18:31.05+0000 lvl=dbug msg=github.com/grafana/timestream-datasource/pkg/timestream.QueryResultToDataFrame(0xc000153e00)
logger=plugin.grafana-timestream-datasource t=2022-04-15T21:18:31.05+0000 lvl=dbug msg="\t/drone/src/pkg/timestream/helper.go:92 +0xb5e"
logger=plugin.grafana-timestream-datasource t=2022-04-15T21:18:31.05+0000 lvl=dbug msg="github.com/grafana/timestream-datasource/pkg/timestream.ExecuteQuery({_, _}, {{0xc0002960f0, 0x22}, {0x0, 0x0}, {0xc000038260, 0x1f}, {0xc000038280, 0x1c}, ...}, ...)"

Measure name DemoMetric11 stores non-scalar values where DemoMetricMultiMeasure6 is a timestamp field in the non-scalar measure value. There are other measures in the same table with scalar values too. So the second query (without measure name predicate) will return null values for DemoMetricMultiMeasure6.

What you expected to happen:
Query should not fail with an exception

How to reproduce it (as minimally and precisely as possible):

Screenshots

Anything else we need to know?:

Environment:

• Grafana version:
• Plugin version: Current
• OS Grafana is installed on:
• User OS & Browser:
• Others:

[yaelleC]

Hi @rajeiyer , thanks for raising the issue.

I'm working on reproducing it, would you be able to confirm that you are using Timestream v1.5.1 and tell me which version of Grafana you are running?
Thanks

[yaelleC]

For future reference, I managed to reproduce with this query:

SELECT measure_name ,
CASE WHEN measure_name = 'task_completed' THEN (SELECT NULL) ELSE time END
FROM $__database.$__table 
LIMIT 10

I also pinpointed the issue to https://github.com/grafana/timestream-datasource/blob/main/pkg/timestream/parsers.go#L211-L215

which should be modified as follows :

func datumParserTimestamp(datum *timestreamquery.Datum) (interface{}, error) {
	if datum.ScalarValue == nil {
		return nil, nil
	}
	return time.Parse("2006-01-02 15:04:05.99999999", *datum.ScalarValue)
}

I have tested the fix and confirms that it works, I am working on adding tests and will submit a PR for it

[yaelleC]

@rajeiyer the fix is included in the release 1.5.2 of the plugin if you upgrade you should not have the issue anymore 😃 .

[rajeiyer]

Thank you for the quick turnaround on this issue. Appreciate your help.

[rajeiyer]

The customer that ran into this issue downloaded the latest and verified the fix. Thanks again.