[minor] /tableConfigs/validate can't convert default timestamp null value to LONG

[dd-willgan]

Calling /tableConfigs/validate will convert a timestamp field to the following

"dateTimeFieldSpecs": [
	{
		"format": "1:MILLISECONDS:EPOCH",
		"sampleValue": null,
		"granularity": "1:MILLISECONDS",
		"singleValueField": true,
		"name": "created_at_epoch_ms",
		"maxLength": 512,
		"dataType": "LONG",
		"transformFunction": null,
		"defaultNullValue": -9223372036854775808,
		"virtualColumnProvider": null,
		"defaultNullValueString": "-9223372036854775808"
	}
]

However, if this config is fed to /tableConfigs/validate again, the following error will be thrown

exception: Cannot convert value: \'-9223372036854776000\' to type: LONG

[Jackie-Jiang]

Seems like a problem of JSON precision loss, where -9223372036854775808 is read as double thus converted to -9223372036854776000.
Do you send the exactly same JSON to the validate API?

[dd-willgan]

Thanks @Jackie-Jiang that makes sense. The JSON sent to validate API originally is

{
  "format": "1:MILLISECONDS:EPOCH",
  "granularity": "1:MILLISECONDS",
  "name": "created_at_epoch_ms",
  "dataType": "LONG"
}

[Jackie-Jiang]

Did some research and find this as a known issue in Swagger and there is no plan to fix it: swagger-api/swagger-ui#2030
If you use Swagger to query Pinot, the result might also lose precision. If you try with command line you should be able to see the correct values returned

[Jackie-Jiang]

To add more context, then validating this field:

{
  "format": "1:MILLISECONDS:EPOCH",
  "granularity": "1:MILLISECONDS",
  "name": "ts",
  "dataType": "LONG"
}

Swagger will show the response as:

{
  "format": "1:MILLISECONDS:EPOCH",
  "granularity": "1:MILLISECONDS",
  "sampleValue": null,
  "singleValueField": true,
  "notNull": false,
  "dataType": "LONG",
  "maxLength": 512,
  "virtualColumnProvider": null,
  "defaultNullValue": -9223372036854776000,
  "defaultNullValueString": "-9223372036854775808",
  "transformFunction": null,
  "name": "ts"
}

Note that the precision loss already happens here.

I tried using the following json as input and it works without throwing exception, but of course the response still lose precision:

{
  "format": "1:MILLISECONDS:EPOCH",
  "granularity": "1:MILLISECONDS",
  "dataType": "LONG",
  "defaultNullValue": -9223372036854775808
  "name": "ts"
}