Should we drop trailing zeros on timestamps?

[beckettsean]

Let me preface by saying this is less of an actual problem with the way the data is returned and more of a problem of perception. I know I was confused by this behavior, but not necessarily inconvenienced by it. I don't really have a strong opinion how we address it, but I would like to have a well-reasoned answer for those, like me, who find it initially jarring.

When returning the timestamp, we drop any trailing zeros. That obscures the precision with which the points were written. For example:

> INSERT cpu,host=serverB,region=us-west value=20.1
> INSERT cpu,host=serverB,region=us-west value=10.2
> INSERT cpu,host=serverB,region=us-west value=-10.2
> INSERT cpu,host=serverB,region=us-west value=-10.0
> select * from cpu
name: cpu
tags: host=serverB, region=us-west
time                value
----                -----
2015-06-12T22:26:55.443124385Z  20.1
2015-06-12T22:27:00.595680237Z  10.2
2015-06-12T22:27:04.02801Z  -10.2
2015-06-12T22:27:10.676721429Z  -10.0

(I verified that JSON and CSV output formats also drop the trailing zeros.)

Note the third returned point has a timestamp of 2015-06-12T22:27:04.02801Z, which is accurate to the tens of microseconds. I was writing points without a timestamp and without a precision, so I know they were stored at nanosecond precision. That is also demonstrated by the other timestamps, which are precise to the nanosecond.

When querying my data I cannot tell from the return whether I wrote that point at nanosecond precision 2015-06-12T22:27:04.028010000Z or 10 microsecond precision 2015-06-12T22:27:04.02801Z.

In discussion with @gunnaraasen he proposed a possible fix, although it would perhaps break the RFC3339 convention.

By default, return full nanosecond precision, e.g. 2015-06-12T22:27:04.028010000Z.

For those users that store at s, ms, or u precision that will mean a lot of trailing zeros, and meaningless ones at that. Although we store all timestamps to nanosecond precision, if the write accuracy is only in milliseconds the last six zeros have no significance.

To combat that we allow the user to specify the precision of the returned timestamps, and then note that precision by truncating the timestamp with the character. For example:

Precision in nanoseconds:
2015-06-12T22:27:04.028010000Z
Precision in microseconds:
2015-06-12T22:27:04.028010u
Precision in milliseconds:
2015-06-12T22:27:04.0280m
Precision in seconds:
2015-06-12T22:27:04s

Alternately we could provide the precision in the column headers, e.g.

> precision ns
> select * from cpu
precision: ns
name: cpu
tags: host=serverB, region=us-west
time                value
----                -----
2015-06-12T22:26:55.443124385Z  20.1
2015-06-12T22:27:00.595680237Z  10.2
2015-06-12T22:27:04.028010000Z  -10.2
2015-06-12T22:27:10.676721429Z  -10.0

> precision ms
> select * from cpu
precision: ms
name: cpu
tags: host=serverB, region=us-west
time                value
----                -----
2015-06-12T22:26:55.443Z    20.1
2015-06-12T22:27:00.595Z    10.2
2015-06-12T22:27:04.028Z    -10.2
2015-06-12T22:27:10.676Z    -10.0

[beckettsean]

All points are stored in nanos but we do not store the write precision separately. In order to prevent lots of trailing zeros for points written in microseconds, milliseconds, or seconds it's good to drop the trailing zeros.

This is something to document, but no development action to take at this time.