Timestamp format

[bengerman13]

URL for relevant page?

https://docs.influxdata.com/influxdb/v1.5/administration/backup_and_restore/

What products and version are you using?

Influxdb 1.5.2

Where did you look before opening the issue?

Issues here, https://tools.ietf.org/html/rfc3339#section-5.6

The docs for backup utility specify that the time specification should be RFC3339, but the backup does not accept some valid RFC3339 timestamps.
For example, the output from GNU date when passing --rfc-3339=seconds. I believe the docs should state that the utility requires ISO8601 timestamp.
running influxd backup -portable -start $(date --rfc-3339=seconds -d "12:00 AM yesterday") backup gives the error backup: parsing time "2018-06-24" as "2006-01-02T15:04:05Z07:00": cannot parse "" as "T"

From the RFC:

 NOTE: ISO 8601 defines date and time separated by "T".
 Applications using this syntax may choose, for the sake of
 readability, to specify a full-date and full-time separated by
 (say) a space character.

[stevebang]

@benbjohnson -- Is this a bug, or do we only support the ISO8601 timestamp?

[benbjohnson]

@stevebang My understanding is that RFC3339 is a subset of ISO 8601 (https://stackoverflow.com/a/522281). We use time.RFC3339 from the Go time package (https://golang.org/pkg/time/).

[stevebang]

@benbjohnson -- Is @bengerman13 incorrect that "the backup does not accept some valid RFC3339 timestamps"?

[bengerman13]

The stackoverflow post from @benbjohnson states (as the RFC does):

The RFC also allows the "T" to be replaced by a space (or other character), while the standard only allows it to be omitted (and only when there is agreement between all parties using the representation).

The difference between date --rfc-3339=seconds and date --iso-8601=seconds is exactly what that describes:

$ date --iso-8601=seconds
2018-06-27T09:30:53-07:00
$ date --rfc-3339=seconds
2018-06-27 09:31:02-07:00

and the error message from the backup utility seems pretty clear that it's unhappy about the space not being a 'T'

This is testable using the official InfluxDB docker container:

$ docker run --rm influxdb bash -c 'influxd backup -since `date --rfc-3339=seconds`'
backup: parsing time "2018-06-27" as "2006-01-02T15:04:05Z07:00": cannot parse "" as "T"

[benbjohnson]

@stevebang Yeah, go ahead and change it to ISO 8601

[stevebang]

After carefully reviewing the RFC3339 spec, others sources (see below), and discussing with others, the RFC 3339 format used by InfluxDB is fully RFC3339-compliant.

Note the following:

• Section 5.6 Internet Date/Time Format The only valid format, as indicated in the ABNF syntax, is date-time = full-date "T" full-time. The spec does not show an alternative format that includes a space.
• Section 5.8 Examples: All of the examples use the required "T" separator. There are no examples with a space replacing the "T", as allowed in ISO 8601.
• [Appendix A. ISO 8601 Collected ABNF] This appendix explains a "formal grammar from ISO 8601" and notes that "due to ambiguities in ISO 8601, some interpretations had to be made. ... ISO 8601 states that the "T" may be omitted under some circumstances. This grammar requires the "T" to avoid ambiguity." Note that "This grammar" refers to the RFC 3339 spec, and it is only the ISO 8601 specification which allows the ambiguity of using the "T" or a space.
• The package time in the Go language correctly shows the following constant as used by InfluxDB for time.RFC3339: RFC3339 = "2006-01-02T15:04:05Z07:00"
• The W3C FEED Validator notes that for a valid RFC 3339 date-time that "an uppercase "T" character MUST be used to separate date and time, and an uppercase "Z" character MUST be present in the absence of a numeric time zone offset" and includes three examples with the required "T" separator.
• The GNU coreutils incorrectly interprets the RFC 3339 specification.

  • See the Golang issue
    time: no Parse format for GNU style RFC3339 #25937.

  • See the GNU Coreutils discussion bug#6132: bug in date: --rfc-3339=TIMESPEC option doesn't print 'T', which includes a reply from one of the RFC 3339 authors, and about the "T" separator, Mr. Klyne replied:

In short: "yes" 

Per section 5.5, the intent in this draft was to specify a timestamp format 
using elements from and compatible with 8601, but eliminating as far as 
reasonable any variations that could make timestamp data harder to process. 
This includes making the 'T' mandatory in date+time values.

Given that GNU was made aware of this issue as early as 2006 and has not addressed it, the the GNU date commands return invalid results for the --rfc-3339 option.

I'm closing this issue since InfluxDB handles the RFC 3339 timestamps correctly, using time.RFC3339 from the Go time package.