GPS time unexpected isot representation

[zastruga]

Description

GPS time is correctly defined as 1980-01-06T00:00:19 TAI as its epoch. Many text formats using GPS time, such as RINEX messages, have defined the ISO or ISO-like text datetimes for GPS to be TAI - 19s (leap seconds at the 1980-01-06T00:00:00 UTC epoch). With GPS time being specified via the format argument and not having a GPS scale since it follows TAI, the isot representation is not as expected by these standard formats.

Expected behavior

In [1]: t = Time(0, format='gps')

In [2]: t.isot
Out[2]: '1980-01-06T00:00:00.000'

In [3]: t.tai.isot
Out[3]: '1980-01-06T00:00:19.000'

As below under actual behavior, I would not expect the t.isot and t.tai.isot values to be the same. I'm not sure if this is a similar issue to #11245 or not, but seems like it could be. Not sure if this would need effectively a GPS time scale in addition to the current TAI to make it work. For now, the only work around I've found is explicitly removing the 19 leap second offset for the .isot in a wrapper method. Am I missing something key or is this the only way to handle this at the moment, and should this functionality be added? Obviously using t.utc.isot is not the correct answer as that would add all of the leap seconds since the GPS epoch for current datetimes.

In [1]: (Time(0, format='gps') + TimeDelta(-19, format='sec')).isot
Out[1]: '1980-01-06T00:00:00.000'

Actual behavior

In [85]: t = Time(0, format='gps')

In [86]: t.isot
Out[86]: '1980-01-06T00:00:19.000'

In [87]: t.tai.isot
Out[87]: '1980-01-06T00:00:19.000'

In [88]: t.utc.isot
Out[88]: '1980-01-06T00:00:00.000'

In [89]: t.gps
Out[89]: 0.0

In [90]: t.utc
Out[90]: <Time object: scale='utc' format='gps' value=0.0>

System Details

Linux-4.19.0-13-amd64-x86_64-with-debian-10.7
Python 3.7.3 (default, Jul 25 2020, 13:03:44)
[GCC 8.3.0]
Numpy 1.16.2
astropy 4.0.1.post1
Scipy 1.5.2
Matplotlib 3.0.2

[taldcroft]

Many text formats using GPS time, such as RINEX messages, have defined the ISO or ISO-like text datetimes for GPS to be TAI - 19s

Is this so widely adopted to the extent that it could be considered a standard?

This might imply the need for a GPS time scale. It would be awkward if isot and iso give different answers than say 'yday` or all the other date-like representations. Adding a new time scale is something we have not done to this point since astropy.time strictly uses ERFA for time scale transformations. I don't have any estimate of how difficult this would be.

I haven't looked in detail, but I do believe that doing (t.tai - 19 * u.s).isot is the current best workaround for getting the RINEX version of GPS ISOT datetime.

[pllim]

@taldcroft , is this a bug or a feature request? Feel free to label accordingly. Thanks!

[zastruga]

I haven't looked in detail, but I do believe that doing (t.tai - 19 * u.s).isot is the current best workaround for getting the RINEX version of GPS ISOT datetime.

Thanks for the revised syntax. We're also doing the reverse to create Time objects from a GPS ISOT, Time(rinex_isot_string, scale='tai') + TimeDelta(19, format='sec').

Without creating a separate GPS time scale, would a different format string make sense for init and output? Naming things is always hard, but maybe t.rinex_isot and Time(rinex_isot_string, format='gps_rinex'). If creating a new time scale ultimately makes more sense, I don't want to push for this lipstick on a pig approach just hacking it in though.

[jbrandmeyer]

I'm not familiar with Astropy's internal definitions for what "time scale" and "format" mean to this project, to help steer you to implementing a new time scale. IMO, the library's behavior here is a bug.

Standards background:
RINEX is a standard medium of exchange for GNSS receiver data used by the scientific GNSS community. The standards are promulgated by the IGS. The spec was born with a number of GPS-isms in it, because GPS was the first and only game in town at one point, but it has slowly been evolving to be more generic for the four constellations currently on-orbit.

https://files.igs.org/pub/data/format/rinex305.pdf

GNSS are matters of national pride and security for the four system operators. As such, each one defines their own timekeeping in terms of their own system operator time (GPS System Time, Galileo System Time, BeiDou System Time, Galileo System Time). The system operators deliberately steer their clocks to follow UTC, and UTC is in turn partly defined by TAI.

So, strictly speaking, GPS system time's epoch was exactly midnight UTC in its epoch. At that point, UTC had accumulated 19 leap seconds relative to TAI. GPS broadcasts its own time in a format of Week Number + second of week. It also broadcasts the integer number of leap seconds relative to UTC as well as a fine offset between GPS time and UTC of up to 1 microsecond using a polynomial that is itself good to about 20 nanoseconds RMS.

See also the GPS ICD: https://www.gps.gov/technical/icwg/IS-GPS-200K.pdf

Each of the system operators handles the differences between their own system time and UTC in their own way. GLONASS system time includes leap seconds and a 3-hour offset. GAL's epoch is not the GPS epoch... but does align its weekly rollover point to GPS. BDS aligns its epoch to a different UTC epoch, but is otherwise semantically very similar to the GPS case. All of them broadcast a fine offset between their own system time and UTC on the order of dozens of nanoseconds, and there is a certain amount of competitiveness and specsmanship in the tolerance thereof.

Now, with that background:

A quirk of RINEX is that it uses a datetime breakdown for its time fields, however, its timestamps have always been measured in GPS time as-if you formed a datetime from the weeknumber + second-of-week format. The lack of leapseconds in the native GPS timebase, and the fact that it is steered to match UTC (modulo leap seconds) means that using TAI is "not wrong" for a GPS timescale, since for most purposes you can ignore the sub-microsecond time-varying offset. If you really wanted to, you could model a GPS timescale by incorporating a record of the submicrosecond differences between GPS time and UTC, either using broadcast ephemeris or IGS reports. That's probably out of scope for astropy, though.

BUT.

Zero in GPS time is exactly midnight '1980-01-06T00:00:00.000' when rendered as an ISOT datetime.

[jbrandmeyer]

(my interest and who I am): I am building a multi-system GNSS receiver to construct a special type of Earth-observing satellite along with zastruga and others. We ran into this specific issue when using the astropy to construct datetime-breakdown output for RINEX as a medium of interchange with some of our partners. Not many libraries successfully handle leap seconds at all, which is part of why we picked this library as the basis for managing some of the different time bases we have to deal with.

[taldcroft]

I have a proof-of-concept fix which adds a new time scale gps_scale and gives the expected behavior in limited testing.

In [24]: t = Time('1980-01-06', scale='gps_scale')

In [25]: t.isot
Out[25]: '1980-01-06T00:00:00.000'

In [26]: t.gps  # SI seconds since the GPS epoch time 1980-01-06 00:00:00 UTC
Out[26]: 0.0

In [27]: t.tai.isot
Out[27]: '1980-01-06T00:00:19.000'

In [28]: t = Time('2020-01-01', scale='gps_scale')

In [29]: t.isot
Out[29]: '2020-01-01T00:00:00.000'

In [30]: t.tai.isot
Out[30]: '2020-01-01T00:00:19.000'

In [31]: t.gps
Out[31]: 1261872000.0

[taldcroft]

@jbrandmeyer - thanks for bringing your expertise to the issue. Does the output I supplied match your expectations? Are there other tests that you would suggest?

If you new to astropy.time, the format is any representation of the time, e.g. ISOT string, or seconds since an epoch, or year-day-of-year, JD, MJD, etc. In this case t.gps is a format (float seconds since the epoch) while gps_scale is a time scale like UTC or TAI. So here the GPS scale is TAI - 19 sec.

[jbrandmeyer]

Yes, those output look good to me. Thanks!

[taldcroft]

See #11269.

[jbrandmeyer]

See also Table 2 of https://files.igs.org/pub/data/format/rinex305.pdf for some additional example GPS-UTC leap second offsets.