Improve clock file handling

CHANGELOG.md

@@ -7,8 +7,17 @@ and this project, at least loosely, adheres to [Semantic Versioning](https://sem
 ## Unreleased
 ### Added
 - Warning when A1DOT parameter used with DDK model
+- Added the limits="warn" or limits="error" to get_TOAs to select handling of uncorrected TOAs
+- Added functions in pint.observatory to request the status of PINT's available clock corrections
+- Added the ability to query clock correction files or observatories for their last corrected MJD
 ### Fixed
 ### Changed
+- Clock correction files that are entirely missing are handled the same way as TOAs past the end of a clock correction file
+- Observatory objects can now note that their clock correction files include a bogus "99999" (or similar) entry at the end
+- Clock correction files are now checked for being in order (necessary for PINT's interpolation to function)
+- Observatories using TEMPO-format clock files now refer to the clock file (for example time_ao.dat) rather than via time.dat
+- Observatories that don't use clock corrections (for example CHIME uses GPS time directly) just use an empty list of clock correction files rather than files containing only zeros
+- Updated Jodrell clock corrections to include post-1997
 
 ## [0.8.6 == 0.8.7] 2022-05-10
 ### Added

docs/explanation.rst

@@ -207,21 +207,54 @@ accuracy.
 Clock corrections
 '''''''''''''''''
 
-Not all the data that PINT uses is easily accessible for programs to
-download. Observatory clock corrections, for example, may need to be
-obtained from the observatory through various means (often talking to a
-support scientist). We intend that PINT should notify you when this is
-necessary, but be aware that you may obtain reduced accuracy if you
-have old clock correction files.
-
-The PINT distribution includes some clock files, but these are not necessarily
-up-to-date enough. They normally live in the ``src/datafiles/`` directory with
-names like ``time_gbt.dat``. There is a ``README.md`` in there describing some
-ways to update your clock files. PINT is also capable of using clock files from
-a TEMPO or TEMPO2 installation, if you have the ``TEMPO`` or ``TEMPO2``
-environment variables set. You can see what clock file your corrections are
-coming from with a command like
-``pint.observatory.get_observatory("GBT").clock_fullpath``.
+Not all the data that PINT uses is easily accessible for programs to download.
+Observatory clock corrections, for example, may need to be obtained from the
+observatory through various means (often talking to a support scientist). PINT
+tries to include all clock correction files from the TEMPO and TEMPO2
+repositories, and we intend to update the PINT source as these are updated.
+PINT should notify you when your clock files are out of date for the data you
+are using; be aware that you may obtain reduced accuracy if you have old clock
+correction files.
+
+Normally, if you try to do some operation that requires unavailable clock
+corrections, PINT will emit a warning but continue. If you want to be stricter,
+you can specify ``limit="error"`` to various functions like
+:func:`pint.toa.get_TOAs`.
+
+If you need to check how up to date your clock corrections are, you can use
+something like ``get_observatory("gbt").last_clock_correction_mjd()``: the
+function :func:`pint.observatory.Observatory.last_clock_correction_mjd` checks
+when clock corrections are valid for. For most telescopes, this combines the
+per-telescope clock correction with PINT's global GPS and BIPM clock
+corrections (both of which cannot be reliably extrapolated too far into the
+future). PINT provides two convenience functions,
+:func:`pint.observatory.list_last_correction_mjds` and
+:func:`pint.observatory.check_for_new_clock_files_in_tempo12_repos`, that will
+help you check the state of your clock corrections.
+
+If you need clock files that are not in PINT, either more recent versions or
+clock files for telescopes not included in PINT, you will have to modify PINT's
+source code. (This is not an ideal situation!) You have two options:
+
+#. Simply replace a clock correction file in the PINT version you are using.
+   The file PINT is using may be deep in a virtualenv directory somewhere, but
+   you should be able to locate it with
+   ``get_observatory("gbt").clock_fullpath``. If you replace this file, you
+   should see updated clock corrections.
+
+#. Modify ``src/pint/observatory/observatories.py`` so that the observatory you
+   are interested in points to the correct clock file. (You may have to redo
+   ``pip install`` for PINT to make this take effect.) If you set
+   ``clock_dir="TEMPO"`` or ``clock_dir="TEMPO2"`` then PINT will look in the
+   clock directory referenced by your environment variables ``$TEMPO`` or
+   ``$TEMPO2`` (and nowhere else; it will no longer find clock corrections for
+   this observatory that are included with PINT). You can also specify a
+   specific directory as ``clock_dir="/home/burnell/clock-files/"``. Editing
+   this file also allows you to choose between TEMPO- and TEMPO2-format clock
+   corrections with the ``clock_fmt`` argument.
+
+The PINT developers recognize that the situation regarding clock corrections is
+not ideal.  Possible more flexible and powerful options are under discussion.
 
 Structure of Pulsar Timing Data Formats
 ---------------------------------------

src/pint/data/runtime/time_ao.dat

@@ -969,10 +969,12 @@
  49683.40      88.861      88.704 3 f  26-NOV-94
  49684.38      89.092      88.856 3 f  27-NOV-94
  49685.39      89.309      89.039 3 f  28-NOV-94
- 49528.1       73.492      71.487 3 f  25-JUN-94  raw readouts
- 49544.9       72.0        73.0   3 f  11-JUL-94  no data, rough interp.-->
- 49545.9       72.0        73.1   3 f  12-JUL-94  --> need more precision
- 49557.79      74.213      74.287 3 f  24-JUL-94  your time.dat
+# Zapped these as they are out of order and the comments suggest they
+# are invalid. Anne Archibald, 2022 May 11
+# 49528.1       73.492      71.487 3 f  25-JUN-94  raw readouts
+# 49544.9       72.0        73.0   3 f  11-JUL-94  no data, rough interp.-->
+# 49545.9       72.0        73.1   3 f  12-JUL-94  --> need more precision
+# 49557.79      74.213      74.287 3 f  24-JUL-94  your time.dat
 50155.0        0.000       -0.007 3    13-Mar-1996  post-upgrade data from
 50156.0        0.000       -0.060 3    14-Mar-1996  aosun:~joe/tac/aoclk.1
 50157.0        0.000       -0.103 3    15-Mar-1996  12 October 1999

src/pint/data/runtime/time_gb140.dat

@@ -1486,8 +1486,9 @@
 51206.500       0.0        24.086 a    28-Jan-99
 51207.500       0.0        24.123 a    29-Jan-99
 51208.500       0.0        24.150 a    30-Jan-99
-51209.500       0.0        24.172 a    31-Jan-99
-51210.500       0.0        24.197 a     1-Feb-99
+# These seem to be repeated. Anne Archibald, 2022 May 11
+# 51209.500       0.0        24.172 a    31-Jan-99
+# 51210.500       0.0        24.197 a     1-Feb-99
 51209.500       0.0        24.190 a    31-Jan-99
 51210.500       0.0        24.218 a     1-Feb-99
 51211.500       0.0        24.272 a     2-Feb-99