Refactor the telescope-related metadata in UVData, UVCal and UVFlag into an attached Telescope object

.github/workflows/macosx_windows_ci.yaml

@@ -27,29 +27,28 @@ jobs:
       matrix:
         os: [macos-latest, windows-latest]
         python-version: ["3.10", "3.11", "3.12"]
+        include:
+          - env_name: pyuvdata_tests_windows
+            os: windows-latest
+          - env_name: pyuvdata_tests_mac_arm
+            os: macos-latest
+          - env_name: pyuvdata_tests
+            os: macos-13
+            python-version: "3.12"
     steps:
       - uses: actions/checkout@main
         with:
           fetch-depth: 0
 
-      - name: Set ENV NAME
-        run: |
-          if [[ "${{ runner.os }}" = "Windows" ]]; then
-            echo "::set-output name=ENV_NAME::pyuvdata_tests_windows"
-          else
-            echo "::set-output name=ENV_NAME::pyuvdata_tests"
-          fi
-        id: env_name
-
       - name: Setup Minimamba
         uses: conda-incubator/setup-miniconda@v3
         with:
           miniforge-variant: Mambaforge
           miniforge-version: latest
           use-mamba: true
           python-version: ${{ matrix.python-version }}
-          environment-file: ci/${{ steps.env_name.outputs.ENV_NAME }}.yml
-          activate-environment: ${{ steps.env_name.outputs.ENV_NAME }}
+          environment-file: ci/${{ matrix.env_name }}.yml
+          activate-environment: ${{ matrix.env_name }}
           run-post: false
 
       - name: Conda Info

.gitignore

@@ -60,6 +60,7 @@ docs/uvdata.rst
 docs/uvcal.rst
 docs/uvbeam.rst
 docs/uvflag.rst
+docs/telescope.rst
 
 # PyBuilder
 target/

.pre-commit-config.yaml

@@ -20,7 +20,7 @@ repos:
     hooks:
       - id: flake8
         additional_dependencies:
-          - flake8-bugbear>=23.12.2
+          - flake8-bugbear>=24.2.6, <24.4
           - flake8-builtins>=2.2.0
           - flake8-comprehensions>=3.14.0
           - flake8-docstrings>=1.7.0

CHANGELOG.md

@@ -4,6 +4,9 @@ All notable changes to this project will be documented in this file.
 ## [Unreleased]
 
 ### Added
+- Several new methods on `Telescope` objects, including the classmethods
+`from_known_telescopes` and `new` to instantiate new objects and the
+`get_enu_antpos` method to get antenna positions in East, North, Up coordinates.
 - Support for writing "MODEL_DATA" and "CORRECTED_DATA" columns has been added to the
 `UVData.write_ms` method.
 - Support for "flexible-Jones" `UVCal` objects -- where different spectral windows can
@@ -27,6 +30,18 @@ time for each time range or the time_array (if there's a time_array and no time_
 - Added new keyword handling for v.6 of the MIR data format within `MirParser`.
 
 ### Changed
+- Telescope-related metadata (including antenna metadata) on `UVData`, `UVCal`
+and `UVFlag` have been refactored into a `Telescope` object (attached to these
+objects as the `telescope` attribute) and most of the code related to these
+attributes from the three objects has been consolidated on the `Telescope` object.
+These attributes are still accessible via their old names on the objects, although
+accessing them that way is now deprecated. Note that the telescope locations are
+now stored under the hood as astropy EarthLocation objects (e.g. `UVData.telescope.location`)
+(or as MoonLocation objects for simulated arrays on the moon if `lunarsky` is
+installed).
+- The `UVData.new` and `UVCal.new` methods now accept a telescope object for
+setting the telescope-related metadata (the older parameters are still accepted
+but deprecated.)
 - When FHD calibration solutions are read in, the `time_range` is now set to be
 one quarter of an integration time before and after the earliest and latest times
 respectively. This is a change from extending it by half an integration time to
@@ -49,6 +64,21 @@ times for those solutions.
 - updated minimum cython dependency to 3.0.
 - Updated minimum optional dependency versions: astropy-healpix>=1.0.2
 
+### Deprecated
+- Accessing the telescope-related metadata through their old attribute names on
+`UVData`, `UVCal` and `UVFlag` rather than via their attributes on the attached
+`Telescope` object (e.g. `UVData.telescope_name` -> `UVData.telescope.name` and
+`UVData.antenna_positions` -> `UVData.telescope.antenna_positions`).
+- Passing telescope-related metadata as separate parameters to `UVData.new` and
+`UVCal.new` rather than `Telescope` objects.
+- The `UVData.get_ENU_antpos` method in favor of `UVData.telescope.get_enu_antpos`.
+- The `Telescope.telescope_location` and `Telescope.telescope_name` attributes
+in favor of `Telescope.location` and `Telescope.name`.
+- The `get_telescope` function in favor of the `known_telescope_location` function
+and the `Telescope.from_known_telescopes` classmethod.
+- The KNOWN_TELESCOPE dict in favor of the `known_telescope_location` function
+and the `Telescope.from_known_telescopes` classmethod.
+
 ### Fixed
 - Fixed a bug in `UVBase` where `allowed_failures` was being ignored if a parameter had
 `required=True` set.

README.md

@@ -150,7 +150,7 @@ Required:
 * python >= 3.10
 * pyyaml >= 5.4.1
 * scipy >= 1.7.3
-* setuptools_scm <7.0|>=7.0.3
+* setuptools_scm >= 7.0.3
 
 Optional:
 

ci/pyuvdata_min_deps_tests.yml

@@ -15,5 +15,5 @@ dependencies:
   - pytest-cov
   - pytest-xdist
   - cython>=3.0
-  - setuptools_scm<7.0|>=7.0.3
+  - setuptools_scm>=7.0.3
   - pip

ci/pyuvdata_tests.yml

@@ -20,7 +20,7 @@ dependencies:
   - pytest-cov
   - pytest-xdist
   - cython>=3.0
-  - setuptools_scm<7.0|>=7.0.3
+  - setuptools_scm>=7.0.3
   - pip
   - pip:
       - lunarsky>=0.2.2

ci/pyuvdata_tests_mac_arm.yml

@@ -0,0 +1,26 @@
+name: pyuvdata_tests_mac_arm
+channels:
+  - conda-forge
+dependencies:
+  - astropy>=6.0
+  - astropy-healpix>=1.0.2
+  - astroquery>=0.4.4
+  - docstring_parser>=0.15
+  - h5py>=3.4
+  - hdf5plugin>=3.2.0
+  - matplotlib  # this is just for the doctests.
+  - numpy>=1.23
+  - pyerfa>=2.0.1.1
+  - python-casacore>=3.5.2
+  - pyyaml>=5.4.1
+  - scipy>=1.7.3
+  - coverage
+  - pytest>=6.2.5
+  - pytest-cases>=3.8.3
+  - pytest-cov
+  - pytest-xdist
+  - cython>=3.0
+  - setuptools_scm>=7.0.3
+  - pip
+  - pip:
+      - lunarsky>=0.2.2

ci/pyuvdata_tests_windows.yml

@@ -19,7 +19,7 @@ dependencies:
   - pytest-cov
   - pytest-xdist
   - cython>=3.0
-  - setuptools_scm<7.0|>=7.0.3
+  - setuptools_scm>=7.0.3
   - pip
   - pip:
       - lunarsky>=0.2.2

docs/conf.py

@@ -32,6 +32,7 @@
 uvcal_file = os.path.join(os.path.abspath("../docs"), "uvcal.rst")
 uvbeam_file = os.path.join(os.path.abspath("../docs"), "uvbeam.rst")
 uvflag_file = os.path.join(os.path.abspath("../docs"), "uvflag.rst")
+telescope_file = os.path.join(os.path.abspath("../docs"), "telescope.rst")
 
 # -- General configuration ------------------------------------------------
 
@@ -347,12 +348,14 @@ def build_custom_docs(app):
     import make_uvcal
     import make_uvbeam
     import make_uvflag
+    import make_telescope
 
     make_index.write_index_rst(readme_file=readme_file, write_file=index_file)
     make_uvdata.write_uvdata_rst(write_file=uvdata_file)
     make_uvcal.write_uvcal_rst(write_file=uvcal_file)
     make_uvbeam.write_uvbeam_rst(write_file=uvbeam_file)
     make_uvflag.write_uvflag_rst(write_file=uvflag_file)
+    make_telescope.write_telescope_rst(write_file=telescope_file)
 
 
 # this is to enable running python in the rst files.

docs/fast_calh5_meta.rst

@@ -0,0 +1,5 @@
+FastCalH5Meta
+=================
+
+.. autoclass:: pyuvdata.uvcal.FastCalH5Meta
+   :members:

docs/make_index.py

@@ -49,9 +49,10 @@ def write_index_rst(readme_file=None, write_file=None):
         "   uvcal\n"
         "   uvbeam\n"
         "   uvflag\n"
+        "   telescope\n"
         "   fast_uvh5_meta\n"
+        "   fast_calh5_meta\n"
         "   utility_functions\n"
-        "   known_telescopes\n"
         "   developer_docs\n"
     )
 

docs/make_telescope.py

@@ -0,0 +1,116 @@
+# -*- mode: python; coding: utf-8 -*-
+
+"""
+Format the Telescope object parameters into a sphinx rst file.
+
+"""
+import copy
+import inspect
+import json
+import os
+
+from astropy.time import Time
+
+from pyuvdata import Telescope
+from pyuvdata.telescopes import _KNOWN_TELESCOPES
+
+
+def write_telescope_rst(write_file=None):
+    tel = Telescope()
+    out = ".. _Telescope:\n\nTelescope\n=========\n\n"
+    out += (
+        "Telescope is a helper class for telescope-related metadata.\n"
+        "Several of the primary user classes need telescope metdata, so they "
+        "have a Telescope object as an attribute.\n\n"
+        "Attributes\n----------\n"
+        "The attributes on Telescope hold all of the metadata required to\n"
+        "describe interferometric telescopes. Under the hood, the attributes are\n"
+        "implemented as properties based on :class:`pyuvdata.parameter.UVParameter`\n"
+        "objects but this is fairly transparent to users.\n\n"
+        "Most commonly, Telescope objects are found as the ``telescope`` attribute\n"
+        "on UVData, UVCal and UVFlag objects and they are typically initialized\n"
+        "when those objects are initialized.\n\n"
+        "Stand-alone Telescope objects can be initialized in many ways: from\n"
+        "arrays in memory using the :meth:`pyuvdata.Telescope.from_params`\n"
+        "class method, from our known telescope information using the\n"
+        ":meth:`pyuvdata.Telescope.from_known_telescopes` class method,\n"
+        "from uvh5, calh5 or uvflag HDF5 files using the "
+        ":meth:`pyuvdata.Telescope.from_hdf5` class method,\n"
+        "or as an empty object (as ``tel = Telescope()``).\n"
+        "When an empty Telescope object is initialized, it has all of these \n"
+        "attributes defined but set to ``None``. The attributes\n"
+        "can be set directly on the object. Some of these attributes\n"
+        "are `required`_ to be set to have a fully defined object while others are\n"
+        "`optional`_. The :meth:`pyuvdata.Telescope.check` method can be called\n"
+        "on the object to verify that all of the required attributes have been\n"
+        "set in a consistent way.\n\n"
+        "Note that angle type attributes also have convenience properties named the\n"
+        "same thing with ``_degrees`` appended through which you can get or set the\n"
+        "value in degrees. Similarly location type attributes (which are given in\n"
+        "geocentric xyz coordinates) have convenience properties named the\n"
+        "same thing with ``_lat_lon_alt`` and ``_lat_lon_alt_degrees`` appended\n"
+        "through which you can get or set the values using latitude, longitude and\n"
+        "altitude values in radians or degrees and meters.\n\n"
+    )
+    out += "Required\n********\n"
+    out += (
+        "These parameters are required to have a basic well-defined Telescope object.\n"
+    )
+    out += "\n\n"
+    for thing in tel.required():
+        obj = getattr(tel, thing)
+        out += "**{name}**\n".format(name=obj.name)
+        out += "     {desc}\n".format(desc=obj.description)
+        out += "\n"
+
+    out += "Optional\n********\n"
+    out += (
+        "These parameters are needed by by one or of the primary user classes\n"
+        "but are not always required. Some of them are required when attached to\n"
+        "the primary classes."
+    )
+    out += "\n\n"
+
+    for thing in tel.extra():
+        obj = getattr(tel, thing)
+        out += "**{name}**\n".format(name=obj.name)
+        out += "     {desc}\n".format(desc=obj.description)
+        out += "\n"
+
+    out += "Methods\n-------\n.. autoclass:: pyuvdata.Telescope\n  :members:\n\n"
+
+    out += (
+        "Known Telescopes\n----------------\n\n"
+        "pyuvdata uses `Astropy sites\n"
+        "<https://docs.astropy.org/en/stable/api/astropy.coordinates."
+        "EarthLocation.html#astropy.coordinates.EarthLocation.get_site_names>`_\n"
+        "for telescope location information, in addition to the following\n"
+        "telescope information that is tracked within pyuvdata. Note that the\n"
+        "location entry is actually stored as an\n"
+        ":class:`astropy.coordinates.EarthLocation` object, which\n"
+        "is shown here using the Geodetic representation. Also note that for\n"
+        "some telescopes we store csv files giving antenna layout information\n"
+        "which can be used if data files are missing that information.\n\n"
+    )
+
+    known_tel_use = copy.deepcopy(_KNOWN_TELESCOPES)
+    for tel, tel_dict in _KNOWN_TELESCOPES.items():
+        if "location" in tel_dict:
+            known_tel_use[tel]["location"] = (
+                tel_dict["location"].to_geodetic().__repr__()
+            )
+
+    json_obj = json.dumps(known_tel_use, sort_keys=True, indent=4)
+    json_obj = json_obj[:-1] + " }"
+    out += ".. code-block:: JavaScript\n\n {json_str}\n\n".format(json_str=json_obj)
+
+    t = Time.now()
+    t.format = "iso"
+    t.out_subfmt = "date"
+    out += "last updated: {date}".format(date=t.iso)
+    if write_file is None:
+        write_path = os.path.dirname(os.path.abspath(inspect.stack()[0][1]))
+        write_file = os.path.join(write_path, "telescope.rst")
+    F = open(write_file, "w")
+    F.write(out)
+    print("wrote " + write_file)

docs/make_uvbeam.py

@@ -31,7 +31,8 @@ def write_uvbeam_rst(write_file=None):
         "this is fairly transparent to users.\n\n"
         "UVBeam objects can be initialized from a file using the\n"
         ":meth:`pyuvdata.UVBeam.from_file` class method\n"
-        "(as ``beam = UVBeam.from_file(<filename>)``) or be initialized as an empty\n"
+        "(as ``beam = UVBeam.from_file(<filename>)``) or be initialized from arrays\n"
+        "in memory using the :meth:`pyuvdata.UVBeam.new` class method or as an empty\n"
         "object (as ``beam = UVBeam()``). When an empty UVBeam object is initialized,\n"
         "it has all of these attributes defined but set to ``None``. The attributes\n"
         "can be set by reading in a data file using the :meth:`pyuvdata.UVBeam.read`\n"
@@ -43,7 +44,7 @@ def write_uvbeam_rst(write_file=None):
     )
     out += "Required\n********\n"
     out += (
-        "These parameters are required to have a sensible UVBeam object and \n"
+        "These parameters are required to have a well-defined UVBeam object and \n"
         "are required for most kinds of beam files."
     )
     out += "\n\n"