Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Functions to automatically generate reader table for documentation #2108

Merged
merged 10 commits into from
Jul 26, 2022
53 changes: 31 additions & 22 deletions doc/source/dev_guide/custom_reader.rst
Original file line number Diff line number Diff line change
Expand Up @@ -88,28 +88,37 @@ The ``reader`` section provides basic parameters for the overall reader.

The parameters to provide in this section are:

- name: This is the name of the reader, it should be the same as the
filename (without the .yaml extension). The naming convention for
this is described above in the :ref:`reader_naming` section above.
- short_name (optional): Human-readable version of the reader 'name'.
If not provided, applications using this can default to taking the 'name',
replacing ``_`` with spaces and uppercasing every letter.
- long_name: Human-readable title for the reader. This may be used as a
section title on a website or in GUI applications using Satpy. Default
naming scheme is ``<space program> <sensor> Level <level> [<format>]``.
For example, for the ``abi_l1b`` reader this is ``"GOES-R ABI Level 1b"``
where "GOES-R" is the name of the program and **not** the name of the
platform/satellite. This scheme may not work for all readers, but in
general should be followed. See existing readers for more examples.
- description: General description of the reader. This may include any
`restructuredtext <http://docutils.sourceforge.net/docs/user/rst/quickref.html>`_
formatted text like links to PDFs or sites with more information on the
file format. This can be multiline if formatted properly in YAML (see
example below).
- sensors: The list of sensors this reader will support. This must be
all lowercase letters for full support throughout in Satpy.
- reader: The main python reader class to use, in most cases the
``FileYAMLReader`` is a good choice.
name
This is the name of the reader, it should be the same as the
filename (without the .yaml extension). The naming convention for
this is described above in the :ref:`reader_naming` section above.
short_name (optional): Human-readable version of the reader 'name'.
If not provided, applications using this can default to taking the 'name',
replacing ``_`` with spaces and uppercasing every letter.
djhoese marked this conversation as resolved.
Show resolved Hide resolved
long_name
Human-readable title for the reader. This may be used as a
section title on a website or in GUI applications using Satpy. Default
naming scheme is ``<space program> <sensor> Level <level> [<format>]``.
For example, for the ``abi_l1b`` reader this is ``"GOES-R ABI Level 1b"``
where "GOES-R" is the name of the program and **not** the name of the
platform/satellite. This scheme may not work for all readers, but in
general should be followed. See existing readers for more examples.
description
General description of the reader. This may include any
`restructuredtext <http://docutils.sourceforge.net/docs/user/rst/quickref.html>`_
formatted text like links to PDFs or sites with more information on the
file format. This can be multiline if formatted properly in YAML (see
example below).
status
The status of the reader (e.g. Nominal, Beta, etc.)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we be clear about what options should be used here? Maybe limit it to a set of 3 or 4?

  • Nominal
  • Beta
  • Alpha

Not sure we need something more than those. Other ideas?

Copy link
Collaborator Author

@BENR0 BENR0 May 12, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will add Alpha. But in the old table there sometimes are comments for status other than the suggested three (for example the seviri_l1b_nc reader says that HRV is currently not supported. I don't know if such comments should be in a different column or should be allowed in the status column?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

...yeah that's difficult to say what is best. It makes the most sense in status, but it also seems ugly and prevents that column from staying small.

Copy link
Collaborator Author

@BENR0 BENR0 May 24, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@djhoese so I kept the comments in the status column for now since only a few of the readers had status comments and it felt overkill to have a extra column which is mostly empty. It looks like this now:

I added questionmarks for readers for which the status was unclear for me (see my comment below)

supports_fsspec
If the reader supports reading data via fsspec (either true or false).
sensors
The list of sensors this reader will support. This must be
all lowercase letters for full support throughout in Satpy.
reader
The main python reader class to use, in most cases the
``FileYAMLReader`` is a good choice.

.. code:: yaml

Expand Down
4 changes: 3 additions & 1 deletion satpy/etc/readers/abi_l1b.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -6,12 +6,14 @@
reader:
name: abi_l1b
short_name: ABI L1b
long_name: GOES-R ABI Level 1b
long_name: GOES-R ABI imager Level 1b data in netcdf format
description: >
GOES-R ABI Level 1b data reader in the NetCDF4 format. The file format is
described in the GOES-R Product Definition and Users' Guide (PUG). Volume
4 of this document can be found
`here <https://www.goes-r.gov/users/docs/PUG-GRB-vol4.pdf>`_.
status: Nominal
supports_fsspec: true
sensors: [abi]
default_channels:
reader: !!python/name:satpy.readers.yaml_reader.FileYAMLReader
Expand Down
4 changes: 3 additions & 1 deletion satpy/etc/readers/abi_l2_nc.yaml
Original file line number Diff line number Diff line change
@@ -1,12 +1,14 @@
reader:
name: abi_l2_nc
short_name: ABI L2 NetCDF4
long_name: GOES-R ABI Level 2 NetCDF4
long_name: GOES-R ABI Level 2 products in netCDF4 format
description: >
GOES-R ABI Level 2+ data reader in the NetCDF4 format. The file format is
described in the GOES-R Product Definition and Users' Guide (PUG) Volume
5. This document can be found
`here <https://www.goes-r.gov/products/docs/PUG-L2+-vol5.pdf>`_.
status: Beta
supports_fsspec: true
sensors: ['abi']
reader: !!python/name:satpy.readers.yaml_reader.FileYAMLReader
# file pattern keys to sort files by with 'satpy.utils.group_files'
Expand Down
6 changes: 5 additions & 1 deletion satpy/etc/readers/olci_l1b.yaml
Original file line number Diff line number Diff line change
@@ -1,6 +1,10 @@
reader:
description: NC Reader for OLCI data
name: olci_l1b
short_name: OLCI Level 1b
long_name: Sentinel-3 A and B OLCI Level 1B data in netCDF4 format
description: NC Reader for OLCI data
status: Nominal
supports_fsspec: true
sensors: [olci]
default_channels: []
reader: !!python/name:satpy.readers.yaml_reader.FileYAMLReader
Expand Down
6 changes: 5 additions & 1 deletion satpy/etc/readers/olci_l2.yaml
Original file line number Diff line number Diff line change
@@ -1,6 +1,10 @@
reader:
description: NC Reader for OLCI data
name: olci_l2
short_name: OLCI Level 2
long_name: Sentinel-3 A and B OLCI Level 2 data in netCDF4 format
description: NC Reader for OLCI data
status: Nominal
supports_fsspec: true
sensors: [olci]
default_channels: []
reader: !!python/name:satpy.readers.yaml_reader.FileYAMLReader
Expand Down
4 changes: 3 additions & 1 deletion satpy/etc/readers/seviri_l1b_hrit.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,9 @@ reader:
short_name: SEVIRI L1b HRIT
long_name: MSG SEVIRI Level 1b (HRIT)
description: >
HRIT reader for EUMETSAT MSG SEVIRI Level 1b files.
HRIT reader for EUMETSAT MSG (Meteosat 8 to 11) SEVIRI Level 1b files.
status: Nominal
supports_fsspec: true
sensors: [seviri]
default_channels: [HRV, IR_016, IR_039, IR_087, IR_097, IR_108, IR_120, IR_134, VIS006, VIS008, WV_062, WV_073]
reader: !!python/name:satpy.readers.yaml_reader.GEOSegmentYAMLReader
Expand Down
61 changes: 61 additions & 0 deletions satpy/utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@
from yaml import BaseLoader

from satpy import CHUNK_SIZE
from satpy.readers import available_readers

try:
from yaml import UnsafeLoader
Expand Down Expand Up @@ -622,3 +623,63 @@ def _merge_storage_options(storage_options, storage_opt_dict):
storage_options = storage_opt_dict

return storage_options


def rst_table_row(columns=None):
"""Create one row for a rst table.

Args:
columns (list[str]): Content of each column.

Returns:
str
"""
row = " * - {}\n".format(columns[0])
columns = [" - {}\n".format(col) for col in columns[1:]]
row = row + "".join(columns)

return row


def rst_table_header(name=None, header=None, header_rows=1, widths="auto"):
"""Create header for rst table.

Args:
name (str): Name of the table
header (list[str]): Column names
header-rows (int): Number of header rows
width (optional[list[int]]): Width of each column as a list. If not specified
defaults to auto and will therefore determined by the backend
(see <https://docutils.sourceforge.io/docs/ref/rst/directives.html#table>)

Returns:
str
"""
if isinstance(widths, list):
widths = " ".join([str(w) for w in widths])

header = rst_table_row(header)

table_header = (f".. list-table:: {name}\n"
f" :header-rows: {header_rows}\n"
f" :widths: {widths}\n\n"
f"{header}")

return table_header


def reader_table():
djhoese marked this conversation as resolved.
Show resolved Hide resolved
"""Create reader table from reader yaml config files.

Returns:
str
"""
table = [rst_table_header("Satpy Readers", header=["Description", "Reader name", "Status", "fsspec support"],
widths=[45, 25, 30, 30])]

reader_configs = available_readers(as_dict=True)
for rc in reader_configs:
table.append(rst_table_row([rc.get("long_name", "").rstrip("\n"), rc.get("name", ""),
rc.get("status", ""), rc.get("supports_fsspec", "false")]))

return "".join(table)