Standardising the AreaDefinition names for GEO satellites/instruments

[sjoro]

Feature Request

areas.yaml-file has multiple area definitions, e.g., for SEVIRI instrument. If a user wants to resample data from a LEO satellite to the official SEVIRI full disk grid (0 degree subsatellite point, 3km resolution) it is impossible to guess what the correct area definition is and a visit to areas.yaml-file is required. Even then, duplicate definitions can be found, i.e. met09globeFull and seviri_0deg and the resolution is of the grid is not clear and it needs to be estimated from the shape or hopefully the description will shed some light on this. Hence, a guideline should be established on how to name the official full disk area definitions for GEO instruments so that a user, knowing the instrument, subsatellite longitude or service, and the resolution of the target grid can easily either figure out the area name or can easily find the correct official target grid from the areas provided in areas.yaml. Consequently, it needs to be ensured that a GEO reader returns an area definition with the same official name and area extent (to the highest possible accuracy, small deviations can occur as area extents are calculated on-fly) than what is defined in the areas.yaml. This, of course, only applies to full disk datasets or file chunks padded to a full disk grid. Furthermore, readers for the same instrument, but in different format (HRIT, native, netCDF), need to return the area definitions with the same official naming for the sake of consistency and clarity.

Describe the solution you'd like
A way to standardize the naming of the "official" area definition names for an instrument. With this, knowing the target instrument grid with the subsatellite point (or service) and resolution, it is easy to guess/infer/derive what the area definition is, and in the best case, skip the need to browse the areas.yaml-file or even create the correct name in a script automatically from metadata. Suggestion for a naming template as a starting point for discussion:
<instrument>_<service or subsatlon>_<resolution>. Some examples on how this would look like, seviri_fes_3km, seviri_0415_1km, seviri_0095_9000, etc...
The representation of the resolution needs to be decided, should it be in pixels, meters, or kilometers. Also, if the name should include the subsatellite point or service.

Describe any changes to existing user workflow
No changes unless users want to start using the new established area definitions. Duplicate existing definitions should be kept for backward compatibility, but could potentially be removed at a later stage in order to reduce confusion and the unnecessary inflation of the areas.yaml-file.

[simonrp84]

It might be better to extend that even further:
<instrument>_<service>_<subsatlon>_<resolution> as, at least with SEVIRI there's a few cases of RSS data and full disk data from 3.4W longitude.

[gerritholl]

See also #1187 #1207 and perhaps #1206 and #1240.

[gerritholl]

It would seem that <sublatlon> can be <sublon> for GEO satellites/instruments (by definition).

Do we aim to limit this to full disc areas or also smaller areas such as CONUS, PACUS, RSS, etc.? MESO would be difficult to do statically.

We should include not only a good name but also a useful description. In your example, I don't know what 0415 refers to.

[sjoro]

@simonrp84 i believe the service would be enough, there's no need for subsatellite longitude. if i remember correctly the times when we've hade satellites over 3.4°W the images have still been rectified to 0° for FES and 9.5° for RSS so those longitudes would be the ones to be used in the area definition. i reserve the right to be wrong here, too ;)
@gerritholl you might have made a mistake in reading my suggestion, i didn't mention <sublatlon>, but <subsatlon> referring to subsatellite longitude. also, i probably should have browsed the open issues :). but here's one for the pile! 0415 is the subsatellite longitude for IODC mission at 41.5°. i agree that the definitions need to come with a useful description too.

[gerritholl]

You're right, my mistake, I did misread — subsatlon makes a lot more sense than sublatlon :)

[djhoese]

Does this standardization apply to the AreaDefinitions created in the readers? Obviously the extents and sizes should be the same, but should the names? One concern I would have is that these complicated names might not be great when put in a filename. I don't think this is a good enough reason not to standardize them as described here, but just wanted to point out this use case. I sometimes create filenames that include the area ID in the filename to identify where the geotiff is not just what it is.

Another point is that some groups/organizations/satellites have named locations. I suppose this is similar to the FES in your SEVIRI example but I'm not familiar with what that is/means. The big one is GOES-WEST, GOES-EAST, GOES-TEST, GOES-STORE where WEST and EAST are the locations the satellites will be in (and the data projected to) when they are in operations. TEST is where the satellite originally is while it is being calibrated/validated/evaluated. STORE is when the satellite isn't being used any more, but is put in a "safe keeping" orbit. So I'm wondering if these names have a place in this new naming scheme? I suppose abi_west_500 isn't too bad.

Lastly, is resolution in meters or when do you go to kilometers?

Edit: About this applying to the readers, I suppose it has to otherwise the area in the YAML wouldn't match the area from the reader.

[simonrp84]

@sjoro I think there were a few tests (maybe the super rapid scan data?) that was rectified to 3.4W. This would also be useful for the new generation of GEOsats, like GOES, where 1 satellite operates multiple services (CONUS, FES, MESO).

(edit) My preference would be for everything in meters. Would be confusing to switch m/km depending on the instrument - representing GAOFEN-4's 50m band in km would be tricky :)

[gerritholl]

Metre being the SI unprefixed unit would be the most correct and flexible, and (at least for now) no problem on how to deal with any decimal point to describe the resolution in sub-metre precision.

[sfinkens]

Nice initiative, @sjoro! I Agree that service/projection_longitude is enough. I have a preference for meters, too. And it would be nice if the areas created by the reader matched the area in the YAML. Let me know if you need some help collecting all the area defs.

[sjoro]

To me the readers need to return the same name. @djhoese the ABI-reader seems to return geos_abi as the name at the moment. if i understood correctly, you use this in filenames? FES is Full Earth Scan, it's EUMETSAT's prime service and i basically defines a location at 0 degree longitude. other services currently are Rapid Scanning Service (RSS) and Indian Ocean Data Coverage (IODC). they describe the service, and are provided from a certain location.

just for aesthetics, i like the resolution in kilometers if the resolution is greater than 1000m, if less than a kilometer then in meters. with this SEVIRI FES area definition would be seviri_fes_3km. to me that's nicer looking than seviri_fes_3000. in the latter case it should probably be seviri_fes_3000m as, especially new users, might wonder what the meaning of 3000 is.

seems like with ABI things get tricky if the different services are considered, e.g. CONUS, MESO. that would call a naming scheme with <instrument>_<location>_<service>_<resolution>. in case of SEVIRI location and service are basically bound together and if using this template it just looks wrong: seviri_rss_rss_3km or we use subsatellite longitude as location seviri_0095_rss_3km.

[gerritholl]

If the unit is included in the label then there is no need anymore to be consistent and it should be fine to have some names with 500 m and another with 3 km. I propose we should rule out decimal points though.

It doesn't sound feasible to include all MESO area definitions in areas.yaml, there are too many. For a standardised name returned by the reader, heir location would need latitude as well as longitude.

[simonrp84]

I assumed that the meso sectors wouldn't be included in the yaml file (as there's infinite variation) but that the in-reader naming convention for the meso sectors should still follow that for the various other (fixed) geostationary areas.

[djhoese]

the ABI-reader seems to return geos_abi as the name at the moment. if i understood correctly, you use this in filenames?

Actually geos_abi is the fallback if the orbital_slot attribute is not available from the file. It also uses the spatial_resolution attribute for the area's description. Here's an example that shows that shows that GOES-East would appear in the filename.

                :spatial_resolution = "1km at nadir" ;
                :orbital_slot = "GOES-East" ;

As for meters versus kilometers, I vote for meters. With sub-1km resolutions becoming more common you get weird periods or other short versions of 0.5 or 0.25 or some other weird spacing and opinions on leading and trailing zeroes.

About ABI's different sectors, ABI isn't the only one. AHI (and I think AMI is supposed to too) has a Japan sector and other moveable regions like ABI's mesos (if I recall correctly).

[simonrp84]

Yes, that's right. To my knowledge MSG is the only platform that devotes an entire satellite to sub-region scanning. All the other GEOs supported by satpy have multiple operational scan modes from one satellite.

[sjoro]

@djhoese by adding the unit in the area name there's no need for leading/trailing zeros, e.g., abi_west_fes_500m or seviri_fes_3km. to me, this notation is the most readable.

as with some (many) instruments providing services from a single location the template could be <instrument>_<location>_<service>_<resolution>, for SEVIRI, and most likely for FCI, this doesn't really work as the service is bound to a location. so maybe the solution here is to have <instrument>_<service>_<resolution> . forcing each instrument to a fixed template just doesn't seem to be the best solution after this discussion. so, maybe we could have a few "official" ways.

[sjoro]

is my suggestion above reasonable? i.e. we implement seviri_fes_3km, seviri_fes_1km, etc... for seviri-readers and in the areas.yaml? i'd like to bump this issue to be active again as we'd need to have a solution for continuing our offline tool development at EUM.

[mraspaud]

I'm good with it. But let @djhoese and @simonrp84 tell us what they think about it!

[djhoese]

I'm fine with that for SEVIRI. I think we'll just have to have a couple different schemes. Or maybe (similar to reader naming) we consider the location and optional prefix to the service.

abi_west_fldk_500m
goes17_fldk_500m (no abi, not good)
goes-west_fldk_500m
g17-abi_fldk_500m

Earlier I suggested abi_west_500, but I'm realizing that AreaDefinitions include the extent so fldk (or conus or m1 or m2) needs to be there too.

😓

[simonrp84]

Having thought about this a bit more, I'm tempted to include the satellite name in the area def too. For seviri it's quite obvious which satellite it is, but for the next-gen GEOs it's confusing: abi, ami, ahi. Easy to get them muddled up.

So how about this?
<sensor>_<platform>_<service>_<resolution>

Some examples:
seviri_msg_fes_3km, seviri_msg_rss_1km, seviri_msg_iodc_3km
abi_goesw_fldk_500m, abi_goese_conus_1km, abi_goesw_pacus_2km
ahi_himawari_fldk_2km, ahi_himawari_jp01_1km
ami_geokompsat_fldk_500m, ami_geokompsat_korea_1km, ami_geokompsat_eastasia_500m

I know these names (especially ami) are quite verbose...but IMO clarity for users is more important than a short name :)

[sjoro]

i'm pretty neutral, i.e. open, on having the platform name in. but as @djhoese said, i think we could have a couple of different schemes. so, the services and platforms that need it could have it. i would suggest just to have the info in a bit different order:
<platform>_<sensor>_<service>_<resolution>.

for SEVIRI/FCI i don't think this is needed. we could have msg_seviri_fes_3km, i have no real objections for this, but MSG is not the official platform and this acronym is nowhere to be found in the metadata. more official naming would need to be e.g. meteosat-8_seviri_fes_3km. this would actually be the same projection as meteosat-9_seviri_fes_3km, and the same for MET10 and for MET11. areas.yaml should definitely not have each of the area definitions spelled out separately as they describe the same exact thing. this could be solved with leaving the satellite number out from the platform -> meteosat_seviri_fes_3km. for the future missions meteosat_fci_fes_1km.

my ordered preference for naming SEVIRI and FCI areas (as examples):

1. seviri_fes_3km
2. msg_seviri_fes_3km, mtg_fci_rss_2km (i know i just ranted MSG/MTG not being official platforms, but still!)
3. meteosat_fci_fes_1km

complications to this naming scheme may arise if/when EUM continues the IODC service with MET09 satellite. MET08 is currently providing this service from 41.5E, it is suggested that MET09 would provide the same service from a different subsatellite longitude. so, msg_seviri_iodc_3km in this case will not remain unique and could mean two different subsatellite longitudes depending on what actual platform the data is coming from. this, of course, is with the assumption that the MET09 service over Indian Ocean would continue having the same official service name as what MET08 has, i.e. IODC.

[simonrp84]

For the sake of clarity and completeness I still prefer to include msg, despite it being non-official**. meteosat is probably a bad choice in case anyone gets confused with first or third gen meteosat.

The possible different position of Met-09 for the IODC coverage is problematic...but maybe that's a challenge we can leave until the future.

** I see that EUM often uses MTG for 3rd gen, so it can't be too much of a crime to use MSG for 2nd gen. ;-)

[djhoese]

This is a tough one. If we include the platform it is definitely going before the sensor. We have this platform naming issue even in the regular satpy metadata; for sensors too. Is it g16 or goes16 or goes-16? Do we allow for capital letters (no)? What about Himawari? There is technically a Himawari 9 but my understanding is it is backup for H8 if H8 goes down. And what do we do when some organization changes the projection properties or moves the satellite but still calls it the same name? goes16_abi_fldk_500m_2? I think meteosat would need the number regardless of it being redundant; that is if we are going for consistent naming schemes.

I only pose questions this time. No solutions. Sorry.

[simonrp84]

Do we need goes16? What's wrong with goeseast, goes_east, goes_e or something like that? Keeping the satellite number out of it is, imo, the best solution as - like you say, there are backup satellites with a different number but that supply the same projection. As this is for an area definition I don't think the actual satellite number - in most cases - adds any value.

[djhoese]

Ah good point. So I guess same issue but goeseast, goes_east, or goes-east. And I guess that points out why the "platform" might come after the sensor in this case (or at least it seems to) because GOES-EAST is the position/location, not the platform. 🤦

[sjoro]

i agree with @simonrp84. area definitions do not need the actual platform with the number as the same service/projection can be served by different platforms. maybe we need to call it pseudo-platform ;)
i feel like we're having at least a bit of a consensus here... i'll let my new team member have a crack at this as an introduction to contributing to Satpy so comments are still welcome at the PR review stage, hehe. we'll start with the SEVIRI-readers. if there are no strong objections, the naming will follow msg_seviri_fes_3km-type of convention.

[pnuu]

Keeping the satellite number out of it is, imo, the best solution

YES! There are still people at FMI talking about "Meteosat-8" products even if they are now done from Meteosat-11, and in between with 9 and 10. These name stick too hard if introduced..