Add an 'asset definition' file to the spec

[cholmes]

One of the main things discussed in the EO session was the notion of a 'product' or 'asset definition' file. This would be a link from an Item, that contains all the definitions of the possible assets in the item. The 'assets' dict in the Item should share the same keys as the asset definition, so one could easily look up the fields. I believe we also decided the asset definition should be optional - assets could also define their fields inline.

We also need to flesh out a bit more the asset fields - what all should have, what the common names are and what they mean, like 'name' and 'type', and which are required.

[matthewhanson]

The Asset Definition would apply to just core STAC as well, so it shouldn't be tied to the EO extension, although that would provide some additional fields.

The more I thought about the Asset Definition though the more I thought about how it would actually be incorporated into the API. As a user for the API I would want to get the asset definitions for all the different types of data that were returned.

I think we should consider combining the Asset Definition in with 'Collections'. Conceptually they are very much the same and it would really help clarify what a Collection is. A Collection is a schema for data that shares common asset definition. When I query the API I get a bunch of results which belong to one or more collections. In the returned geojson at the top level I should also get the Collection definitions, which contain all the info about assets and other info specific to a Collection.

Thus a Landsat collection record would look like this (I left out wavelengths for all the bands, and descriptions, so it would be more complete in practice, but all the bands are here).

{
	"name" : "landsat-toa"
	"platform" : "landsat-8"
	"instrument" : "OLI_TIRS"
	"bands": {
		"1":  {
			"common_name": "coastal",
			"gsd": 30.0,
			"accuracy: 5.0,
			"wavelength": 0.47,
			"fwhm": 0.04
		},
		"2": {
			"common_name": "blue"
			"gsd": 30.0
		},
		"3": {
			"common_name": "green"
			"gsd": 30.0
		},
		"4": {
			"common_name": "red"
			"gsd": 30.0
		},
		"5": {
			"common_name": "nir"
			"gsd": 30.0
		},
		"6": {
			"common_name": "swir16"
			"gsd": 30.0
		},
		"7": {
			"common_name": "swir22"
			"gsd": 30.0
		},
		"8": {
			"common_name": "pan"
			"gsd": 15.0
		},
		"9": {
			"common_name": "cirrus"
			"gsd": 30.0
		},
		"10": {
			"common_name": "lwir11"
			"gsd": 60.0
		},
		"11": {
			"common_name": "lwir12"
			"gsd": 60.0
		}
	 }
	 "assets" = {
		"B01": {
			"type": "GeoTIFF"
			"required": "true"
			"bands" : ["1"]
		},
		"B02": {
			"type": "GeoTIFF"
			"required": "true"
			"bands" : ["2"]
		},
		"B03": {
			"type": "GeoTIFF"
			"required": "true"
			"bands" : ["3"]
		},
		"B04": {
			"type": "GeoTIFF"
			"required": "true"
			"bands" : ["4"]
		},
		"B05": {
			"type": "GeoTIFF"
			"required": "true"
			"bands" : ["5"]
		},
		"B06": {
			"type": "GeoTIFF"
			"required": "true"
			"bands" : ["6"]
		},
		"B07": {
			"type": "GeoTIFF"
			"required": "true"
			"bands" : ["7"]
		},
		"B08": {
			"type": "GeoTIFF"
			"required": "true"
			"bands" : ["8"]
		},
		"B09": {
			"type": "GeoTIFF"
			"required": "true"
			"bands" : ["9"]
		},
		"B10": {
			"type": "GeoTIFF"
			"required": "true"
			"bands" : ["10"]
		},
		"B11": {
			"type": "GeoTIFF"
			"required": "true"
			"bands" : ["11"]
		},
}

The problem I'm having with just a link to an Asset Definition is that it seems like it's too flexible and would largely be up the client to figure out retrieving that link and using it.

Collections on the other hand give us a clear endpoint /collection, consistent with WFS 3.0 and conceptually fits what we need here.

[cholmes]

Sorry, just saw this note.

I've been feeling like the 'asset definition' isn't well thought out enough myself. I think for 0.4.0 release we should just drop the 'asset definition', and just go ahead with this collection idea, constrained to the EO profile. This should also be maximally useful to Open-EO/openeo-api#64 - they can hopefully borrow / give feedback on our collection level data.

I think it could make sense to add a couple more common fields to the asset object in the Item json, just as optional recommendations.

The other thing I've been thinking about is the catalog.json, that provides some keywords, description, etc for static catalogs. We could consider combining that too? I've been trying to figure out how that fits in with the API as well.

[matthewhanson]

@cholmes what sorts of fields do you think might be added to the Asset object?

Can you elaborate on the catalog.json? Are you thinking that would be like collection level metadata?

Also, if we use collections, we'll need to update the API to allow retrieving collection level metadata.

We should consider too including any collection level metadata in the return response for all included collections. For instance, if we search landsat and get the FeatureCollection, at the top level of the featurecollection we could have:

{
   "type": "FeatureCollection",
   "Features": [...],
   "Collections": {
      "landsat-8": { /*collection metadata*/ }
   }
}

[cholmes]

So I was just thinking about the fields we had in asset dictionary - recommending them direct to the asset:

   "name" : "4-band analytic image"
   "description" : "A 4-band analytic PlanetScope Image"
      "type" : "Geotiff"
      "required" : "true" (or false)
      "bands" : [ "1", "2", "4", "3" ]

Required makes less sense at the item level - it's there or not. And bands is eo. So I guess it's really name, description and type. I think we should make a stab at 'common types', since I've seen even just 'geotiff' all over the map - 'geotif', 'gtiff', etc. I also wonder if we might do a 'sub type' so you could say 'geotiff' / 'cloud-optimized' and 'metadata' / 'json'.

The other one I thought might be useful is 'group'. When I tried auto-generating html from the stac json I ended up with: https://cholmes.github.io/pdd-stac/d417bee23f4ecbca42ed4718b3ea6e31.html But found myself wanting to automatically generate the headings based on the 'group'. So it'd let people group things however they want, but a little JS html converter could use the group names for 'headings' and put the groups of files under it, instead of just a dump of all the assets.

[cholmes]

My thoughts aren't that fleshed out for catalog.json. For static catalogs it's just an entry point, and has some additional collection level metadata. But it seems to 'sit' at about the same level, so yeah, was thinking we could just combine the collection and the catalog.json in to a single thing.

catalog.json also right now only exists in the static catalogs, so it also needs a recommendation for what to do in the active API.

[cholmes]

I'm not sure if I'm yet convinced to make it a required endpoint, want to keep core stac api not requiring tons of work to make sure everything exposed has to be like schema validated. But I think I like it as a recommended endpoint. I could see situations where items returned just link to a known landsat 'collection' endpoint instead of api's providing their own.

Similarly I think including the collection level data at a featurecollection level is cool, but hesitate to make it required. But perhaps we strongly recommend a link to the collection metadata at the least.

[cholmes]

I think we've moved away from this direction a bit, so I'm closing this.