Train Model on Optimal Cloud Analysis product

[jacobbieker]

DARDAR has some very detailed maps from CALIPSO and CloudSat satellites. Unfortunately, the data ends around 2016-2017. There could be some use in still training a model on historical data, then using it to predict labels on newer clouds? It includes detailed breakdowns of the types of clouds as well, compared to most cloud masks are binary cloud/no-cloud without considering the different between a thin cloud blocking only some irradiance, and a thick cloud blocking essentially all of it for example.

[jacobbieker]

While actually using DARDAR doesn't seem very feasible for making a model, this product: https://navigator.eumetsat.int/product/EO:EUM:DAT:MSG:OCA was calibrated off DARDAR and gives more detailed cloud properties, but is only every hour. But could give something similar?

[jacobbieker]

Also could be useful are these products: https://data.eumetsat.int/product/EO:EUM:DAT:METOP:IASIL1C-ALL?query=IASI%20L1C&s=extended and https://data.eumetsat.int/product/EO:EUM:DAT:METOP:AVHRRL1?query=&s=simple that give more detailed cloud and data streams. So in training we could give higher resolution masks, or more cloud properties, and then have it predict those along with the images in the future? Would probably need to mask the loss from the areas of the image not covered by these products, as these are not geo satellites.

[jacobbieker]

Also, these products have the advantage of being from the product store, so we can download them without needing to wait some indeterminate time, like the other product.

[jacobbieker]

Try it with https://github.com/facebookresearch/xcit which is available from timm