Open Satellite Image Cloud Detection Resources (OpenSICDR)

We collect the latest open-source tools and datasets for cloud and cloud shadow detection, and launch this online project (Open Satellite Image Cloud Detection Resources, i.e., OpenSICDR) to promote the sharing of the latest research outputs of the field. If you would like to provide new resources, please kindly contact Dr. Zhiwei Li at dr.lizhiwei(AT)gmail.com or submit an update request.

Source:

Zhiwei Li, Huanfeng Shen, Qihao Weng, Yuzhuo Zhang, Peng Dou, Liangpei Zhang. Cloud and Cloud Shadow Detection for Optical Satellite Imagery: Features, Algorithms, Validation, and Prospects. ISPRS Journal of Photogrammetry and Remote Sensing, vol. 188, pp. 89-108, 2022. (Link, PDF)

Contributors:

• Dr. Zhiwei Li, Wuhan University, dr.lizhiwei(AT)gmail.com
• Ms. Yuzhuo Zhang, Wuhan University, yuzhuozhang816(AT)whu.edu.cn

Update logs:

Feb 28, 2024: Added two new cloud detection datasets, GF1MS-WHU and GF2MS-WHU.

June 5, 2024: 1) Added one new cloud detection dataset, CloudSEN12; Added parts of cloud mask products in Google Earth Engine;

Open-Source Datasets for Cloud and Cloud Shadow Detection

Name	Image Source	References	Descriptions	Link
L7_Irish	Landsat-7 (30 m)	Scaramuzza et al., 2012; USGS., 2016a	Contains 206 Landsat-7 scenes from nine global latitude zones with manually generated masks, of which only 45 scenes are labeled for cloud shadows.	Link
L8_SPARCS	Landsat-8 (30 m)	Hughes and Hayes, 2014; USGS., 2016c	Contains 80 subsets of Landsat-8 scenes with a size of 1000×1000 pixels that are labeled for both clouds and cloud shadows.	Link
L8_Biome	Landsat-8 (30 m)	Foga et al., 2017; USGS., 2016b	Contains 96 Landsat-8 scenes from eight global biomes with manually generated cloud masks, of which 32 scenes are labeled for cloud shadows.	Link
95-Cloud	Landsat-8 (30 m)	Mohajerani and Saeedi, 2019	Contains 95 Landsat-8 images and associated pixel-level cloud labels that is an extension of the previously established 38-Cloud dataset.	Link
Snow-Cloud Validation Masks	Landsat-8 (30 m)	Stillinger and Collar, 2019	Contains 13 Landsat-8 images and corresponding clouds and snow labels at mid-latitude mountainous regions.	Link
RICE_dataset	Landsat-8 (30 m)	Lin et al., 2019	Contains 450 Landsat-8 images and corresponding cloud-free images and cloud labels with a size of 512×512 pixels in one of two subsets of the dataset.	Link
WHU Cloud Dataset	Landsat-8 (30 m)	Ji et al., 2021	Contains 7 Landsat-8 images and corresponding cloud-free historical images and cloud and shadow masks in six different regions.	Link
S2-Hollstein	Sentinel-2 (10 m)	Hollstein et al., 2016	Consists 5,647,725 pixels based on images acquired over the entire globe with cloud, cirrus, snow, shadow, and water labels.	Link
S2-BaetensHagolle	Sentinel-2 (10 m)	Baetens et al., 2018, 2019	Provides cloud masks for 38 Sentinel-2 scenes selected in 2017 or 2018, each with cloud and cloud shadow labels.	Link
T-S2/T-PS	Sentinel-2 (10 m) PlanetScope (3 m)	Shendryk et al., 2019	Contains 4,993 Sentinel-2 and 4,943 PlanetScope subscenes with a size of 512×512 pixels and only RGB and NIR bands over the Wet Tropics of Australia, each is labeled at the block level.	Link
Sentinel-2 Cloud Mask Catalogue	Sentinel-2 (10 m)	Francis et al., 2020	Comprises 20 m resolution cloud masks for 513 subscenes, of which 424 subscenes are labeled for cloud shadows.	Link
Sentinel-2 KappaZeta	Sentinel-2 (10 m)	Domnich et al., 2021	Contains 4403 labeled image blocks with a size of 512×512 pixels from 155 Sentinel-2 images over the Northern European terrestrial area.	Link
WHUS2-CD	Sentinel-2 (10 m)	Li et al., 2021	Contains 32 Sentinel-2 images distributed in Mainland China and its reference cloud masks labeled at 10 m resolution.	Link
CloudSEN12	Sentinel-2 (10 m)	Aybar et al., 2022	Contains 49,400 Sentinel-2 image patches, each sized 509×509 pixels, evenly distributed across all continents except Antarctica.	Link
GF1_WHU	Gaofen-1 WFV (16 m)	Li et al., 2017	Contains 108 globally distributed GF-1 WFV scenes and their manually labeled cloud and cloud shadow masks.	Link
Levir_CS	Gaofen-1 WFV (16 m)	Wu et al., 2021	Contains 4,168 globally distributed Gaofen-1 WFV scenes (down sampled to 160 m resolution) and the corresponding geographical data, cloud, and snow labels.	Link
GF1MS-WHU GF2MS-WHU	Gaofen-1 PMS (8 m) Gaofen-2 PMS (8 m)	Zhu et al., 2024	Contains 141 unlabeled and 33 well-annotated 8-m Gaofen-1 PMS multispectral images; Contains 163 unlabeled and 29 well-annotated 4-m Gaofen-2 multispectral images.	Link
WDCD dataset	Gaofen-1 PMS (8 m) Ziyuan-3 MUX (5.8 m)	Li et al., 2020	Contains over 200,000 globally distributed Gaofen-1 image blocks labeled at the block level for training and 30 Gaofen-1 and Ziyuan-3 scenes labeled at the pixel level for validation and testing.	Link
N/A	Gaofen series (N/A)	Sun et al., 2020	Contains 745 paired NIR-R-G composited images and corresponding pixel-level labels with a size of 256×256 pixels.	Link
AIR-CD	Gaofen-2 PMS (4 m)	He et al., 2021	Contains 34 Gaofen-2 full images and the corresponding cloud labels distributed at different regions of China.	Link
HRC_WHU	Google Earth (0.5 m to 15 m)	Li et al., 2019	Comprises 150 globally distributed high-resolution images (0.5 m to 15 m resolution, three RGB channels) and the corresponding cloud masks.	Link

Open-Source Tools for Cloud and Cloud Shadow Detection

	Name	Applicable Images (Primarily)	References	Descriptions (Data and Method)	Link
Landsat	Fmask	Landsat 4-8 Sentinel-2	Zhu et al., 2012 & 2015	Mono-temporal Physical rule based	Link
	Tmask	Landsat 4-8	Zhu and Woodcock, 2014	Multi-temporal Temporal change based	Link
	MSScvm	Landsat MSS	Braaten et al., 2015	Multi-source Physical rule based	Link
	MFmask	Landsat 4-8	Qiu et al., 2017	Multi-source Physical rule based	Link
	MCM-GEE	Landsat-8	Mateo-García et al., 2018	Multi-temporal Temporal change based	Link
	Cloud-Net	Landsat-8	Mohajerani and Saeedi, 2019	Mono-temporal DL based	Link
	Cmask	Landsat-8	Qiu et al., 2020	Multi-temporal Temporal change based	Link
	DAGANS	Landsat-8 Proba-V	Mateo-Garcia et al., 2020	Mono-temporal DL based	Link
	FCNN	Landsats-8 Sentinel-2	López-Puigdollers et al., 2021	Mono-temporal DL based	Link
Sentinel-2	MAJA	Sentinel-2 VENμS Landsat-8	Hagolle et al., 2010	Multi-temporal Temporal change based	Link
	cB4S2	Sentinel-2	Hollstein et al., 2016	Mono-temporal Machine learning based	Link
	Sen2Cor	Sentinel-2	Main-Knorn et al., 2017	Mono-temporal Physical rule based	Link
	s2cloudless	Sentinel-2	Zupanc, 2017	Mono-temporal Machine learning based	Link
	FORCE	Sentinel-2 Landsat 4-8	Frantz et al., 2018	Mono-temporal Physical rule based	Link
	KappaMask	Sentinel-2	Domnich et al., 2021	Mono-temporal DL based	Link
	CD-FM3SF	Sentinel-2	Li et al., 2021	Mono-temporal DL based	Link
Gaofen	MFC	Gaofen-1 WFV	Li et al., 2017	Mono-temporal Physical rule based	Link
	GeoInfoNet	Gaofen-1 WFV	Wu et al., 2021	Mono-temporal DL based	Link
Others	N/A	HR images	Xie et al., 2017	Mono-temporal DL based	Link

Open-Source Cloud and Cloud Shadow Mask Products in Google Earth Engine

[1] Sentinel-2: Cloud Probability. [Link]

[2] Sentinel-2: Cloud Score+. [Link]

References

• Aybar, C., Ysuhuaylas, L., Loja, J., Gonzales, K., Herrera, F., Bautista, L., Yali, R., Flores, A., Diaz, L., Cuenca, N., Espinoza, W., Prudencio, F., Llactayo, V., Montero, D., Sudmanns, M., Tiede, D., Mateo-García, G., Gómez-Chova, L., 2022. CloudSEN12, a global dataset for semantic understanding of cloud and cloud shadow in Sentinel-2. Scientific Data 9. https://doi.org/10.1038/s41597-022-01878-2
• Baetens, L., Desjardins, C., Hagolle, O., 2019. Validation of copernicus Sentinel-2 cloud masks obtained from MAJA, Sen2Cor, and FMask processors using reference cloud masks generated with a supervised active learning procedure. Remote Sensing 11, 1–25. https://doi.org/10.3390/rs11040433
• Baetens, L., Hagolle, O., 2018. Sentinel-2 reference cloud masks generated by an active learning method [Data set]. https://doi.org/10.5281/zenodo.1460961
• Braaten, J.D., Cohen, W.B., Yang, Z., 2015. Automated cloud and cloud shadow identification in Landsat MSS imagery for temperate ecosystems. Remote Sensing of Environment 169, 128–138. https://doi.org/10.1016/j.rse.2015.08.006
• Domnich, M., Sünter, I., Trofimov, H., Wold, O., Harun, F., Kostiukhin, A., Jarveoja, M., Veske, M., Tamm, T., Voormansik, K., Olesk, A., Boccia, V., Longepe, N., Cadau, E.G., 2021. KappaMask: AI-Based Cloudmask Processor for Sentinel-2. Remote Sensing 13, 4140. https://doi.org/10.3390/rs13204100
• Foga, S., Scaramuzza, P.L., Guo, S., Zhu, Z., Dilley, R.D., Beckmann, T., Schmidt, G.L., Dwyer, J.L., Joseph Hughes, M., Laue, B., 2017. Cloud detection algorithm comparison and validation for operational Landsat data products. Remote Sensing of Environment 194, 379–390. https://doi.org/10.1016/j.rse.2017.03.026
• Francis, A., Mrziglod, J., Sidiropoulos, P., Muller, J.-P., 2020. Sentinel-2 Cloud Mask Catalogue [Data set]. Zenodo. https://doi.org/10.5281/zenodo.4172871
• Frantz, D., Ha?, E., Uhl, A., Stoffels, J., Hill, J., 2018. Improvement of the Fmask algorithm for Sentinel-2 images: Separating clouds from bright surfaces based on parallax effects. Remote Sensing of Environment 215, 471–481. https://doi.org/10.1016/j.rse.2018.04.046
• Hagolle, O., Huc, M., Pascual, D.V., Dedieu, G., 2010. A multi-temporal method for cloud detection, applied to FORMOSAT-2, VENμS, LANDSAT and SENTINEL-2 images. Remote Sensing of Environment 114, 1747–1755. https://doi.org/10.1016/j.rse.2010.03.002
• He, Q., Sun, X., Yan, Z., Fu, K., 2021. DABNet: Deformable Contextual and Boundary-Weighted Network for Cloud Detection in Remote Sensing Images. IEEE Transactions on Geoscience and Remote Sensing 1–16. https://doi.org/10.1109/TGRS.2020.3045474
• Hollstein, A., Segl, K., Guanter, L., Brell, M., Enesco, M., 2016. Ready-to-use methods for the detection of clouds, cirrus, snow, shadow, water and clear sky pixels in Sentinel-2 MSI images. Remote Sensing 8, 1–18. https://doi.org/10.3390/rs8080666
• Hughes, M.J., Hayes, D.J., 2014. Automated detection of cloud and cloud shadow in single-date Landsat imagery using neural networks and spatial post-processing. Remote Sensing 6, 4907–4926. https://doi.org/10.3390/rs6064907
• Ji, S., Dai, P., Lu, M., Zhang, Y., 2021. Simultaneous Cloud Detection and Removal from Bitemporal Remote Sensing Images Using Cascade Convolutional Neural Networks. IEEE Transactions on Geoscience and Remote Sensing 59, 732–748. https://doi.org/10.1109/TGRS.2020.2994349
• Li, J., Wu, Z., Hu, Z., Jian, C., Luo, S., Mou, L., Zhu, X.X., Molinier, M., 2021. A Lightweight Deep Learning-Based Cloud Detection Method for Sentinel-2A Imagery Fusing Multiscale Spectral and Spatial Features. IEEE Transactions on Geoscience and Remote Sensing 1–19. https://doi.org/10.1109/TGRS.2021.3069641
• Li, Y., Chen, W., Zhang, Y., Tao, C., Xiao, R., Tan, Y., 2020. Accurate cloud detection in high-resolution remote sensing imagery by weakly supervised deep learning. Remote Sensing of Environment 250. https://doi.org/10.1016/j.rse.2020.112045
• Li, Z., Shen, H., Cheng, Q., Liu, Y., You, S., He, Z., 2019. Deep learning based cloud detection for medium and high resolution remote sensing images of different sensors. ISPRS Journal of Photogrammetry and Remote Sensing 150, 197–212. https://doi.org/10.1016/j.isprsjprs.2019.02.017
• Li, Z., Shen, H., Li, H., Xia, G., Gamba, P., Zhang, L., 2017. Multi-feature combined cloud and cloud shadow detection in GaoFen-1 wide field of view imagery. Remote Sensing of Environment 191, 342–358. https://doi.org/10.1016/j.rse.2017.01.026
• Lin, D., Xu, G., Wang, X., Wang, Y., Sun, X., Fu, K., 2019. A Remote Sensing Image Dataset for Cloud Removal. arXiv preprint arXiv:1901.00600.
• López-Puigdollers, D., Mateo-García, G., Gómez-Chova, L., 2021. Benchmarking deep learning models for cloud detection in landsat-8 and sentinel-2 images. Remote Sensing 13, 1–20. https://doi.org/10.3390/rs13050992
• Main-Knorn, M., Pflug, B., Louis, J., Debaecker, V., Müller-Wilm, U., Gascon, F., 2017. Sen2Cor for Sentinel-2, in: Bruzzone, L., Bovolo, F., Benediktsson, J.A. (Eds.), Image and Signal Processing for Remote Sensing XXIII. SPIE, p. 3. https://doi.org/10.1117/12.2278218
• Mateo-García, G., Gómez-Chova, L., Amorós-López, J., Mu?oz-Marí, J., Camps-Valls, G., 2018. Multitemporal cloud masking in the Google Earth Engine. Remote Sensing 10, 7–9. https://doi.org/10.3390/rs10071079
• Mateo-Garcia, G., Laparra, V., Lopez-Puigdollers, D., Gomez-Chova, L., 2020. Cross-Sensor Adversarial Domain Adaptation of Landsat-8 and Proba-V Images for Cloud Detection. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing 14, 747–761. https://doi.org/10.1109/JSTARS.2020.3031741
• Mohajerani, S., Saeedi, P., 2019. Cloud-Net: An End-To-End Cloud Detection Algorithm for Landsat 8 Imagery, in: International Geoscience and Remote Sensing Symposium (IGARSS). IEEE, pp. 1029–1032. https://doi.org/10.1109/IGARSS.2019.8898776
• Qiu, S., He, B., Zhu, Z., Liao, Z., Quan, X., 2017. Improving Fmask cloud and cloud shadow detection in mountainous area for Landsats 4–8 images. Remote Sensing of Environment 199, 107–119. https://doi.org/10.1016/j.rse.2017.07.002
• Qiu, S., Zhu, Z., Woodcock, C.E., 2020. Cirrus clouds that adversely affect Landsat 8 images: What are they and how to detect them? Remote Sensing of Environment 246, 111884. https://doi.org/10.1016/j.rse.2020.111884
• Scaramuzza, P.L., Bouchard, M.A., Dwyer, J.L., 2012. Development of the landsat data continuity mission cloud-cover assessment algorithms. IEEE Transactions on Geoscience and Remote Sensing 50, 1140–1154. https://doi.org/10.1109/TGRS.2011.2164087
• Shendryk, Y., Rist, Y., Ticehurst, C., Thorburn, P., 2019. Deep learning for multi-modal classification of cloud, shadow and land cover scenes in PlanetScope and Sentinel-2 imagery. ISPRS Journal of Photogrammetry and Remote Sensing 157, 124–136. https://doi.org/10.1016/j.isprsjprs.2019.08.018
• Stillinger, T., Collar, N., 2019. Snow-Cloud Validation Masks for Multispectral Satellite Data [Data set]. https://doi.org/10.5281/zennodo.3240937
• Sun, H., Li, L., Xu, M., Li, Q., Huang, Z., 2020. Using Minimum Component and CNN for Satellite Remote Sensing Image Cloud Detection. IEEE Geoscience and Remote Sensing Letters 1–5. https://doi.org/10.1109/LGRS.2020.3014358
• USGS., 2016a. L7 Irish Cloud Validation Masks. U.S. Geological Survey data release. https://landsat.usgs.gov/landsat-7-cloud-cover-assessment-validation-data.
• USGS., 2016b. L8 Biome Cloud Validation Masks. U.S. Geological Survey, data release. https://landsat.usgs.gov/landsat-8-cloud-cover-assessment-validation-data.
• USGS., 2016c. L8 SPARCS Cloud Validation Masks. U.S. Geological Survey data release. https://www.usgs.gov/core-science-systems/nli/landsat/spatial-procedures-automated-removal-cloud-and-shadow-sparcs.
• Wu, X., Shi, Z., Zou, Z., 2021. A geographic information-driven method and a new large scale dataset for remote sensing cloud/snow detection. ISPRS Journal of Photogrammetry and Remote Sensing 174, 87–104. https://doi.org/10.1016/j.isprsjprs.2021.01.023
• Xie, F., Shi, M., Shi, Z., Yin, J., Zhao, D., 2017. Multilevel cloud detection in remote sensing images based on deep learning. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing 10, 3631–3640. https://doi.org/10.1109/JSTARS.2017.2686488
• Zhu, S., Li, Z., Shen, H., 2024. Transferring Deep Models for Cloud Detection in Multisensor Images via Weakly Supervised Learning. IEEE Transactions on Geoscience and Remote Sensing 62, 5609518. https://doi.org/10.1109/TGRS.2024.3358824
• Zhu, Z., Wang, S., Woodcock, C.E., 2015. Improvement and expansion of the Fmask algorithm: Cloud, cloud shadow, and snow detection for Landsats 4-7, 8, and Sentinel 2 images. Remote Sensing of Environment 159, 269–277. https://doi.org/10.1016/j.rse.2014.12.014
• Zhu, Z., Woodcock, C.E., 2014. Automated cloud, cloud shadow, and snow detection in multitemporal Landsat data: An algorithm designed specifically for monitoring land cover change. Remote Sensing of Environment 152, 217–234. https://doi.org/10.1016/j.rse.2014.06.012
• Zhu, Z., Woodcock, C.E., 2012. Object-based cloud and cloud shadow detection in Landsat imagery. Remote Sensing of Environment 118, 83–94. https://doi.org/10.1016/j.rse.2011.10.028
• Zupanc, A., 2017. Improving cloud detection with machine learning. https://medium.com/sentinel-hub/improving-cloud-detection-with-machine-learning-c09dc5d7cf13.