Vol. 37, issue 02, article # 10

Abstract:

Problem of developing algorithm based upon neutral networks and machine learning to find clouds on hyperspectral images are under consideration. It is required that the network is not a "black box," but allows an analysis of the reasons for decision making and classification results. Presented hybrid model includes decision tree trained to overcast recognition (model 1) on pre-selected features of an image in combination with convolutional neural network (model 2). Model 2 uses the result of model 1 and brightness in a selected band of an image. Model 1 finds cloud cores, and model 2 finds cloud edges. Results of testing the hybrid model on data of HYPERION sensor are presented. Data obtained over three surface types (ocean, plant, and urban region) are considered. Overall accuracy, as well as commission and omission errors are assessed. It is shown that the hybrid model can find 85% cloud pixels, only if the neural network is trained on an image where the contrast attains a maximum in the same spectral band. The results of this work can be applied to solve the general problem of analyzing and processing multispectral satellite images and further in environmental science and monitoring of changes in vegetation, ocean and glaciers.

Keywords:

multispectral satellite image, cloud detection, spectral index, machine learning model, convolutional neural network, explainable model

References:

1. Li Z., Shen H., Li H., Xid G., Gamba P., Zhang L. Multi-feature combined cloud and cloud-shadow detection in Gaofen-1 widefield of view imagery // Remote Sens. Environ. 2017. V. 191. P. 342–358. DOI: 10.1016/j.rse.2017.01.026.
2. Bo P., Fenzhen S., Yunshan M. A cloud and cloud shadow detection methods based on fuzzy c-Means algorithm // IEEE J. Sel. Top. Appl. Earth. Obs. Remote Sens. 2020. V. 13. P. 1714–1727. DOI: 10.1109/JSTARS.2020.2987844.
3. Sun L., Mi X., Wei J., Wang J., Tian X., Yu H., Gan P. A cloud detection algorithm generating method for remote sensing data at visible to short-wave infrared wavelengths // ISPRS J. Photogramm. 2017. V. 125, N D24. P. 70–88. DOI: 10.1016/j.isprsjprs.2016.12.005.
4. Sun L., Wei J., Wang J., Mi X., Guo Y., Lv Y., Yang Y., Gan P., Zhou X., Jio C., Jiawei C., Tian X. A universal dynamic threshold cloud detection algorithm (UNSADA) supported by a prior surface // J. Geophys. Res.: Atmos. 2016. V. 121, N. 12. P. 7172–7196. DOI: 10.1002/2015JD024722.
5. Mateo-Garcia G., Gomez-Chova L., Amoros-Lopez J., Munoz-Mari J., Camps-Valls G. Multitemporal cloud masking in the Google Earth Engine // Remote Sens. 2018. V. 10, N 7. P.1079. DOI: 10.3390/rs10071079.
6. Lyapustin A., Wang Y., Frey R. An automatic cloud mask algorithm based on time series of MODIS measurements // J. Geophys. Res. 2008. V. 113. P. D16207. DOI: 10.1029/2007JD009641.
7. Bian J., Li A., Liu Q., Huang C. Cloud and snow discrimination for CCD images of HJ-1A/B constellation based on spectral signature and spatio-temporal context // Remote Sens. 2016. V. 8, N 31. DOI: 10.3390/rs8010031.
8. Belov A.M., Denisova A.Yu. Algoritm vyyavleniya sluchainykh iskazhenii v sostave stseny na serii raznovremennykh izobrazhenii DZZ odnoi i toi zhe territorii // Komp'yuternaya optika. 2019. V. 43, N 5. P. 869–885. DOI: 10.18287/2412-6179-2019-43-5-869-885.
9. Hagolle O., Huo M., Villa Pascual D., Dedieu G. A multi-temporal method for cloud detection, applied to Formosat-2, VeNmS, Landsat, and Sentinel-2 images // Remote Sens. Environ. 2010. V. 114, N 8. P. 1747–1755. DOI: 10.1016/j.rse.2010.03.002.
10. Zhu X., Helmer E.H. An automatic method for screening clouds and cloud shadows in optical satellite image time series in cloudy region // RSE. 2018. V. 214. P. 135–153. DOI: 10.1016/j.rse.2018.05.024.
11. Vizil'ter Yu.V., Gorbatsevich V.S., Zheltov S.Yu. Strukturno-funktsional'nyi analiz i sintez glubokikh konvolyutsionnykh neironnykh setei // Komp'yuternaya optika. 2019. V. 43, N 5. P. 886–900. DOI: 10.18287/2412-6179-2019-43-5-886-900.
12. Shendryk Y., Rist Y., Ticehurst C., Thorburn P. Deep learning for multi-modal classification of cloud, shadow and land cover scenes in PlanetScope and Sentinel-2 imagery // ISPRS J. Photogramm. 2019. V. 157. P. 124–136. DOI: 10.1016/j.isprsjprs.2019.08.018.
13. Andreev A.I., Shamilova Yu.A. Detektirovanie oblachnosti po dannym KA HIMAWARI-8 s primeneniem svertochnoi neironnoi seti // Issled. Zemli iz kosmosa. 2021. N 2. P. 42–52. DOI: 10.31857/S0205961421010036.
14. Zheng M, Tang W., Zhao X. Hyperparameter optimization of neural network-driven spatial models accelerated using cyber-enabled high-performance computing // Int. J. Geogr. Inf. Sci. 2019. V. 33. P. 314–345. DOI: 10.1080/13658816.2018.1530355.
15. Fu H., Shen Y., Liu J, He G., Chen J., Liu P., Qian J., Li J. Cloud detection for FY meteorology satellite based on ensemble thresholds and random forests approach // Remote Sens. 2019. V. 11, N 1. P. 44. DOI: 10.3390/rs11010044.
16. Ghasemian N., Akhoondzadeh M. Integration of VIR and thermal bands for cloud, snow/ice and thin cirrus detection in MODIS satellite images // Proc. of the Third International Conference on Intelligent Decision Science, Tehran, Iran, May 16–18. 2018. Р. 1–37.
17. Liu H., Zeng D., Tian Q. Super-pixel cloud detection using hierarchical fusion CNN // Proc. of the 2018 IEEE Fourth International Conference on Multimedia Big Data. 2018. P. 1–6. DOI: 10.1109/BigMM.2018.8499091.
18. Wang L., Chen Y., Tang L., Fan R., Yao Y. Object-based convolutional neural networks for cloud and snow detection in high-resolution multispectral imagers // Water. 2018. V. 10, N 11. P. 1666. DOI: 10.3390/w10111666.
19. Gilpin L., Bau D., Yuan B., Bajwa A., Specter M., Kagal L. Explaining explanations: An overview of interpretability of machine learning // IEEE 5th International Conference on Data Science and Advanced Analytics (DSAA) 2018, Turin, Italy. P. 80–89. DOI: 10.1109/DSAA.2018.00018.
20. Štrumbelj E., Kononenko I. Explaining prediction models and individual predictions with feature contributions // Knowl. Inf. Syst. 2014. V. 41. P. 647–665.
21. Goodwin N.R., Collet L.J., Denham R.J., Flood N., Tindall D. Cloud and cloud shadow screening across Queensland, Australia: An automated method for LandsatTM/ETA + time-series // Remote Sens. Environ. 2013. V. 134. P. 50–65. DOI: 10.1016/j.rse.2013.02.019.
22. Mishra P. Ob"yasnimye modeli iskusstvennogo intellekta na Python. M.: DMK-Press, 2022. 298 p.
23. Hastie T., Tibshirani R., Friedman J. Additive models, trees, and related methods // The Elements of Statistical Learning. Springer, 2009. P. 295–336.
24. Chollet F. Xception: Deep learning with depthwise separable convolutions // IEEE Conference on Computer Vision and Pattern Recognition (CVPR), USA, 2017. P. 1800–1807. DOI: 10.1109/CVPR.2017.195.