Vol. 39, issue 02, article # 9

Abstract:

This article addresses the problem of cloud detection in hyperspectral satellite images using an interpretable neural network classifier for partial cloudiness. For effective solution, the preliminary selection of spectral channels and derived features is performed using decision trees trained with labeled satellite data of the HYPERION sensor. The selected channels and features are then used for building a convolutional neural network based on a modified Unet architecture. Modifications to the original Unet architecture enable simplifying the network structure, avoiding overfitting, assessing the importance of spatial and spectral features, analyzing classification results, and explaining decision-making processes. Feature selection and evaluation of their importance are critical stages in developing adequate machine learning and deep learning models combined with the analysis of their generalization ability. The suggested feature selection method is based on iterative training of decision trees to identify significant features in terms of classification accuracy. The operation of the convolutional neural network is interpreted and the importance of spatial and spectral features is assessed by evaluating Shapley vectors. The results of testing a neural network with HYPERION images made over three surface types (ocean, vegetation, and urbanized territory) are presented; its accuracy and commission and omission errors are estimated. The model enables semantic segmentation of images with thin clouds with accuracy over 95% in selected spectral bands and with selected features. The importance of input features, caused by their distribution across spectral channels and the relative positions of pixels in an image, for the detection of thick and thin clouds in hyperspectral satellite images is analyzed. The presented neural network model is designed for working with limited data volumes, enables applying augmentation, and can be used to assess the importance of selected spectral channels and spatial features.

Keywords:

cloud detection, feature selection semantic segmentation, interpretable machine learning model

References:

1. Nikolaeva O.V. Algoritm obnaruzheniya oblachnykh ob"ektov na mnogospektral'nykh snimkakh // Komp'yuternaya optika. 2022. V. 46, N 5. P. 808–817. DOI: 10.18287/2412-6179-CO-1076.
2. Bloshchinskii V.D., Kramareva L.S., Shamilova Yu.A. Detektirovanie oblachnogo pokrova s ispol'zovaniem neironnoi seti po dannym pribora MSU-GS kosmicheskogo apparata «ARKTIKA-M» N 1 // Optika atmosf. i okeana. 2024. V. 37, N 2. P. 99–104. DOI: 10.15372/AOO20240202; Bloshchinskiy V.D., Kramareva L.S., Shamilova Yu.A. Cloud cover detection using a neural network based on MSU-GS instrument data of Arktika-M No. 1 satellite // Atmos. Ocean. Opt. 2024. V. 37, N 3. P. 394–399.
3. Li Z., Shen H., Li H., Xid G., Gamba P., Zhang L. Multi-feature combined cloud and cloud-shadow detection in Gaofen-1 widefield of view imagery // Remote Sens. Environ. 2017. V. 191. P. 342–358. DOI: 10.1016/j.rse.2017. 01.026.
4. Bo P., Fenzhen S., Yunshan M. A cloud and cloud shadow detection methods based on fuzzy c-means algorithm // IEEE J Sel Top Appl Earth Obs Remote Sens. 2020. V. 13. P. 1714–1727. DOI: 10.1109/JSTARS.2020.2987844.
5. Astafurov V.G., Skorokhodov A.V. Klassifikatsiya oblakov po sputnikovym snimkam na osnove tekhnologii neironnykh setei // Sovremennye problemy distantsionnogo zondirovaniya Zemli iz kosmosa. 2011. V. 8, N 1. P. 65–72.
6. Sun L., Mi X., Wei J., Wang J., Tian X., Yu H., Gan P. A cloud detection algorithm generating method for remote sensing data at visible to short-wave infrared wavelengths // ISPRS J. Photogramm. Remote Sens. 2017. V. 125, N D24. P. 70–88. DOI: 10.1016/j. isprsjprs.2016.12.005.
7. Sun L., Wei J., Wang J., Mi X., Guo Y., Lv Y., Yang Y., Gan P., Zhou X., Jio C., Jiawei C., Tian X. A universal dynamic threshold cloud detection algorithm (UNSADA) supported by a prior surface // J. Geophys. Res. Atm. 2016. V. 121, N 12. P. 7172–7196. DOI: 10.1002/2015JD024722.
8. Mateo-Garcia G., Gomez-Chova L., Amoros-Lopez J., Munoz-Mari J., Camps-Valls G. Multitemporal cloud masking in the Google Earth Engine // Remote Sens. 2018. V. 10, N 7. P. 1079. DOI: 10.3390/rs10071079.
9. Lyapustin A., Wang Y., Frey R. An automatic cloud mask algorithm based on time series of MODIS measurements // J. Geophys. Res. 2008. V. 113. DOI: 10.1029/2007JD009641.
10. Bian J., Li A., Liu Q., Huang C. Cloud and snow discrimination for CCD images of HJ-1A/B constellation based on spectral signature and spatio-temporal context // Remote Sens. 2016. V. 8, N 1. P. 31. DOI: 10.3390/rs8010031.
11. Belov A.M., Denisova A.Yu. Algoritm vyyavleniya sluchainykh iskazhenii v sostave stseny na serii raznovremennykh izobrazhenii DZZ odnoi i toi zhe territorii // Komp'yuternaya optika. 2019. V. 43, N 5. P. 869–885. DOI: 10.18287/2412-6179-2019-43-5-869-885.
12. Hagolle O., Huc M., Villa Pascual D., Dedieu G. A multi-temporal method for cloud detection, applied to Formosat-2, VeNμS, Landsat and Sentinel-2 images // Remote Sens. Environ. 2010. V. 114, N 8. P. 1747–1755. DOI: 10.1016/j.rse.2010.03.002.
13. Zhu X., Helmer E.H. An automatic method for screening clouds and cloud shadows in optical satellite image time series in cloudy region // Remote Sens. Environ. 2018. V. 214. P. 135–153. DOI: 10.1016/j.rse.2018. 05.024.
14. Vizil'ter Yu.V., Gorbatsevich V.S., Zheltov S.Yu. Strukturno-funktsional'nyi analiz i sintez glubokikh konvolyutsionnykh neironnykh setei // Komp'yuternaya optika. 2019. V. 43, N 5. P. 886–900. DOI: 10.18287/2412-6179-2019-43-5-886-900.
15. Shendryk Y., Rist Y., Ticehurst C., Thorburn P. Deep learning for multi-modal classification of cloud, shadow and land cover scenes in PlanetScope and Sentinel-2 imagery // ISPRS J. Photogramm. Remote Sens. 2019. V. 157. P. 124–136. DOI: 10.1016/j.isprsjprs.2019. 08.018.
16. Andreev A.I., Shamilova Yu.A. Detektirovanie oblachnosti po dannym KA HIMAWARI-8 s primeneniem svertochnoi neironnoi seti // Issled. Zemli iz kosmosa. 2021. N 2. P. 42–52. DOI: 10.31857/S0205961421010036.
17. Wang L., Chen Y., Tang L., Fan R., Yao Y. Object-based convolutional neural networks for cloud and snow detection in high-resolution multispectral imagers // MDPI Sens. 2018. V. 10, N 11. P. 1666. DOI: 10.3390/ w10111666.
18. Liu H., Zeng D., Tian Q. Super-pixel cloud detection using hierarchical fusion CNN // IEEE Fourth International Conference on Multimedia Big Data (BigMM) 2018. Xi'an, China. P. 1–6. DOI: 10.1109/BigMM. 2018.8499091.
19. Fu H., Shen Y., Liu J., He G., Chen J., Liu P., Qian J., Li J. Cloud detection for FY meteorology satellite based on ensemble thresholds and random forests approach // Remote Sens. 2019. V. 11, N 1. P. 44. DOI: 10.3390/rs11010044.
20. Ghasemian N., Akhoondzadeh M. Integration of VIR and thermal bands for cloud, snow/ice and thin cirrus detection in MODIS satellite images // Proc. of the Third International Conference Intelligent Decision Science. Tehran, Iran. 2018. P. 1–37.
21. Dostovalova A.M., Gorshenin A.K., Starichkova Yu.V., Arzamasov K.M. Sravnitel'nyi analiz modifikatsii neirosetevykh arkhitektur U-Net v zadache segmentatsii meditsinskikh izobrazhenii // Digital Diagnostics. 2024. V. 5, N 4. P. 833–853. DOI: 10.17816/DD629866.
22. Ronneberger O., Fischer P., Brox T. U-Net: Convolutional networks for biomedical image segmentation // Medical Image Computing and Computer-Assisted Intervention (MICCAI). 2015. V. 9351. P. 234–241. DOI: 10.1007/978-3-319-24574-4_28.
23. Mishra P. Ob"yasnimye modeli iskusstvennogo intellekta na Python. M.: DMK Press, 2022. 298 p.
24. Gilpin L., Bau D., Yuan B., Bajwa A., Specter M., Kagal L. Explaining explanations: An overview of interpretability of machine learning // IEEE 5th International Conference on Data Science and Advanced Analytics (DSAA) 2018. Turin. Italy. P. 80–89. DOI: 10.1109/DSAA.2018.00018.
25. Minkin A.S., Nikolaeva O.V. Raspoznavanie oblakov na giperspektral'nykh sputnikovykh izobrazheniyakh s ispol'zovaniem ob"yasnimoi modeli mashinnogo obucheniya // Optika atmosf. i okeana. 2024. V. 37, N 2. P. 149–157. DOI: 10.15372/AOO20240209; Minkin A.S., Nikolaeva O.V. Cloud recognition in hyperspectral satellite images using an explainable machine learning model // Atmos. Ocean. Opt. 2024. V. 37, N 3. P. 400–408.
26. Štrumbelj E., Kononenko I. Explaining prediction models and individual predictions with feature contributions // Knowl. Inf. Syst. 2014. V. 41. P. 647–665. DOI: 10.1007/s10115-013-0679-x.
27. Lundberg S.M., Su-In L. . A unified approach to interpreting model predictions // arXiv.1705.07874. 2017. 10 p.
28. Minkin A.S., Nikolaeva O.V., Russkov A.A. Czhatie giperspektral'nykh dannykh metodom glavnykh komponent // Komp'yuternaya optika. 2021. V. 45, N 2. P. 235–244. DOI: 10.18287/2412-6179-CO-806.
29. Tsyrul'nikov M.D., Gaifulin D.R., Svirenko P.I., Uspenskii A.B. Usvoenie dannykh sputnikovykh meteorologicheskikh nablyudenii v Gidromettsentre Rossii // Meteorol. i gidrol. 2021. N 12. P. 80–93. DOI: 10.52002/0130-2906-2021-12-80-93.
30. Iguyon I., Elisseeff A. An introduction to variable and feature selection // J. Mach. Learn. Res. 2003. V. 3. P. 1157–1182. DOI: 10.1162/153244303322753616.
31. Huang S., Tang L., Hupy J., Wang Y., Shao G. A commentary review on the use of normalized difference vegetation index (NDVI) in the era of popular remote sensing // J. Forestly Res. 2020. V. 32, N 5. P. 1–6. DOI: 10.1007/s11676-020-01155-1.
32. Jin D., Lee K.-S., Choi S., Seong N.-H., Jung D., Sim S., Woo J., Jeon U., Yugyeong B., Han K.-S. An improvement of snow/cloud discrimination from machine learning using geostationary satellite data // Int. J. Digital Earth. 2022. V. 15. P. 2355–2375. DOI: 10.1080/17538947.2022.2152886.
33. McFeeters S.K. The use of the Normalized Difference Water Index (NDWI) in the delineation of open water features // Int. J. Remote Sens. 1996. V. 17, N 7. P. 1425–1432. DOI: 10.1080/01431169608948714.
34. Gao B.-C. NDWI – a normalized difference water index for remote sensing of vegetation liquid water from space // Remote Sens. Environ. 1996. V. 58. P. 257–266. DOI: 10.1016/s0034-4257(96)00067-3.
35. Hastie T., Tibshirani R., Friedman J. Additive models, trees, and related methods // The Elements of Statistical Learning. Chapter 9. // The Elements of Statistical Learning. New York: Springer, 2009. P. 295–336. DOI: 10.1007/978-0-387-84858-7_9.
36. Breiman L., Friedman J.H., Olshen R.A. Classification and Regression Trees. New York: Chapman and Hall, 1984. DOI: 10.1201/9781315139470.
37. Kruse F.A., Boardman J.W., Huntington J.F. Comparison of airborne hyperspectral data and EO-1 Hyperion for mineral mapping // IEEE Trans. Geosci. Remote Sens. 2003. V. 41, N 6. P. 1388–1400. DOI: 10.1109/ TGRS.2003.812908.
38. Bryus P., Bryus E., Gedek P. Prakticheskaya statistika dlya spetsialistov Data Science. SPb.: BKHV-Peterburg, 2021. 346 p.
39. Ribeiro M.T., Singh S., Guestrin C. Why should i trust you? explaining the predictions of any classifier // Proc. of the 22nd ACM SIGKDD International Conference Knowledge Discovery and Data Mining. 2016. P. 1135–1144. DOI: 10.1145/2939672.2939778.
40. Bol’on-Canedo V., Alonso-Betanzos A. Ensembles for feature selection: A review and future trends // Information Fusion. 2019. V. 52. P. 1–12. DOI: 10.1016/J.INFFUS.2018.11.008.
41. Igantov D., Kwuida L. On Shapley value interpretability in concept-based learning with formal concept analysis // Ann. Math. Artif. Intell. 2022. V. 90. P. 1197–1222. DOI: 10.1007/s10472-022-09817-y.
42. Aas K., Jullum M., Løland A. Explaining individual predictions when features are dependent: More accurate approximations to Shapley values // Artif. Intell. 2021. V. 298. P. 103502. DOI: 10.1016/j.artint.2021.103502.