Vol. 38, issue 05, article # 1

Abstract:

Quantitative analysis of the gas mixture absorption spectra is complicated by noise. The parameters of standard filters are related to the entire analyzed spectral range. This means that the filter parameters being optimal for strong absorption lines are not optimal for weak absorption lines and vice versa. An approach to create adaptive filter for denoising experimental spectra based on the combination of a windowed version of a standard filter with the independent component analysis is suggested and implemented with the Savitzky–Golay filter as an example. The numerical simulation was carried out at normal conditions for the absorption spectra of the model of mid-latitude summer atmosphere in the 100–1000 GHz spectral range. The efficiency of the suggested adaptive and the standard versions of Savitzky–Golay filter was compared using a quantitative criterion of the proximity between two spectral curves. Experimental validation of efficiency of the suggested adaptive Savitzky–Golay filter was conducted on the example of 200 ppm SO₂ and 10000 ppm H₂O gas mixture. The SO₂ concentration was evaluated using multivariate curve resolution method. The relative error in the concentration retrieved after noise reduction by this filter was 3.7 times less compared to the standard Savitzky–Golay filter. Thus, the suggested adaptive Savitsky–Goley filter makes it possible to increase the efficiency of noise suppression in experimental spectral data.

Keywords:

IR and terahertz molecular absorption spectroscopy, adaptive spectral filter, Savitzky–Golay filter

Figures:

References:

1. Cox J.A. Signal-to-noise ratio dependence on frame time, time delay and integration (TDI), and pulse shaping // Opt. Eng. 1982. V. 21, N 3. P. 528–536. DOI: 10.1117/12.7972941.
2. Barnes J.A, Chi A.R., Cutler L.S., Healey D.J., Leeson D.B., Mcgunigal T.E., Mullen J.A., Smith W.L., Sydnor R.L., Vessot R.F.C., Winkler G.M.R. Characterization of frequency stability // IEEE T. Instrum. Meas. 1971. N 2. P. 105–120. DOI: 10.1109/tim.1971.5570702.
3. Fell A.F. Biomedical applications of derivative spectroscopy // TrAC-Trend. Anal. Chem. 1983. V. 2, N 3. P. 63–66. DOI: 10.1016/0165-9936(83)85010-9.
4. Jingsong Li., Benli Yu., Weixiong Zhao, Weidong Chen. A review of signal enhancement and noise reduction techniques for tunable diode laser absorption spectroscopy // Appl. Spectrosc. Rev. 2014. V. 49, N 8. P. 666–691. DOI: 10.1080/05704928.2014.903376.
5. Zhang L., Li Y., Wei Y., Wang Z., Zhang T., Gong W., Zhang Q. Enhancement of direct absorption spectroscopy utilizing an improved particle swarm algorithm // Photonics. 2022. V. 9, N 6. P. 412. DOI: 10.3390/photonics9060412.
6. Chabuda A., Durka P., Żygierewicz J. High frequency SSVEP-BCI with hardware stimuli control and phase-synchronized comb filter // IEEE T. Neur. Sys. Reh. 2017. V. 26, N 2. P. 344–352. DOI: 10.1109/TNSRE.2017.2734164.
7. Martinek R., Zidek J. Use of adaptive filtering for noise reduction in communications systems // International Conference on Applied Electronics. Pilsen, Czech Republic. 2010. P. 1–6.
8. Lita I., Visan D.A., Oprea S., Cioc B.I. Hardware design for noise reduction in data acquisition modules // 30th International Spring Seminar on Electronics Technology. 2007. P. 462–466. DOI: 10.1109/ISSE.2007.4432900.
9. Thenua R.K., Agrawal S.K. Hardware implementation of adaptive algorithms for noise cancellation // Int. J. Inf. Elect. Eng. 2012. V. 2, N 2. P. 1–4.
10. Li J., Yu B., Zhao W., Chen W.A. Review of signal enhancement and noise reduction techniques for tunable diode laser absorption spectroscopy // Appl. Spectrosc. Rev. 2014. V. 49, N 8. P. 666–691. DOI: 10.1080/05704928.2014.903376.
11. Gusheng Zhang, He Hao, Yichen Wang, Ying Jiang, Jinhui Shi, Jing Yu, Xiaojuan Cui, Jingsong Li, Sheng Zhou, Benli Yu. Optimized adaptive Savitzky–Golay filtering algorithm based on deep learning network for absorption spectroscopy // Spectrochim. Acta A: 2021. V. 263. P. 120187. DOI: 10.1016/j.saa.2021.120187.
12. Tanji Jr A.K., de Brito M.A., Alves M.G., Garcia R.C., Chen G.L., Ama N.R. Improved noise cancelling algorithm for electrocardiogram based on moving average adaptive filter // Electronics. 2021. V. 10, N 19. P. 2366. DOI: 10.3390/electronics10192366.
13. Azami H., Mohammadi K., Bozorgtabar B. An improved signal segmentation using moving average and Savitzky–Golay filter // J. Signal Inf. Proc. 2012. V. 3, N 1. P. 39–44. DOI: 10.4236/jsip.2012.31006.
14. Kumar A., Sodhi S.S. Comparative analysis of gaussian filter, median filter and denoise autoenocoder // 7th International Conference on Computing Sustainable Global Development. 2020. P. 45–51. DOI: 10.23919/INDIACom49435.2020.9083712.
15. Schmid M., Rath D., Diebold U. Why and how Savitzky–Golay filters should be replaced // ACS Measurement Science Au. 2022. V. 2, N 2. P. 185–196. DOI: 10.1021/acsmeasuresciau.1c00054.
16. Rinnan Å., Van Den Berg F., Engelsen S.B. Review of the most common pre-processing techniques for near-infrared spectra // TrAC-Trend. Anal. Chem. 2009. V. 28, N 10. P. 1201–1222. DOI: 10.1016/j.trac.2009.07.007.
17. Sadeghi M., Behnia F., Amiri R. Window selection of the Savitzky–Golay filters for signal recovery from noisy measurements // IEEE T. Instrum. Meas. 2020. V. 69, N 8. P. 5418–5427. DOI: 10.1109/TIM.2020.2966310.
18. Zimmermann B., Kohler A. Optimizing Savitzky–Golay parameters for improving spectral resolution and quantification in infrared spectroscopy // Appl. Spec. 2013. V. 67, N 8. P. 892–902. DOI: 10.1366/12-06723.
19. Kistenev Y.V., Shapovalov A.V., Vrazhnov D.A., Nikolaev V.V. Kalman filtering in the problem of noise reduction in the absorption spectra of exhaled air // Proc. SPIE. 2016. V. 10035. P. 72–77. DOI: 10.1117/12.2249139.
20. Kalambet Y., Maltsev S., Kozmin Y. Noise filtering: The ultimate solution? // Analytics. 2011. Т. 1, № 1. С. 50–56.
21. Dombi J., Dineva A. Adaptive Savitzky–Golay filtering and its applications // Int. J. Adv. Intell. Parad. 2020. V. 16, N 2. P. 145–156. DOI: 10.1504/IJAIP.2020.107011.
22. Leleux D.P., Claps R., Chen W., Tittel F.K., Harman T.L. Applications of Kalman filtering to realtime trace gas concentration measurements // Appl. Phys. 2002. V. 74. P. 85–93. DOI: 10.1007/s003400100751.
23. Hyvärinen A., Oja E. Independent component analysis: Algorithms and applications // Neur. Netw. 2000. V. 13, N 4–5. P. 411–430. DOI: 10.1016/S0893-6080(00)00026-5.
24. Altynbekov A.A., Borisov A.V., Skiba V.E., Kistenev Y.V. The possibility of increasing the efficiency of terahertz absorption spectra noise reduction using a sliding window variant of Savitzky–Golay filter // Proc. SPIE. 2023. V. 12920. P. 239–244. DOI: 10.1117/12.3009811.
25. Kistenev Y.V., Kuzmin D.A., Sandykova E.A., Shapovalov A.V. Quantitative comparison of the absorption spectra of the gas mixtures in analogy to the criterion of Pearson // Proc. SPIE, 2015. V. 9680. P. 797–804. DOI: 10.1117/12.2205606.
26. Gordon I.E., Rothman L.S., Hargreaves R.J., Hashemi R., Karlovets E.V., Skinner F.M., Conway E.K., Hill C., Kochanov R.V., Tan Y., Wcisło P., Finenko A.A., Nelson K., Bernath P.F., Birk M., Boudon V., Campargue A., Chance K.V., Coustenis A., Drouin B.J, Flaud J.-M., Gamache R.R., Hodges J.T., Jacquemart D., Mlaver E.J., Nikitin A.V., Perevalov V.I., Rotger M., Tennyson J., Toon G.C., Tran H., Tyuterev Vl.G., Adkins E.M., Baker A., Barbe A., Canè E., Császár A.G., Egorov O., Fleisher A.J., Fleurbaey H., Foltynowicz A., Furtenbacher T., Harrison J.J., Hartmann J.-M., Horneman V.-M., Huang X., Karman T., Karns J., Kassi S., Kleiner I., Kofman V., Kwabia-Tchana F., Lee T.J., Long D.A., Lukashevskaya A.A., Lyulin O.M., Makhnev V.Yu., Matt W., Massie S.T., Melosso M., Mikhailenko S.N, Mondelain D., Muller H.S.P., Naumenko O.V., Perrin A., Polyansky O.L., Raddaoui E., Raston P.L., Reed Z.D., Rey M., Richard , Tόbiás R., Sadiek I., Schwenke D.W., Starikova E., Sung K., Tamassia F., Tashkun S.A., Vander Auwera J., Vasilenko I.A., Vigasin A.A., Villanueva G.L., Vispoel B., Wagner G., Yachmenev A., Yurchenko S.N. The HITRAN2020 molecular spectroscopic database // J. Quant. Spectros. Radiat. Trasfer. 2022. V. 277. P. 107949.
27. Hitran on the web. URL: https://hitran.iao.ru/ (last access: 20.12.2024).
28. De Juan A., Tauler R. Multivariate Curve Resolution: 50 years addressing the mixture analysis problem – a review // Anal. Chim. Acta. 2021. V. 1145. P. 59–78. DOI: 10.1016/j.aca.2020.10.051.