Accuracy of COVID-19 Prediction Modeling Techniques

Authors

  • Panithee Thammawijaya Department of Disease Control, Ministry of Public Health, Thailand
  • Rapeepong Suphanchaimat Division of Epidemiology, Department of Disease Control, Ministry of Public Health, Thailand; International Health Policy Program, Ministry of Public Health, Thailand

DOI:

https://doi.org/10.59096/osir.v17i3.270877

Keywords:

COVID-19, forecast, predictive accuracy, compartmental model, ARIMA, LSTM

Abstract

The unprecedented impact of the COVID-19 pandemic has revealed that forecasting capability is critically needed in making strategic decisions and formulating reasonable countermeasures. This study aimed to assess the predictive accuracy in forecasting the numbers of COVID-19 cases using Thailand’s national COVID-19 surveillance database from January 2020– June 2021 based on three analytical models: a susceptible-exposed-infectious-recovery compartmental model, an auto-regressive integrated moving average model, and a long short-term memory (LSTM) network model. All forecasting methods had model parameters adjusted weekly according to the most recent situation and predictive accuracy measures, including the mean absolute percentage error (MAPE). We found that the MAPE values ranged from 19.65%–22.54%, 28.95%–32.35%, 47.78%–53.55%, and 75.03–84.91% for forecasting one, two, four, and eight weeks ahead, respectively. Among the three models, the LSTM model had slightly higher accuracy than the other two models within the same forecasting range. These prediction models can be used for short-range forecasts in other similar settings while long-range forecasting requires monitoring and updating periodically.

References

World Health Organization. The impact of COVID-19 on global health goals [Internet]. Geneva: World Health Organization; 2021 May 21 [cited 2024 Jul 21]. <https://www.who.int/news-room/spotlight/the-impact-of-covid-19-on-global-health-goals>

World Health Organization. The impact of COVID-19 on health and care workers: a closer look at deaths [Internet]. Gevena: World Health Organization; 2021 [cited 2024 Jul 21]. 18 p. <https://iris.who.int/handle/10665/345300>

World Health Organization. Statement on the fifteenth meeting of the IHR (2005) emergency committee on the COVID-19 pandemic [Internet]. Geneva: World Health Organization; 2023 May 5 [cited 2024 Aug 30]. <https://www.who.int/news/item/05-05-2023-statement-on-the-fifteenth-meeting-of-the-international-health-regulations-(2005)-emergency-committee-regarding-the-coronavirus-disease-(covid-19)-pandemic>

World Health Organization. WHO COVID-19 dashboard [Internet]. Geneva: World Health Organization; 2024 [cited 2024 Jul 21]. <https://data.who.int/dashboards/covid19/cases?n=c>

Mathematica. Surging the public health workforce: lessons learned from the COVID-19 response at state, tribal, local, and territorial public health agencies [Internet]. Washington (DC): Office of the Assistant Secretary for Planning and Evaluation; 2023 Sep [cited 2024 Jul 21]. <https://aspe.hhs.gov/sites/default/files/documents/da3419d752f6f8c0359a126a9045fefb/surging-the-ph-workforce.pdf>

Song X, Xiao J, Deng J, Kang Q, Zhang Y, Xu J. Time series analysis of influenza incidence in Chinese provinces from 2004 to 2011. Medicine. 2016 Jun;95(26):e3929. doi:10.1097/MD.0000000000003929.

Karasinghe N, Peiris S, Jayathilaka R, Dharmasena T. Forecasting weekly dengue incidence in Sri Lanka: Modified Autoregressive Integrated Moving Average modeling approach. PLOS ONE. 2024 Mar 8;19(3):e0299953. doi:10.1371/journal.pone.0299953.

Abdelaziz M, Ahmed A, Riad A, Abderrezak G, Djida A. Forecasting daily confirmed COVID-19 cases in Algeria using ARIMA models. East Mediterr Health J. 2023 Jul 31;29(7):515–9. doi:10.26719/emhj.23.054.

Newton EA, Reiter P. A model of the transmission of dengue fever with an evaluation of the impact of ultra-low volume (ULV) insecticide applications on dengue epidemics. Am J Trop Med Hyg. 1992 Dec;47(6):709–20. doi:10.4269/ajtmh.1992.47.709.

Abernethy GM, Glass DH. Optimal COVID-19 lockdown strategies in an age-structured SEIR model of Northern Ireland. J R Soc Interface. 2022 Mar;19(188):20210896. doi:10.1098/rsif.2021.0896.

Li G, Li Y, Han G, Jiang C, Geng M, Guo N, et al. Forecasting and analyzing influenza activity in Hebei Province, China, using a CNN-LSTM hybrid model. BMC Public Health. 2024 Aug 12;24(1):2171. doi:10.1186/s12889-024-19590-8.

Xu J, Xu K, Li Z, Meng F, Tu T, Xu L, et al. Forecast of Dengue Cases in 20 Chinese Cities Based on the Deep Learning Method. Int J Environ Res Public Health. 2020 Jan 10;17(2):453. doi: 10.3390/ijerph17020453.

Iqbal M, Al-Obeidat F, Maqbool F, Razzaq S, Anwar S, Tubaishat A, Khan MS, Shah B. COVID-19 Patient Count Prediction Using LSTM. IEEE Trans Comput Soc Syst. 2021 Feb 19;8(4):974–81. doi:10.1109/TCSS.2021.3056769.

Santangelo OE, Gentile V, Pizzo S, Giordano D, Cedrone F. Machine learning and prediction of infectious diseases: a systematic review. Mach Learn Knowl Extr. 2023 Feb 1;5(1):175–98. doi:10.3390/make5010013.

Absar N, Uddin N, Khandaker MU, Ullah H. The efficacy of deep learning based LSTM model in forecasting the outbreak of contagious diseases. Infect Dis Model. 2022 Mar;7(1):170–83. doi:10.1016/j.idm.2021.12.005.

Sembiring I, Wahyuni SN, Sediyono E. LSTM algorithm optimization for COVID-19 prediction model. Heliyon. 2024 Feb 16;10(4):e26158. doi:10.1016/j.heliyon.2024.e26158.

World Health Organization. Thailand responding to the novel coronavirus [Internet]. Geneva: World Health Organization; 2020 Jan 13 [cited 2024 Aug 12]. <https://www.who.int/thailand/news/detail/13-01-2020-thailand-responding-to-the-novel-coronavirus>

Jindahra P, Wongboonsin K, Wongboonsin P. Demographic and initial outbreak patterns of COVID-19 in Thailand. J Pop Research. 2022;39(4):567–88. doi:10.1007/s12546-021-09276-y.

Thailand Department of Disease Control. DDC COVID-19 Interactive Dashboard [Internet]. Nonthaburi: Department of Disease Control; [cited 2024 Aug 12] <https://ddc.moph.go.th/covid19-dashboard/?dashboard=main>. Thai.

TAT Newsroom. Thailand ends COVID-19 emergency decree on 30 September 2022 [Internet]. Bangkok: Tourism Authority of Thailand; 2022 Sep 23 [cited 2024 Aug 12]. <https://www.tatnews.org/2022/09/thailand-ends-covid-19-emergency-decree-on-30-september-2022/>

Suphanchaimat R, Teekasap P, Nittayasoot N, Phaiyarom M, Cetthakrikul N. Forecasted Trends of the new COVID-19 epidemic due to the Omicron Variant in Thailand, 2022. Vaccines. 2022 Jun 27;10(7):1024. doi:10.3390/vaccines10071024.

Winalai C, Anupong S, Modchang C, Chadsuthi S. LSTM-Powered COVID-19 prediction in central Thailand incorporating meteorological and particulate matter data with a multi-feature selection approach. Heliyon. 2024 Apr 26;10(9):e30319. doi:10.1016/j.heliyon.2024.e30319.

Immunization Center, Thailand Ministry of Public Health. Covid-19 vaccination infographic [Internet]. San Francisco (CA): Internet Archive; [cited 2024 Aug 30]. <https://web.archive.org/web/20210802001320/https://dashboard-vaccine.moph.go.th/>

Institute for Population and Social Research. Mahidol population gazette: population of Thailand, 2020 (Vol.29: January 2020). Nakhon Pathom: Institute for Population and Social Research, Mahidol University; 2020 Mar 17 [cited 2024 Aug 12]. <https://ipsr.mahidol.ac.th/en/population-of-thailand-2020-vol-29-january-2020/>

Annuziato A, Asikainen T. Effective reproduction number estimation from data series. Luxembourg: Publications Office of the European Union; 2020 [cited 2024 Aug 30]; <https://publications.jrc.ec.europa.eu/repository/bitstream/JRC121343/r0_technical_note_v3.4_1.pdf>

Xin H, Li Y, Wu P, Li Z, Lau EHY, Qin Y, et al. Estimating the latent period of coronavirus disease 2019 (COVID-19). Clin Infect Dis. 2022 May 3;74(9):1678–81. doi:10.1093/cid/ciab746.

Gregory CO, Hall A. CDC Yellow Book 2024: COVID-19 [Internet]. Atlanta: U.S. Centers for Disease Control and Prevention; [cited 2024 Aug 12]. <https://wwwnc.cdc.gov/travel/yellowbook/2024/infections-diseases/covid-19#:~:text=People%20with%20COVID%2D19%20can>

Luo G, Zhang X, Zheng H, He D. Infection fatality ratio and case fatality ratio of COVID-19. Int J Infect Dis. 2021 Dec;113:43–6. doi:10.1016/j.ijid.2021.10.004.

R Core Team. R: a language and environment for statistical computing [Internet]. Vienna: R Foundation for Statistical Computing; 2016 [cited 2024 Aug 12]. <https://www.R-project.org/>

R Core Team. R: fit best ARIMA model to univariate time series [Internet]. Vienna: R Foundation for Statistical Computing; [cited 2024 Aug 12]. <https://search.r-project.org/CRAN/refmans/forecast/html/auto.arima.html>

Xu L, Magar R, Barati Farimani A. Forecasting COVID-19 new cases using deep learning methods. Computers in Biology and Medicine. 2022 May;144:105342. doi:10.1016/j.compbiomed.2022.105342.

Makridakis S, Wheelwright SC, Hyndman RJ. Forecasting: methods and applications. Journal of the Operational Research Society. 1984;35(1):79. doi:10.1057/jors.1984.11.

Hyndman RJ, Athanasopoulos G. Forecasting: principles and practice. 3rd ed. Melbourne: Otexts; 2021.

Chharia A, Jeevan G, Jha RA, Liu M, Berman JM, Glorioso C. Accuracy of US CDC COVID-19 forecasting models. Front Public Health. 2024 Jun 26;12:1359368. doi:10.3389/fpubh.2024.1359368.

Ahmad G, Ahmed F, Rizwan MS, Muhammad J, Fatima SH, Ikram A, et al. Evaluating data-driven methods for short-term forecasts of cumulative SARS-CoV2 cases. PLoS One. 2021 May 21;16(5):e0252147. doi:10.1371/journal.pone.0252147.

Rahman MS, Chowdhury AH, Amrin M. Accuracy comparison of ARIMA and XGBoost forecasting models in predicting the incidence of COVID-19 in Bangladesh. PLOS Glob Public Health. 2022 May 18;2(5):e0000495. doi:10.1371/journal.pgph.0000495.

Rguibi MA, Moussa N, Madani A, Aaroud A, Zine-dine K. Forecasting Covid-19 Transmission with ARIMA and LSTM Techniques in Morocco. SN Comput Sci. 2022;3(2):133. doi:10.1007/s42979-022-01019-x.

Zhao D, Zhang H, Cao Q, Wang Z, He S, Zhou M, et al. The research of ARIMA, GM(1,1), and LSTM models for prediction of TB cases in China. PLoS One. 2022 Feb 23;17(2):e0262734. doi:10.1371/journal.pone.0262734.

Vynnycky E, White RG, Fine PEM. An introduction to infectious disease modelling. New York: Oxford University Press; 2011.

Keeling MJ, Rohani P. Modeling infectious diseases in humans and animals. Princeton (NJ): Princeton University Press; 2008.

Published

2024-09-24

How to Cite

Thammawijaya, P. ., & Suphanchaimat, R. (2024). Accuracy of COVID-19 Prediction Modeling Techniques. Outbreak, Surveillance, Investigation & Response (OSIR) Journal, 17(3), 146–154. https://doi.org/10.59096/osir.v17i3.270877

Issue

Section

Original article