Evaluation of individual and ensemble probabilistic forecasts of COVID-19 mortality in the United States
Author
Cramer, Estee YRay, Evan L
Lopez, Velma K
Bracher, Johannes
Brennen, Andrea
Castro Rivadeneira, Alvaro J
Gerding, Aaron
Gneiting, Tilmann
House, Katie H
Huang, Yuxin
Jayawardena, Dasuni
Kanji, Abdul H
Khandelwal, Ayush
Le, Khoa
Mühlemann, Anja
Niemi, Jarad
Shah, Apurv
Stark, Ariane
Wang, Yijin
Wattanachit, Nutcha
Zorn, Martha W
Gu, Youyang
Jain, Sansiddh
Bannur, Nayana
Deva, Ayush
Kulkarni, Mihir
Merugu, Srujana
Raval, Alpan
Shingi, Siddhant
Tiwari, Avtansh
White, Jerome
Abernethy, Neil F
Woody, Spencer
Dahan, Maytal
Fox, Spencer
Gaither, Kelly
Lachmann, Michael
Meyers, Lauren Ancel
Scott, James G
Tec, Mauricio
Srivastava, Ajitesh
George, Glover E
Cegan, Jeffrey C
Dettwiller, Ian D
England, William P
Farthing, Matthew W
Hunter, Robert H
Lafferty, Brandon
Linkov, Igor
Mayo, Michael L
Parno, Matthew D
Rowland, Michael A
Trump, Benjamin D
Zhang-James, Yanli
Chen, Samuel
Faraone, Stephen V
Hess, Jonathan
Morley, Christopher P
Salekin, Asif
Wang, Dongliang
Corsetti, Sabrina M
Baer, Thomas M
Eisenberg, Marisa C
Falb, Karl
Huang, Yitao
Martin, Emily T
McCauley, Ella
Myers, Robert L
Schwarz, Tom
Sheldon, Daniel
Gibson, Graham Casey
Yu, Rose
Gao, Liyao
Ma, Yian
Wu, Dongxia
Yan, Xifeng
Jin, Xiaoyong
Wang, Yu-Xiang
Chen, YangQuan
Guo, Lihong
Zhao, Yanting
Gu, Quanquan
Chen, Jinghui
Wang, Lingxiao
Xu, Pan
Zhang, Weitong
Zou, Difan
Biegel, Hannah
Lega, Joceline
McConnell, Steve
Nagraj, V P
Guertin, Stephanie L
Hulme-Lowe, Christopher
Turner, Stephen D
Shi, Yunfeng
Ban, Xuegang
Walraven, Robert
Hong, Qi-Jun
Kong, Stanley
van de Walle, Axel
Turtle, James A
Ben-Nun, Michal
Riley, Steven
Riley, Pete
Koyluoglu, Ugur
DesRoches, David
Forli, Pedro
Hamory, Bruce
Kyriakides, Christina
Leis, Helen
Milliken, John
Moloney, Michael
Morgan, James
Nirgudkar, Ninad
Ozcan, Gokce
Piwonka, Noah
Ravi, Matt
Schrader, Chris
Shakhnovich, Elizabeth
Siegel, Daniel
Spatz, Ryan
Stiefeling, Chris
Wilkinson, Barrie
Wong, Alexander
Cavany, Sean
España, Guido
Moore, Sean
Oidtman, Rachel
Perkins, Alex
Kraus, David
Kraus, Andrea
Gao, Zhifeng
Bian, Jiang
Cao, Wei
Lavista Ferres, Juan
Li, Chaozhuo
Liu, Tie-Yan
Xie, Xing
Zhang, Shun
Zheng, Shun
Vespignani, Alessandro
Chinazzi, Matteo
Davis, Jessica T
Mu, Kunpeng
Pastore Y Piontti, Ana
Xiong, Xinyue
Zheng, Andrew
Baek, Jackie
Farias, Vivek
Georgescu, Andreea
Levi, Retsef
Sinha, Deeksha
Wilde, Joshua
Perakis, Georgia
Bennouna, Mohammed Amine
Nze-Ndong, David
Singhvi, Divya
Spantidakis, Ioannis
Thayaparan, Leann
Tsiourvas, Asterios
Sarker, Arnab
Jadbabaie, Ali
Shah, Devavrat
Della Penna, Nicolas
Celi, Leo A
Sundar, Saketh
Wolfinger, Russ
Osthus, Dave
Castro, Lauren
Fairchild, Geoffrey
Michaud, Isaac
Karlen, Dean
Kinsey, Matt
Mullany, Luke C
Rainwater-Lovett, Kaitlin
Shin, Lauren
Tallaksen, Katharine
Wilson, Shelby
Lee, Elizabeth C
Dent, Juan
Grantz, Kyra H
Hill, Alison L
Kaminsky, Joshua
Kaminsky, Kathryn
Keegan, Lindsay T
Lauer, Stephen A
Lemaitre, Joseph C
Lessler, Justin
Meredith, Hannah R
Perez-Saez, Javier
Shah, Sam
Smith, Claire P
Truelove, Shaun A
Wills, Josh
Marshall, Maximilian
Gardner, Lauren
Nixon, Kristen
Burant, John C
Wang, Lily
Gao, Lei
Gu, Zhiling
Kim, Myungjin
Li, Xinyi
Wang, Guannan
Wang, Yueying
Yu, Shan
Reiner, Robert C
Barber, Ryan
Gakidou, Emmanuela
Hay, Simon I
Lim, Steve
Murray, Chris
Pigott, David
Gurung, Heidi L
Baccam, Prasith
Stage, Steven A
Suchoski, Bradley T
Prakash, B Aditya
Adhikari, Bijaya
Cui, Jiaming
Rodríguez, Alexander
Tabassum, Anika
Xie, Jiajia
Keskinocak, Pinar
Asplund, John
Baxter, Arden
Oruc, Buse Eylul
Serban, Nicoleta
Arik, Sercan O
Dusenberry, Mike
Epshteyn, Arkady
Kanal, Elli
Le, Long T
Li, Chun-Liang
Pfister, Tomas
Sava, Dario
Sinha, Rajarishi
Tsai, Thomas
Yoder, Nate
Yoon, Jinsung
Zhang, Leyou
Abbott, Sam
Bosse, Nikos I
Funk, Sebastian
Hellewell, Joel
Meakin, Sophie R
Sherratt, Katharine
Zhou, Mingyuan
Kalantari, Rahi
Yamana, Teresa K
Pei, Sen
Shaman, Jeffrey
Li, Michael L
Bertsimas, Dimitris
Skali Lami, Omar
Soni, Saksham
Tazi Bouardi, Hamza
Ayer, Turgay
Adee, Madeline
Chhatwal, Jagpreet
Dalgic, Ozden O
Ladd, Mary A
Linas, Benjamin P
Mueller, Peter
Xiao, Jade
Wang, Yuanjia
Wang, Qinxia
Xie, Shanghong
Zeng, Donglin
Green, Alden
Bien, Jacob
Brooks, Logan
Hu, Addison J
Jahja, Maria
McDonald, Daniel
Narasimhan, Balasubramanian
Politsch, Collin
Rajanala, Samyak
Rumack, Aaron
Simon, Noah
Tibshirani, Ryan J
Tibshirani, Rob
Ventura, Valerie
Wasserman, Larry
O'Dea, Eamon B
Drake, John M
Pagano, Robert
Tran, Quoc T
Ho, Lam Si Tung
Huynh, Huong
Walker, Jo W
Slayton, Rachel B
Johansson, Michael A
Biggerstaff, Matthew
Reich, Nicholas G
Affiliation
Department of Mathematics, University of ArizonaIssue Date
2022-04-08
Metadata
Show full item recordPublisher
National Academy of SciencesCitation
Cramer, E. Y., Ray, E. L., Lopez, V. K., Bracher, J., Brennen, A., Castro Rivadeneira, A. J., Gerding, A., Gneiting, T., House, K. H., Huang, Y., Jayawardena, D., Kanji, A. H., Khandelwal, A., Le, K., Mühlemann, A., Niemi, J., Shah, A., Stark, A., Wang, Y., … Reich, N. G. (2022). Evaluation of individual and ensemble probabilistic forecasts of COVID-19 mortality in the United States. Proceedings of the National Academy of Sciences of the United States of America.Rights
Copyright © 2022 the Author(s). Published by PNAS. This open access article is distributed under Creative Commons Attribution License 4.0 (CC BY).Collection Information
This item from the UA Faculty Publications collection is made available by the University of Arizona with support from the University of Arizona Libraries. If you have questions, please contact us at repository@u.library.arizona.edu.Abstract
Significance: This paper compares the probabilistic accuracy of short-term forecasts of reported deaths due to COVID-19 during the first year and a half of the pandemic in the United States. Results show high variation in accuracy between and within stand-alone models and more consistent accuracy from an ensemble model that combined forecasts from all eligible models. This demonstrates that an ensemble model provided a reliable and comparatively accurate means of forecasting deaths during the COVID-19 pandemic that exceeded the performance of all of the models that contributed to it. This work strengthens the evidence base for synthesizing multiple models to support public-health action.Note
Open access articleEISSN
1091-6490PubMed ID
35394862Version
Final published versionae974a485f413a2113503eed53cd6c53
10.1073/pnas.2113561119
Scopus Count
Collections
Except where otherwise noted, this item's license is described as Copyright © 2022 the Author(s). Published by PNAS. This open access article is distributed under Creative Commons Attribution License 4.0 (CC BY).
Related articles
- Predictive performance of multi-model ensemble forecasts of COVID-19 across European nations.
- Authors: Sherratt K, Gruson H, Grah R, Johnson H, Niehus R, Prasse B, Sandmann F, Deuschel J, Wolffram D, Abbott S, Ullrich A, Gibson G, Ray EL, Reich NG, Sheldon D, Wang Y, Wattanachit N, Wang L, Trnka J, Obozinski G, Sun T, Thanou D, Pottier L, Krymova E, Meinke JH, Barbarossa MV, Leithauser N, Mohring J, Schneider J, Wlazlo J, Fuhrmann J, Lange B, Rodiah I, Baccam P, Gurung H, Stage S, Suchoski B, Budzinski J, Walraven R, Villanueva I, Tucek V, Smid M, Zajicek M, Perez Alvarez C, Reina B, Bosse NI, Meakin SR, Castro L, Fairchild G, Michaud I, Osthus D, Alaimo Di Loro P, Maruotti A, Eclerova V, Kraus A, Kraus D, Pribylova L, Dimitris B, Li ML, Saksham S, Dehning J, Mohr S, Priesemann V, Redlarski G, Bejar B, Ardenghi G, Parolini N, Ziarelli G, Bock W, Heyder S, Hotz T, Singh DE, Guzman-Merino M, Aznarte JL, Morina D, Alonso S, Alvarez E, Lopez D, Prats C, Burgard JP, Rodloff A, Zimmermann T, Kuhlmann A, Zibert J, Pennoni F, Divino F, Catala M, Lovison G, Giudici P, Tarantino B, Bartolucci F, Jona Lasinio G, Mingione M, Farcomeni A, Srivastava A, Montero-Manso P, Adiga A, Hurt B, Lewis B, Marathe M, Porebski P, Venkatramanan S, Bartczuk RP, Dreger F, Gambin A, Gogolewski K, Gruziel-Slomka M, Krupa B, Moszyński A, Niedzielewski K, Nowosielski J, Radwan M, Rakowski F, Semeniuk M, Szczurek E, Zielinski J, Kisielewski J, Pabjan B, Holger K, Kheifetz Y, Scholz M, Przemyslaw B, Bodych M, Filinski M, Idzikowski R, Krueger T, Ozanski T, Bracher J, Funk S
- Issue date: 2023 Apr 21
- How New Mexico Leveraged a COVID-19 Case Forecasting Model to Preemptively Address the Health Care Needs of the State: Quantitative Analysis.
- Authors: Castro LA, Shelley CD, Osthus D, Michaud I, Mitchell J, Manore CA, Del Valle SY
- Issue date: 2021 Jun 9
- The United States COVID-19 Forecast Hub dataset.
- Authors: Cramer EY, Huang Y, Wang Y, Ray EL, Cornell M, Bracher J, Brennen A, Rivadeneira AJC, Gerding A, House K, Jayawardena D, Kanji AH, Khandelwal A, Le K, Mody V, Mody V, Niemi J, Stark A, Shah A, Wattanchit N, Zorn MW, Reich NG, US COVID-19 Forecast Hub Consortium
- Issue date: 2022 Aug 1
- Challenges of COVID-19 Case Forecasting in the US, 2020-2021.
- Authors: Lopez VK, Cramer EY, Pagano R, Drake JM, O'Dea EB, Adee M, Ayer T, Chhatwal J, Dalgic OO, Ladd MA, Linas BP, Mueller PP, Xiao J, Bracher J, Castro Rivadeneira AJ, Gerding A, Gneiting T, Huang Y, Jayawardena D, Kanji AH, Le K, Mühlemann A, Niemi J, Ray EL, Stark A, Wang Y, Wattanachit N, Zorn MW, Pei S, Shaman J, Yamana TK, Tarasewicz SR, Wilson DJ, Baccam S, Gurung H, Stage S, Suchoski B, Gao L, Gu Z, Kim M, Li X, Wang G, Wang L, Wang Y, Yu S, Gardner L, Jindal S, Marshall M, Nixon K, Dent J, Hill AL, Kaminsky J, Lee EC, Lemaitre JC, Lessler J, Smith CP, Truelove S, Kinsey M, Mullany LC, Rainwater-Lovett K, Shin L, Tallaksen K, Wilson S, Karlen D, Castro L, Fairchild G, Michaud I, Osthus D, Bian J, Cao W, Gao Z, Lavista Ferres J, Li C, Liu TY, Xie X, Zhang S, Zheng S, Chinazzi M, Davis JT, Mu K, Pastore Y Piontti A, Vespignani A, Xiong X, Walraven R, Chen J, Gu Q, Wang L, Xu P, Zhang W, Zou D, Gibson GC, Sheldon D, Srivastava A, Adiga A, Hurt B, Kaur G, Lewis B, Marathe M, Peddireddy AS, Porebski P, Venkatramanan S, Wang L, Prasad PV, Walker JW, Webber AE, Slayton RB, Biggerstaff M, Reich NG, Johansson MA
- Issue date: 2024 May
- Comparative assessment of methods for short-term forecasts of COVID-19 hospital admissions in England at the local level.
- Authors: Meakin S, Abbott S, Bosse N, Munday J, Gruson H, Hellewell J, Sherratt K, CMMID COVID-19 Working Group, Funk S
- Issue date: 2022 Feb 21