Agreement of BDMS Utilization Review Technology Version 2 (BURT 2)in Comparison with Utilization Management (UM) Physicians’Opinions to Assess Appropriateness of Continuation of Hospital Stay
Article Sidebar
Main Article Content
Abstract
OBJECTIVES: This study aims to find the agreement between the Bangkok Dusit Medical Services (BDMS) Utilization Review Technology Version 2 (BURT2), an artificial intelligence (AI) driven application, and Utilization Management (UM) physicians’ opinions to assess appropriateness of continuation of hospital stay.
MATERIALS AND METHODS: This retrospective study gathered de-identified patients’ data from the Health Information System (HIS) of a network hospital of BDMS. The study included patients’ data admitted in December 2021. A sample size, calculated from all data, was 274 cases.Inclusion criteria were patients with age more than 3 months and length of stay (LoS) of not more than 7 days (LoS ≤ 7 days). All data were processed by BURT 2 to predict the appropriateness of continuation of hospital stay of patients at each admission day. BURT 2 is an AI application specially developed to classify admitted cases on appropriateness for continuation of hospital stay. The application employed convolution neural network (CNN)and natural language processing (NLP) techniques on top of a rule-based algorithm, similar to its predecessor BURT 1. Outputs from BURT 2 were compared with UM Physicians’ opinions. BURT 2 was trained until the agreement or accuracy reached 90%.
RESULTS: Among 274 cases, of which 45.3% were male, 53.3% were diagnosed as simple diseases, the majority (42.7%) received services at Internal Medicine Unit. Almost all of cases (95.3%) stayed in hospital for less than four days. The comparison between BURT 2 outputs and UM Physicians’ opinions on the appropriateness of continuation of hospital stay in 274 cases showed an agreement of 96%, with 95% sensitivity, 96%specificity, 95% positive predictive value (PPV) and 97% negative predictive value (NPV).
CONCLUSION: BURT 2 had adequate agreement for predicting an appropriateness of continuation of hospital stay. It enabled an initial screening of appropriate continuation of hospital stay, increasing UM nurse work effectiveness, reducing an inappropriate continuation of hospital stay and reducing medical expenses from an inappropriate admission.
Article Details
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
This is an open access article distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
References
Piravej N, Lohakitsatian A, Piriyaprasarth R, et al. Utilization Management. Bangkok: 2021. (Accessed October 10, 2021 at https://www.si.mahidol.ac.th/th/division/um/admin/download_files/123_48_1toaznc.pdf).
Bailit HL, Sennett C. Utilization management as a cost-containment strategy. Health Care Financ Rev 1992;1991(Suppl):87-93.
Panyasorn J, Banomyong P, Phetchunsakul K, et al. Development of BDMS Utilization Review Technology (BURT): An Artificial Intelligence Tool Using Thai Natural Language Processing to Assess Appropriateness of Hospitalization. BKK Med J 2020;16:182-95.
Bulut M, Cebicci H, Sigirli D, et al. The comparison of modified early warning score with rapid emergency medicine score: a prospective multicentre observational cohort study on medical and surgical patients presenting to emergency department. Emerg Med Jl 2014;31(6):476-81. doi: 10.1136/ emermed-2013-202444. Epub 2013 Apr 6.
Alper E, O’Malley TA, Greenwald J, et al. Hospital discharge and readmission. UpToDate Waltham, MA: UpToDate. 2022. (Accessed October 2, 2021, at https://www.uptodate.com/ contents/hospital-discharge-and-readmission?search=hospita ldischarge-and-readmission&source=search_result&selected Title=1~150&usage_type=default&display_rank=1).
Gaughan J, Gravelle H, Siciliani L. Delayed discharges and hospital type: evidence from the English NHS. Fiscal Studies 2017;38(3):495-519. doi: 10.1111/j.1475-5890.2017.12141.
Gomez B, Mintegi S, Bressan S, et al. Validation of the “stepby-step” approach in the management of young febrile infants. Pediatrics. 2016;138(2). (Accessed October 2, 2021, at https:// doi:10.1542/peds.2015-4381).
Queensland government. Practice Guidelines for the Febrile illness Emergency management in children. 2019. Management. (Accessed October 6, 2021,at https://www.childrens.health. qld.gov.au/guideline-febrile-illness-emergency-managementin-children/).
International Association for Ambulatory Surgery. Discharge Process and Criteria Following Day Surgery. Bangkok: 2021. (Accessed October 17, 2021, at https://www.iaas-med. com›index.php).
Shane AL, Mody RK, Crump JA, et al. Infectious Diseases Society of America clinical practice guidelines for the diagnosis and management of infectious diarrhea. Clin Infect Dis2017;65(12):e45-80. doi: 10.1093/cid/cix669.
Children’s Hospital of Philadelphia. Inpatient Discharge Criteria 2020. (Accessed October 18, 2021, at https://www. iaas-med.com›index.php).
Korttila K. Recovery from outpatient anaesthesia: factors affecting outcome. Anaesthesia. 1995;50:22-8. (Accessed October 18, 2021, at https://doi.org/10.1111/j.1365-2044.1995. tb06186.x).
Chapter: Clinical Anesthesiology: Anesthetic Management: Ambulatory, Non operating Room, & Office-Based Anesthesia. Anesthesia: Discharge Criteria. [Internet] (Accessed October 18, 2021, https://www.brainkart.com/article/Anesthesia- -Discharge-Criteria_27205/).
Hajian-Tilaki K. Sample size estimation in diagnostic test studies of biomedical informatics. J Biomed Informs 2014;48:193-204. doi: 10.1016/j.jbi.2014.02.013.
Ostrowski TR, Ostrowski T. The basic four measures and their derivates in dichotomous diagnostic tests. Int J Clin Biostat Biom 2020;6:026. doi: 10.23937/2469-5831/1510026.