Development of BDMS Utilization Review Technology (BURT): An Artificial Intelligence Tool Using Thai Natural Language Processing to Assess Appropriateness of Hospitalization
Article Sidebar
Main Article Content
Abstract
OBJECTIVES: To develop an effective artificial intelligence (AI) driven platform to optimize the process of assessing appropriateness of hospitalization.
MATERIALS AND METHODS: Anonymized data of 22,020 insured-patient admissions in a BDMS network hospital were included to build a prediction model based on a comprehensive guideline for appropriate hospitalization. To develop Thai Natural Language Processing (NLP) model, 77,707 sentences from medical records were used and separated into two datasets, 80% for training and 20% for testing. A combined NLP and rule-based algorithms formed an AI engine and outputs were displayed using a web-based application. An expert panel of five Utilization Management (UM) physicians had several collaborative discussions to fine tune the NLP model, application of clinical criteria, and classification engine. Eventually, NLP model in the latest version (BURT1.1), had satisfactory features with overall higher than 99% accuracy, precision, recall, and F1.
RESULTS: Performance of BURT1.1 was assessed using 300 cases randomly selected from the main dataset, against other methods, including concurrent review by UM nurses at the participating hospital, and UM nurses at Bangkok Hospital Headquarters (BHQ). Agreement upon UM Physician Panel consensus was set as one of the performance indicators, and BURT1.1 showed a favorable outcome with the highest rate of agreement (86%) among all the methods. The precision rate was 99% as compared to insurance claim approval status. Additionally, dramatic time savings were achieved with 0.59 second of processing time as compared to 10-15 minutes per case by conventional manual review.
CONCLUSION: BURT1.1 should be effectively implemented as an automatic daily tool to screen inappropriate hospitalization. It can immediately identify patients at high risk of inappropriate hospitalization that require further assessment by UM nurse, thus providing feedback to attending physicians on the completeness and quality of documentation, with parallel notification to UM physicians. Ultimately, BURT1.1 can contribute to increase UM efficiency, speeding up the claim process, reducing health care costs due to unnecessary hospitalization, and reduction of claim denials.
Article Details
This is an open access article distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
References
2. Sherman Folland, Allen C. Goodman, Miron Stano. The Economics of Health and Health Care, 2013
3. Thomas M. Wickizer1 and Daniel Lessler. Utilization Management: Issues, Effects, and Future Prospects. Annual Review of Public Health
2002;23:233-254
4. Nelson BD1, Gardner RM. Decision support for concurrent utilization review using a HELP-embedded expert system. Proc Annu Symp Comput Appl Med Care. 1993:176-82.
5. Annual report 2012 – 2019. Office of Insurance Commission. Thailand.
6. J Richards . Systems and methods for automated processing and assessment of an insurance disclosure via a network. US Patent App. 11/504,463, 2007
7. William D. KirshPeter M. KramerJeffrey T. King. System and method for standardized and automated appeals process. 2012
8. Healthcare. September 2017. Copyright © McKinsey & Company. Artificial intelligence in
health insurance. Smart claims management with self-learning software.
9. Lynette Hirschman ∗, Guy Story ∗, Elaine Marsh ∗, Margaret Lyman †, Naomi Sager. experiment in automated health care evaluation from narrative medical records. Computers and Biomedical Research. Volume 14, Issue 5, October 1981, Pages 447-463
10. https://www.optum.com/resources/library/ai-powered-services.html
11. https://www.xsolis.com/solutions/provider-overview
12. Rational Identification of Simple Disease Cases in Bangkok Dusit Medical Services Hospitals using Relative Weight and Case Mixed Index.
Chatree Duangnet1, Jinhatha Panyasorn2, Noppadol Phengpinit2, Warangkana Jarunai2, Natamon Worachat3, Ritthikrai Taweecharoen4, Chaiyos Kunanusont4
1Executive Advisor and Director, Bangkok Dusit Medical Services (BDMS), 2 Utilization Management and Third-party-payer Services, Bangkok Hospital Headquarters (BHQ), 3Total Quality and Cost Improvement (TQCI), BHQ, 4Bangkok Health Research Center (BHRC), BDMS.
13. Chalermpol Tapsai, Phayung Meesad and Herwig Unger. An Overview on the Development of Thai Natural Language Processing. Information Technology Journal. Vol. 15, No. 2, July - December 2019
14. Kim, Y. 2014. Convolutional neural networks for sentence classification. arXiv:1408.5882.
15. Lai, S.; Xu, L.; Liu, K.; and Zhao, J. 2015. Recurrent convolutional neural networks for text classification. In AAAI, 2267–2273.
16. Zhou, Z., Zhu, X., He, Z., and Qu, Y. Question classification based on hybrid neural networks. In 2016 4th International Conference on Electrical &Electronics Engineering and Computer Science (ICEEECS 2016). Atlantis Press, 2016b.
17. PyThaiNLP team, © Copyright 2017-2019, PyThaiNLP (Apache Software License 2.0) https://www.thainlp.org/pythainlp/docs/dev/index.html
18. Stephen B Johnson 1 , Suzanne Bakken, Daniel Dine, Sookyung Hyun, Eneida Mendonça, Frances Morrison, Tiffani Bright, Tielman Van Vleck, Jesse Wrenn, Peter Stetson. J Am Med Inform Assoc. An Electronic Health Record Based on Structured Narrative. Jan-Feb 2008;15(1):54-64.
19. Johnson, Rie and Zhang, Tong. Deep pyramid convolutional neural networks for text categorization. In ACL, 2017.
20. Alon Jacovi, Oren Sar Shalom, and Yoav Goldberg. Understanding convolutional neural networks for text classification. arXiv preprint arXiv:1809.08037, 2018.
21. Scottish Intercollegiate Guidelines Network. Bronchiolitis in children, a national clinical guideline. 2006.
22. Ministry of Public Health. Clinical Practice Guideline for Influenza. 3 ed 27 September 2011.
23. Haffner: J, Parsons D. Clinical Presentation Guidelines For Use in the Minor Injury Units. 2011.
24. Ministry of Public Health - Lebanon. Medical necessity for Inpatient admission: Banner Health System Observation Group. 2013.
25. PHO Alliance inc/Primary Health Alliance inc. Primary Options for Acute Care(POAC) Clinical Guideline: Acute Adult Dehydration. 2015.
26. Thai Society of Pediatric Gastroenterology and Hepatology. Clinical Practice Guideline for Acute Diarrhea in Children. 2019.
27. Sumithra Velupillai 1 , Hanna Suominen 2 , Maria Liakata 3 , Angus Roberts 4 , Anoop D Shah 5 , Katherine Morley 6 , David Osborn 7 , Joseph Hayes 8 , Robert Stewart 9 , Johnny Downs 10 , Wendy Chapman 11 , Rina Dutta 12. Using Clinical Natural Language Processing for Health Outcomes Research: Overview and Actionable Suggestions for Future Advances. J Biomed Inform. 2018 Dec;88:11-19
28. B D Nelson, R M Gardner, G Hedrick, and P Gould. Computerized decision support for concurrent utilization review using the HELP system. J Am Med Inform Assoc. 1994 Jul-Aug; 1(4): 339–352.
29. L A Lenert 1 , M Tovar. Automated Linkage of Free-Text Descriptions of Patients With a Practice Guideline. Proc Annu Symp Comput Appl Med Care ACTIONS. 1993;274-8.
30. Brent A. Metfessela. aHealthTechnics, Inc., Eagan, Minnesota, USA. Chapter 4: Knowledge Representation. An Automated Tool for an Analysis of Compliance to Evidence-based Clinical Guidelines. Amsterdam: IOS Press © 2001 IMIA
31. Carol Friedman, Thomas C Rindflesch, Milton Corn. Natural Language Processing: State of the Art and Prospects for Significant Progress, a Workshop Sponsored by the National Library of Medicine. J Biomed Inform. 2013 Oct;46(5):765-73