Original author: Dianne Daniel
Published: October 29, 2020
Source: https://www.canhealth.com/
Consider the following urgent situations:
- Concerned parents rush their child to The Hospital for Sick Children (SickKids) emergency department in Toronto. After checking in with the triage nurse, they go to the waiting area and a short while later, they receive a message that a urine test and X-ray have been ordered and are told what to do next. By the time they see the physician, she’s already armed with the results and makes an informed decision about their child’s care on the spot.
- The condition of an inpatient at St. Michael’s Hospital in Toronto starts to deteriorate, putting him at high risk for an adverse event. A secure email is immediately sent to the nurse’s desk and the attending physician receives a page, allowing for early care intervention and preventing a sudden trip to the ICU.
- A critically ill infant in the neonatal intensive care unit (NICU) at Southlake Regional Health Centre in Newmarket, Ontario, starts to develop sepsis, a condition that appears suddenly and can turn fatal within hours. Clinicians are alerted and quickly intervene with life-saving measures.
The common thread between all three of these research scenarios, currently under way in Ontario? A branch of artificial intelligence called machine learning that is poised to reshape medicine, one predictive model at a time.
“This is a major data and computing revolution of our time,” said Dr. Amol Verma, an internist and scientist at St. Michael’s. “If we’re going to figure out how to harness it for healthcare, to really improve how we deliver care for our patients, it’s going to take a lot of hard work.”
Dr. Verma is part of a multi-disciplinary team at the hospital working on an early warning system called CHARTwatch, designed to reduce mortality and improve the quality of care of patients on the general internal medicine ward. One of the first Canadian hospitals to establish an in-house data science and advanced analytics team – including a multimillion-dollar infrastructure investment and the creation of a vice-president of Data and Analytics position – St. Michael’s recognizes machine learning as a “key area of healthcare development and innovation,” he said.
In simple terms, machine learning is the process of using algorithms to teach a computer to make accurate decisions and predictions based on data. The goal of CHARTwatch is to improve real-time clinical decisions by automating the process of rapidly collecting and analyzing data from the hospital’s electronic medical record (EMR).
Whereas similar predictive models, widely used in the U.K., were designed to analyze five to 10 data points and use simple statistical algorithms, CHARTwatch analyzes more than 100 data elements stored in the hospital’s EMR. Its complex algorithms run like a digital assistant in the background, operating in parallel to the hospital staff’s existing workflow and making accurate predictions based on real-time patient information, including length of stay.
It took two years, a great deal of input from clinicians and analysis of 10 years’ worth of historical data to develop the model, which is designed to provide 24 to 48 hours advance notice when a patient is at risk of deteriorating. Clinicians are alerted by secure email, page or through the hospital’s physician sign-out tool and patient risk scores are assigned according to three categories: green, yellow or red.
Before launching their model in a clinical setting this past summer, the St. Michael’s team conducted a real-time comparison between the model and clinicians. After comparing more than 3,000 prediction instances over a four-month period, they determined their model was 20 percent more accurate than clinicians at identifying patients who were either going to die or end up in the ICU, said Dr. Verma.
“I think what’s really exciting about this is once you deploy to the real world, it’s actually not that the model is better than the clinician. Ultimately, it’s the combined intelligence: How does the model inform the clinical judgement of the doctor, who can then help the patient,” he said.
Now that the model is live, researchers are carefully monitoring results through weekly team meetings and a full year of qualitative and quantitative evaluation is planned. Anecdotal evidence shows that CHARTwatch is accurately identifying outlier patients, prompting clinicians to reassess patients they may have otherwise missed. At the same time, the model sometimes tells clinicians what they already know and in some instances, the clinicians override the model’s decision.
An added benefit is that the model is being used to strengthen communication between patients, their families and clinicians. “We had patients who had a very long and complicated hospital stay and their family members were very anxious about leaving them overnight on the ward,” said Dr. Verma. “The model predicted the patient was low risk and we were able to share that with family and reassure them.”
Improving communication is also one of the objectives of researchers at SickKids who are working to develop early warning systems to detect sepsis and cardiac arrest, as well as predictive models to better manage patient flow and patient volumes.
The hospital’s multidisciplinary research team follows a structured pipeline to advance machine learning projects, starting with clinical use case design, followed by data acquisition and preparation, model development, model and user validation, and ending with clinical integration. Legal, privacy and ethical considerations – including steps to identify model bias and ensure fairness and equity regarding who will benefit from the model – are emphasized throughout the process.
The end goal is to translate academic studies into real-world, meaningful action, said Dr. Devin Singh, physician lead for Clinical AI and Machine Learning in Pediatric Emergency Medicine at SickKids.
“You could run these models as decision support, where all that’s happening is an alert is firing but there’s no impact directly on patient care,” said Dr. Singh, who recently launched a health technology start-up lab called Hero AI to help translate leading-edge research at SickKids into the clinical setting. “The more interesting step is to avoid that altogether and go straight into action,” he added.
What’s different about their approach is that they are circumventing the challenge of alert fatigue – which occurs when a predictive model generates false positive results – by setting their model thresholds high. For example, a model could be set to react only when it is 99 percent certain it has identified a positive case or that a waiting ER patient requires a specific test.
“The sheer act of programming your model that way means you’re going to end up missing a bunch of cases that come through, but when the model does fire, you’re so confident the result is correct, you can add automation to that workflow,” explained Dr. Singh.
One project, expected to go live in the ER this year, is a model that predicts downstream events after a patient arrives. The model collects and analyzes triage data from the electronic health record, such as demographic details, vital signs and preferred language, as well as nursing notes, to accurately detect common ailments that are most likely to require routine tests such as urinalysis, X-ray or ultrasound. It then triggers the automatic generation of orders and notifies patients to complete the tests while they wait to be seen.
“Those tests can now be done ahead of time, during the period of time that you’re waiting, adding efficiency and value to your stay, and by the time the doctor sees you, they can go straight to decision making,” he said. “That’s really powerful.”
Instead of trying to program a predictive model that will be right for every patient, every time, the approach focuses on automating care for a subset of patients with confidence. The end result is a two-pronged ER workflow where testing is automated for those patients identified by the model, while remaining cases follow the normal pathway of care.
“Our hypothesis is by adding efficiency for 20 percent of patients, we’re actually adding efficiencies to everybody, said Dr. Singh.
Other leading-edge research being explored at SickKids focuses on using available data to predict patient volumes, including weather and traffic patterns, COVID case numbers, prescription usage and possibly even Google search hits. Researchers are also looking at developing a chatbot to serve as a virtual assistant for parents at home, helping them to know when they need to take a child to the ER.
“If you come too early and you’re too reflexive in your response to unwell children, then you start to overcrowd and overwhelm a publicly funded healthcare system and that leads to adverse outcomes,” he explained, noting that a late trip to the ER can also be devastating.
Carolyn McGregor, the Canada Research Chair (Alumni) in Health Informatics based at Ontario Tech University in Oshawa, said the simplest way to think about machine learning is to call it automated decision making. Known for her breakthrough work to detect early onset of sepsis in infants in neonatal intensive care units (NICUs), Dr. McGregor is now working with a variety of stakeholders to further advance machine learning in healthcare.
Her patented approach to monitoring infants in the NICU – which resulted in the Artemis Cloud Health Analytics-as-a-Service (HAaaS) platform – examines the interplay between respiration rate variability and heart rate variability. The model essentially watches for changes in heart rate or breathing that are signs a child is dealing with infection, and then alerts physicians to intervene and decide next steps.
The cloud environment supporting Artemis is provisioned by the Centre for Advanced Computing at Queen’s University in Kingston, Ontario. Artemis runs continuously in the background of the NICU, processing roughly three million data points per infant per hour, as it analyzes changes in infant physiology.
The approach is unique in that it allows treating physicians to ‘see’ changes in behaviour they might not pick up at bedside. “The second by second numbers (on a monitor) help them to see in the moment if the patient is still alive, but it doesn’t answer questions about how the patient’s physiology is changing such that they are showing signs of infection, a hemorrhage or some other condition,” explained Dr. McGregor.
Based on a study of Artemis deployments at McMaster Children’s Hospital in Hamilton, Ontario, and Southlake Regional Health Centre in Newmarket, Ontario, the platform was proven to maintain service availability as high as 99.7 percent and McGregor’s team was on the cusp of releasing a decision support protocol for early detection of sepsis when the COVID-19 pandemic launched them into a holding pattern. “We’re looking forward to when we can restart the studies,” she said.
One of the challenges to the approach of using real-time physiological monitoring to inform predictive models is signal quality. In the NICU, the advantage is that babies sleep for long periods, allowing for uninterrupted data flow. The data from McMaster’s NICU is collected from Philips Intellivue monitors, while Southlake uses GE Dash monitors. As Dr. McGregor’s team now works to apply the platform in other real-world environments, they need access to low-cost, easy-to-wear sensors capable of transmitting data at a fast frequency to a public cloud.
“We’re trying to create the digital twin of the human and then do analysis on the information of that digital twin,” she explained. “At the moment, we’re still having trouble getting that exact replica and ensuring the digital twin is an actual representative of the human.”
Last year, Ontario Tech University partnered with University of Technology Sydney (Australia) to launch a Joint Research Centre in Artificial Intelligence for Health and Wellness. As a co-director of the centre, Dr. McGregor is applying her expertise to improve health, wellness, resilience and adaption in several different populations. The extended technology platform is called Athena Cloud, and incorporates environmental and activity data in addition to physiological data.
The centre is collaborating with the Canadian Space Agency to predict how astronauts adapt to weightlessness, as well as with firefighters, tactical officers and members of the Department of National Defence to monitor physiological changes that occur during intense training scenarios, such as running into a burning building. Another project is working to develop an early indicator of aggressive behaviour in mental health patients.
“As much as I realize the possibilities are infinite, I’ve been very fortunate that other people are coming to me and saying, ‘Can we use it to do this?’” said McGregor. “The answer is yes you can and the beauty of that is we’re solving real problems.”