Queensland public hospitals are saving $2.5 million (USD) a year by using a tool that predicts patient admissions – and their injury type – hours, weeks or even ten years in advance. The benefit to the state as a whole could be as high as $80 million per year from improved patient outcomes. The tool uses algorithms to identify patterns in historic patient admission and discharge data, allowing hospitals to plan when to open and close beds, alter patient discharge plans and adjust staff vacations to create optimal patient flow rates. It is being used by around 50 hospitals in Queensland and has an accuracy rate of up to 95%.
Results & Impact
The financial return from implementing the Patient Admission Prediction Tool in Queensland is estimated to be as high as $80 million per year. This comprises $2.5 million in efficiency savings and $77.5 million from improved patient outcomes. If implemented across Australia it could save as much as $18.5 million per year from efficiency improvements.
Commonwealth Scientific and Industrial Research Organisation, Queensland Health
The Patient Admission Prediction Tool was first developed and tested by the Commonwealth Scientific and Industrial Research Organization in 2010 using patient admission and discharge data from two public hospitals. Regression analysis models were run to identify patterns in the records. To test the system, scientists made predictions using historical data and compared the results with the actual number of patients admitted, calculating the percentage difference between the predicted and actual number of arrivals. This system produced a model which could predict patient admissions and injury type with 90% to 95% accuracy on a day-to-day basis. The system was initially implemented in about 30 Queensland hospitals in 2011 and fully rolled out in 2014. The tool has been integrated into a daily trigger warning system, whereby staff undertake a series of measures to free up bed space in response to specific levels of anticipated demand. At the start of each day, the predicted number of arrivals is reviewed to see what demand for beds will be. If wait times are expected to rise above a certain level, a series of interventions is introduced to free up bed space, such as rescheduling elective procedures, adjusting discharge patterns or moving patients to different wards. The tool is also used for scheduling operations, planning when to open or close beds and managing staffing levels on a longer term basis.
Cost & Value
Running since 2011
The Patient Admission Prediction Tool's accuracy varies according to the time span of the prediction being run. While daily forecasts are 90% to 95% accurate, predictions for four-hour periods can be wrong by as much as 40% while hourly forecasts can have an error margin as high as 50%. A second problem is the need to manage culture change. When implemented in Queensland, there was resistance from some hospital staff who were reluctant to adjust to new working patterns.
Queensland hospitals are saving $2.5 million (USD) per year by using a web-based analytics tool to predict the number of patients who will arrive each day and what their injuries will be, with up to 95% accuracy.
The Patient Admission Prediction Tool (PAPT) allows hospital staff to see the net patient inflow or outflow, how urgent their injuries will be and what impact this will have on bed availability, hours, days, weeks or even years in advance. Predictions are based on historical admission and discharge data as the flow of patients and type of injuries are highly predictable and conform to long-term trends, despite day-to-day variation.
“For many people that work at hospitals there is an understanding that certain days are busier, but it’s about putting a number to their gut instinct and intuition,” said Justin Boyle, Research Scientist at the Commonwealth Scientific and Industrial Research Organization (CSIRO), which built the PAPT. “With this application, we are quantifying that number with a worst and best case scenario.”
First Implemented in 2011, the tool is now being used in around 50 hospitals across Queensland to help manage patient flow. The system has delivered significant savings and performance improvements. Gold Coast Hospital improved its waiting time performance by 20% using the tool, while one estimate put the financial returns for the state of Queensland as high as $80 million USD per year. This comprises $77.5 million from improved patient outcomes, while hospitals benefit from $2.5 million worth of efficiency savings.
Hospital waiting times became a pressing problem in Australia in 2011, causing the government to introduce the National Emergency Access Target. This required all public hospitals to process 90% of A&E patients within a four-hour window by 2015. This meant either admitting, discharging or transferring arrivals to another medical facility. Between 2011 and 2012, the average process time for a patient in an emergency department was estimated to be almost eight and a half hours.
“We had initial discussions with doctors and emergency physicians who were trying to get on top of the demand for public healthcare, and one of the solutions to enable more proactive planning was based on knowing what was coming in at the door of the hospital,” said Boyle.
PAPT provides hospitals with this information through daily breakdowns of patient flow and injury type, displayed through pie and bar charts, with error margins also shown. Expected arrivals are ranked using Australia’s triage scale, a five-category system that grades patients from most to least urgent. Daily predictions can also be shown in hourly plots, allowing staff to know what bed space needs to be available at any specific point in time.
“It’s a really big game-changer for hospitals to join the dots in relating demand to available capacity,” said Boyle. “It’s just a matter of taking that next step and, from asking what demand is going to be, relating that to the number of beds in a facility and the number of patients in those beds. That’s the step that bed managers can take to say, ‘I need to open beds before this big glug arrives’ or ‘I can close beds,’ which represents big cost savings for the hospital.”
To develop the system, scientists at CSIRO used five years of patient admission and discharge data from two public hospitals. The aim was to detect subtle and consistent patterns of variation in types and numbers of patients presenting on any given day. Box-Jenkins Autoregressive Moving Average Analysis was used to identify spikes of patient activity across short time frames of one to two weeks. Another approach was to take a single day of the week and examine admission patterns at four-week intervals across a three-year period, with the data points weighted to give greater value to the most recent information.
The model was tested by splitting patient admission data into training and evaluation datasets. A year’s worth of forecasts were made for a period researchers possessed data for, allowing them to evaluate the model’s accuracy by comparing predictions to actual patient admissions and calculating the mean absolute percentage error. This was done by comparing predictions and admissions data for each day, with the difference recorded as a percentage figure. Depending on the quantity of data available, the system was able to deliver predictions that were 90% or 95% accurate.
To run the system, staff either manually input information from new patients or integrate the tool with hospital IT systems, allowing new records to be automatically logged in its database.
At many Queensland hospitals, the tool has been implemented into day-to-day work patterns using a trigger warning system. At the start of the day, normally around 7:15 am, bed managers examine the day’s admission predictions. If waiting times are at risk of being too high, reviews of available bed capacity are launched across the hospital’s wards. If the net in-flow is still too great, executive managers consult with staff from the different wards to try and free up space, while non-urgent surgeries can be re-arranged if necessary.
PAPT allows staff to calculate which days of the week will be busiest and how high demand will be. Surgery plans can therefore be adjusted and slots scheduled without the patient having presented themselves. Peak yearly periods, including the winter surge, can also be better planned for, particularly when decisions on staff vacation and how many bed openings often have to be made well in advance. One reason the tool is able to look so far into the future is that population growth and demographic change is built into the model. This is particularly valuable for hospital staff in terms of showing how any patient group, such as those suffering cardiac problems, is likely to develop.
Although the tool can predict how many patients will suffer from any type of injury, the granularity of PAPT’s forecasts depends on at least 10 patients presenting themselves per day in a specific category.
Accuracy also varies depending on the time interval predicted and the time of year. When there is more variation in historic admissions, particularly around public holidays, prediction accuracy is reduced. Once the time span being forecast is reduced below a day, the error margin can also significantly increase. While daily predictions can be made with 90% or 95% accuracy, forecasts for four-hour periods can have error margins as large as 40%, increasing to 50% for hourly predictions.
Nevertheless, it is estimated that if the tool were rolled out across Australia as a whole, it could unlock efficiency savings of $18.5 million per year.
(Picture credit: Pixabay/Taokinesis)