BackgroundWe aimed to select and externally validate a benchmark method for emergency ambulance services to use to forecast the daily number of calls that result in the dispatch of one or more ambulances. The study was conducted using standard methods known to the UK's NHS to aid implementation in practice.MethodsWe selected our benchmark model from a naive benchmark and 14 standard forecasting methods. Mean absolute scaled error and 80 and 95\% prediction interval coverage over a 84 day horizon were evaluated using time series cross validation across eight time series from the South West of England. External validation was conducted by time series cross validation across 13 time series from London, Yorkshire and Welsh Ambulance Services. ResultsA model combining a simple average of Facebook's Prophet and regression with ARIMA Errors (1, 1, 3)(1, 0, 1, 7) was selected. Benchmark MASE, 80 and 95\% prediction intervals were 0.68 (95% CI 0.67 - 0.69), 0.847 (95% CI 0.843 - 0.851), and 0.965 (95% CI 0.949 - 0.977), respectively. Performance in the validation set was within expected ranges for MASE, 0.73 (95% CI 0.72 - 0.74) 80\% coverage (0.833; 95% CI 0.828-0.838), and 95\% coverage (0.965; 95% CI 0.963-0.967).ConclusionsWe provide a robust externally validated benchmark for future ambulance demand forecasting studies to improve on. Our benchmark forecasting model is high quality and usable by ambulance services. We provide a simple python framework to aid its implementation in practice.

This paper provides a framework for conceptualising levels of open science and open working within computer modelling and simulation. We aim to support researchers to share their models and working so that others are free to use, reproduce, adapt and build upon, and re-share their work. We introduce a six level framework of increasing complexity: not open, open access, open artefacts, open models, open environment and open infrastructure. For each we provide practical advice on what aspects of open science researchers must consider, what options are available to them, and what challenges they will need to overcome. We illustrate our open science framework using a stylised discrete-event simulation model. All code used in this paper is available, cloud executable and reusable under an MIT license.

BACKGROUND: it is a stated ambition of many healthcare systems to eliminate delayed transfers of care (DTOCs) between acute and step-down community services. OBJECTIVE: This study aims to demonstrate how, counter to intuition, pursual of such a policy is likely to be uneconomical, as it would require large amounts of community capacity to accommodate even the rarest of demand peaks, leaving much capacity unused for much of the time. METHODS: Some standard results from queueing theory-a mathematical discipline for considering the dynamics of queues and queueing systems-are used to provide a model of patient flow from the acute to community setting. While queueing models have a track record of application in healthcare, they have not before been used to address this question. RESULTS: Results show that 'eliminating' DTOCs is a false economy: the additional community costs required are greater than the possible acute cost saving. While a substantial proportion of DTOCs can be attributed to inefficient use of resources, the remainder can be considered economically essential to ensuring cost-efficient service operation. For England's National Health Service (NHS), our modelling estimates annual cost savings of £117m if DTOCs are reduced to the 12% of current levels that can be regarded as economically essential. CONCLUSION: This study discourages the use of 'zero DTOC' targets and instead supports an assessment based on the specific characteristics of the healthcare system considered.

OBJECTIVES: to identify risk factors associated with prolonged length of hospital stay and staying in hospital longer than medically necessary following primary knee replacement surgery. DESIGN: Retrospective, longitudinal observational study. SETTING: Elective knee replacement surgeries between 2016 and 2019 were identified using routinely collected data from an NHS Trust in England. PARTICIPANTS: There were 2295 knee replacement patients with complete data included in analysis. The mean age was 68 (SD 11) and 60% were female. OUTCOME MEASURES: We assessed a binary length of stay outcome (>7 days), a continuous length of stay outcome (≤30 days) and a binary measure of whether patients remained in hospital when they were medically fit for discharge. RESULTS: the mean length of stay was 5.0 days (SD 3.9), 15.4% of patients were in hospital for >7 days and 7.1% remained in hospital when they were medically fit for discharge. Longer length of stay was associated with older age (b=0.08, 95% CI 0.07 to 0.09), female sex (b=0.36, 95% CI 0.06 to 0.67), high deprivation (b=0.98, 95% CI 0.47 to 1.48) and more comorbidities (b=2.48, 95% CI 0.15 to 4.81). Remaining in hospital beyond being medically fit for discharge was associated with older age (OR=1.07, 95% CI 1.05 to 1.09), female sex (OR=1.71, 95% CI 1.19 to 2.47) and high deprivation (OR=2.27, 95% CI 1.27 to 4.06). CONCLUSIONS: the regression models could be used to identify which patients are likely to occupy hospital beds for longer. This could be helpful in scheduling operations to aid hospital efficiency by planning these patients' operations for when the hospital is less busy.

OBJECTIVES: While there has been significant research on the pressures facing acute hospitals during the COVID-19 pandemic, there has been less interest in downstream community services which have also been challenged in meeting demand. This study aimed to estimate the theoretical cost-optimal capacity requirement for 'step down' intermediate care services within a major healthcare system in England, at a time when considerable uncertainty remained regarding vaccination uptake and the easing of societal restrictions. METHODS: Demand for intermediate care was projected using an epidemiological model (for COVID-19 demand) and regressing upon public mobility (for non-COVID-19 demand). These were inputted to a computer simulation model of patient flow from acute discharge readiness to bedded and home-based Discharge to Assess (D2A) intermediate care services. Cost-optimal capacity was defined as that which yielded the lowest total cost of intermediate care provision and corresponding acute discharge delays. RESULTS: Increased intermediate care capacity is likely to bring about lower system-level costs, with the additional D2A investment more than offset by substantial reductions in costly acute discharge delays (leading also to improved patient outcome and experience). Results suggest that completely eliminating acute 'bed blocking' is unlikely economical (requiring large amounts of downstream capacity), and that health systems should instead target an appropriate tolerance based upon the specific characteristics of the pathway. CONCLUSIONS: Computer modelling can be a valuable asset for determining optimal capacity allocation along the complex care pathway. With results supporting a Business Case for increased downstream capacity, this study demonstrates how modelling can be applied in practice and provides a blueprint for use alongside the freely-available model code.

In healthcare, opportunities to use real-time data to support quick and effective decision-making are expanding rapidly, as data increases in volume, velocity and variety. In parallel, the need for short-term decision-support to improve system resilience is increasingly relevant, with the recent COVID-19 crisis underlining the pressure that our healthcare services are under to deliver safe, effective, quality care in the face of rapidly-shifting parameters.
A real-time hybrid model (HM) which combines real-time data, predictions, and simulation, has the potential to support short-term decision-making in healthcare. Considering decision-making as a consequence of situation awareness focuses the HM on what information is needed where, when, how, and by whom with a view toward sustained implementation. However the articulation between real-time decision-support tools and a sociotechnical approach to their development and implementation is currently lacking in the literature.
Having identified the need for a conceptual framework to support the development of real-time HMs for short-term decision-support, this research proposed and tested the Integrated Hybrid Analytics Framework (IHAF) through an examination of the stages of a Design Science methodology and insights from the literature examining decision-making in dynamic, sociotechnical systems, data analytics, and simulation. Informed by IHAF, a HM was developed using real-time Emergency Department data, time-series forecasting, and discrete-event simulation. The application started with patient questionnaires to support problem definition and to act as a formative evaluation, and was subsequently evaluated using staff interviews.
Evaluation of the application found multiple examples where the objectives of people or sub-systems are not aligned, resulting in inefficiencies and other quality problems, which are characteristic of complex adaptive sociotechnical systems. Synthesis of the literature, the formative evaluation, and the final evaluation found significant themes which can act as antecedents or evaluation criteria for future real-time HM studies in sociotechnical systems, in particular in healthcare. The generic utility of IHAF is emphasised for supporting future applications in similar domains.

The purpose of a modelling and simulation (M&S) study for real-world operations management applications is to support decision-making and inform potential action, therefore investigating the aspects of the modelling process which influence trust is important. Previous work has considered the question of trust through the lens of model validation. However, whilst a simulation model may be technically well executed, stakeholders’ trust in the results may also depend upon intangible factors such as interpersonal relationships. Existing literature has also focused on the credibility of the simulation practitioner, however the credibility attribute belongs to the stakeholder, and it ignores the trust aspects that may exist between the stakeholders and the model itself. In this paper, we argue that different facets of trust emerge throughout the stages of a simulation study, and both influence, and are influenced by, the interaction between the model, the modeller and the stakeholders of the study. We present a synthesis of existing literature and extend it by proposing a formative model of trust which presents a conceptualisation of this tripartite relationship. Our contribution is the identification of the different facets of trust in the lifecycle of a modelling and simulation study. We argue that these interacting facets converge via the three-way relationship between modeller, model and stakeholders toward epistemic trust in the knowledge generated by the simulation study and ultimately model acceptability and implementation. To the best of our knowledge, ours is the first study that focuses solely on the question of trust in an M&S study.