Exeter Test Group

Medical tests have an ever greater impact on health care. They can be anything which provides information which helps diagnose disease, screens for early disease, monitors response to treatment or provides an indication of the outlook for a patient (their prognosis). Good tests are essential for successful stratified medicine approaches, and are critical to delivering the promises of precision medicine. Tests can take a wide variety of forms, but the most common are blood tests, X-rays, scans or questionnaires.

Based in the Faculty of Health and Life Sciences and the University of Exeter Medical School, The Exeter Test Group is an established centre of excellence in test evaluation working collaboratively with many other groups particularly PenTAG, ESMI, PenARC, Health Economics, Health Statistics, Cancer diagnosis (DISCO) and Molecular Genetics. We are a diverse and multi-skilled group of researchers united by our interest in solving the challenges of how to identify which tests make the greatest contribution to health, and which represent the best value for money as many tests are increasingly expensive and consume valuable time to conduct and incorporate into practice.

With backgrounds in psychological assessment, public health, clinical epidemiology, systematic review, realist evaluation, health care modelling and the use of expert elicitation for decision-analytic models, we draw on a wide range of expertise and experience in order to evaluate medical tests for real world applications. As well as evaluation, we have interests in helping improve the interpretation of research on tests by doctors, nurses, professions allied to medicine and patients themselves.

We are closely involved in national policy making on new tests, working closely with the National Screening Committee and NICE. We are headed by Professor Chris Hyde, an expert in health technology assessment of tests.

The aims of this theme are to improve the health of people in the South West and beyond by:

• producing generalisable research evidence about diagnostic strategies
• developing and spreading evidence-based diagnostic pathways in the South West Peninsula, and
• increasing the South West Peninsula’s capacity to deliver world-class research in diagnostics.

These aims recognise that knowledge about individual tests is maximised only when it is translated into evidence about diagnostic strategies that are, in turn, integrated with evidence about patient management in diagnostic pathways. Linkage of evidence is thus important.

Prof Tim McDonald will be talking to us about “A Newborn Screening Programme for Neonatal Diabetes”.

In particular, Tim will discuss his experience of the hurdles and challenges of generating the evidence required to satisfy the National Screening Committee that a test should be considered for implementation into newborn screening in the UK.

Please contact Jaime Peters for details on how to join the zoom meeting.

Future sessions

To be confirmed.

If you are interested in presenting/discussing your work at one of our meetings, we'd be happy to hear from you.

Please get in touch if you would like to receive alerts about future sessions and similar events by emailing Zhivko Zhelev.

Mar 7th: Prof Krasimira Tsaneva-Atanasova talked about her work on "Digital tests as diagnostic aid for mental health".

Feb 7th: Dr Stuart Wright from the University of Manchester presented his recent work “Evaluating the cost-effectiveness of risk-stratified breast cancer screening in the UK”.

Jan 17th: Prof Richard Riley (University of Birmingham) discussed his recent paper "Stability of clinical prediction models developed using statistical or machine learning methods". A copy of the paper is available here.

6th Dec: Dr Sian de Bell and Dr Zhivko Zhelev (University of Exeter) discussed their HS&DR project commissioned by NHS@Home "Remote monitoring in adults with chronic physical conditions."

1st Nov: Dr Elham Nikram (University of Exeter) preseneted on "Neurofilament light levels predict clinical progression and death in multiple system atrophy". A recording of her talk can be found here.

4th Oct: Prof Ed Wilson (University of Exeter) presented on "What is the value of undertaking a study to determine the clinical and cost-effectiveness of late pregnancy ultrasound to prevent adverse perinatal outcomes in nulliparous women? Or Is a routine third-trimester ultrasound scan worth it, and is a new RCT worth it?" A recording can be found here.

6th Sept: Prof Bethany Shinkins (University of Leeds), Prof Sian Taylor-Phillips (University of Warwick), and Prof Chris Hyde discussed “Current issues in evaluations of screening tests/programmes”. A recording can be found here.

5th July: Dr Bogdan Grigore presented recent work on "Screening for autism spectrum disorder: searching for better evidence from screening and intervention programmes"

7th June: Prof Mike Messenger spoke about "Regulation of In Vitro Diagnostic Medical Devices – a time of change and opportunity"

3rd May: Dr Stuart Hogarth from the University of Cambridge presented "An unravelling patchwork? Regulatory expansion and contraction in the evaluation of diagnostic tests in the UK NHS". A recording can be found here.

5th April: We discussed the 2020 paper by Cocco et al "Target Product Profiles for medical tests: a systematic review of current methods"

2nd March: Prof Chris Hyde reported on the findings of the TEST project.

TEST - Test Evaluation using Structured Tools - is an MRC funded project being jointly run between Birmingham and Exeter. The project will produce a framework and supporting tools based on the principles of comparative and efficient evaluation to: (a) categorise the intended way by which a test benefits patients; (b) identify other (unintended) impacts to patients; (c) consider which comparisons and trade–offs need to be investigated; (d) identify the key outcomes that must be measured to evaluate intended and unintended effects; and (e) select the most efficient portfolio of study designs required to fully assess the likely impact of a test. In addition (f) we will produce sample size tools to assist researchers planning new studies.

8th February: Zhivko Zhelev presented on the evaluation of AI systems for medical imaging in screening programmes

11th January: Prof Stuart Logan discussed "The tyranny of categories" A recording can be found here.

2nd November: Bada Yang, from the University of Amsterdam, discussed "Comparative diagnostic accuracy studies: an overview of study designs and reporting practices" A recording can be found here.

5th October: Jaime Peters presented on a recent systematic review focussing on methodological variation in economic evaluations of lung cancer screening. The abstract for the paper can be found here, with a recording of the talk here.

6th July: Rachel Nice, from the Royal Devon and Exeter Hospital, presented on Pre-treatment antibodies to anti-TNFα drugs; are they there and do we care?

1st June: Georgina Thompson and Chris Hyde presented on their recent methodological work 'How easy is it to identify the study design from the title and abstract of articles indexed as “diagnostic test accuracy studies” in EMBASE in 2012 and 2019?' A recording can be found here.

30th March: Goda Kijauskaite, Evidence Review Manager for the UK National Screening Committee, spoke about her work on A comparison of standalone deep learning-based AI systems against human readers in classifying mammograms: a systematic review and meta-analysis.

2nd March: Prof Caroline Wright discussed her work on Using SNP-chips to detect very rare pathogenic variants: retrospective population-based diagnostic evaluation. Caroline's paper can be found here.

2nd February: Prof Chris Hyde lead discussion of a Cochrane systematic review on Covid antibody testing. A recording can be found here.

12th January: Dr Nick Binney from the Department of Medical Ethics at Erasmus presented “On the origins of sensitivity and specificity”. Nick's paper can be found here. With the recording here.

1st December: Dr Bev Shields presented on CE marking software as a medical device – our experience of the MODY calculator and Dr TJ McKinley talked about A Bayesian re-calibration approach for the MODY calculator.

3rd November: Prof Sue Mallett, University College London, discussed the methodological and disease challenges of RT-PCR testing for SARS-CoV-2 underlying the methods and presentation of results in an individual participant data review. Her paper can be found here.

13th October: Dr Katie Young, Postdoctoral Research Associate from UoE, talked about her analysis of routine HbA1c data, and Dr Zhivko Zhelev presented a recent rapid review on point of care testing for HbA1c.

7th July: Prof Chris Hyde presented a paper on antibody testing for COVID19 and discussed application of the new QUADAS-2C tool.

3rd March: Dr Bogdan Grigore, Research Fellow from UoE, talked about the development, validation and effectiveness of diagnostic prediction tools for colorectal cancer in primary care, based on the findings from a recently conducted systematic review.

4th February: Dr Tanimola Martins, a Research Fellow from the UoE DISCO group talked about his research on ethnic inequalities in diagnosis and outcomes of common cancers. You can watch a video of the session here.‌

14th January: Discussion of the recently published paper "An analysis reveals differences between pragmatic and explanatory diagnostic accuracy studies" by Patrick M. Bossuyt, Maria Olsena, Chris Hyde and J
er
emie F. Cohenc. The discussion was facilitated by Prof Chris Hyde. You can download the paper from here.

12^th November: Dave Wright, Professor in Statistics from our College of Medicine and Health, will talk about his research on competing risks model in screening for pre-eclampsia and the related ASPRE trial.

1^st October: Michael Smith, Consultant Ophthalmologist at the RD&E talked about his research in testing strategies for glaucoma. Glaucoma is a disease of the optic nerve which affects the peripheral vision before progressing to involve the central vision. It is a common condition, with over a million assessments undertaken every year in UK glaucoma clinics. An important aspect of these examinations is the measurement of the peripheral vision by an automated perimeter. There are several testing strategies which can be employed by the perimeter to produce the final visual field result. We compared the two most commonly used algorithms. 

2^nd July: Dr Chris Wright, Senior Lecturer in Medical Imaging, talked about high resolution digital infra-red thermal imaging (DITI) and its application as “breast thermography”. This technology offers the prospect of simpler, cheaper, less invasive investigation of suspected breast cancer without the risk of ionising radiation. An audio recording of the session with slides is available here (but you need an UoE account).  

4^th June: Prof Hyde talked about his work with a group developing reporting guidelines on comparative accuracy studies – QUADAS-2C.

1^st May: We had the first TEST CLUB FOR STUDENTS! With financial support from the UoE Alumni Annual Fund, ‌the Test Club has now extended its activity to include undergraduate students. Students will have the opportunity to present and discuss their research projects; hear about the work of local researchers and participate in discussions on hot topics, such as the use of artificial intelligence in diagnostic decision making, or specific aspects of research related to medical tests. All sessions and the related slides will be available online and could be used as an educational resource. The first session included two presentations:

Steven Cox, Diagnostic Radiography Student, talked about peer support. He presented the results from a survey on the extent and impact of peer support within diagnostic radiography undergraduate programmes throughout UK higher education. He also shared his experience running the Medical Imaging programme’s Peer Assisted Learning Scheme (PALS), which aims to give first year students exceptional support.

Rob Meertens, Senior Lecturer on the Medical Imaging programme, talked about his research on near infrared spectroscopy (NIRS) for the potential application of measuring in vivo haemodynamic markers in bone tissue, an area currently without an accepted standardised approach.

To watch a full video of the session, please click here (accessible only to UoE staff and students! use Table of content to navigate between presentations).

4^th March: Dr Katie Pearson talked about her research on Cow’s Milk Protein Allergy (CMPA). CMPA is a growing problem with significant diagnostic challenges and life-long consequences for patients. Katie, who is a paediatric Academic Clinical Fellow and the mum of a child with CMPA, discussed the possible reasons for the increase in prescriptions for specialist milk formula, the diagnostic dilemmas that clinicians face when suspect CMPA and future research opportunities.

19^th February: Prof Chris Hyde introduced the MRC special call on methods research to support the assessment of diagnostic health technologies and the development of guidelines for diagnostic services and the rest of the meeting was open for discussion on new methodological approaches we could develop which would help both developers of tests and/or policy-makers in the short to medium term.

8^th January: Sarah Bailey from the DISCO group presented: “Only one poo this time”: Implementing and Evaluating Faecal Immunochemical Testing in the South West. In 2017, NICE guidance was published recommending the use of faecal immunochemcial testing (FIT) for patients with ‘low risk but not no risk’ symptoms of colorectal cancer (DG30). FITs measure the amount of haemoglobin in a single stool sample and replace the traditional faecal occult blood test (FOB). A large body of research has examined the use of FIT in screening for colorectal cancer, and in patients in secondary care settings, but there is a relative dearth of evidence for their use in the symptomatic primary care-based population. Since June 2018, FITs have been in use in the South West. In collaboration with the South West Cancer Alliance, the Sarah and her colleagues are conducting an evaluation of FIT in symptomatic patients. In this talk, she discussed the project and the challenges they were facing, as well as the potential implications of the findings.

In December’s Test Club (4th December 2019), Professor Chris Hyde talked about the test accuracy of PET-CT for different applications, drawing on experiences from a number of Cochrane reviews. PET-CT combines information about the functional activity, such as glucose up-take, of the brain (or other organ in the body) with structural information from a CT scan. As such it represents a potentially important improvement on information currently available from structure alone. The Cochrane Collaboration have completed a series of five systematic reviews on the test accuracy of different versions of PET-CT for the early diagnosis of Alzheimer’s disease dementia in people with mild cognitive impairment, but have consistently concluded the use of PET-CT cannot be recommended for routine clinical use. Chris was involved with two of these reviews and discussed the reasons why the Cochrane Reviews have come to their conclusions and ideas on a future research agenda to resolve the challenges identified. We also touched on how the accuracy studies of brain PET-CT illustrate some of the methodological challenges associated with test accuracy as an evaluative measure in tests generally.

For our September session of Test Club, we heard from Dr Waney Squier. Dr Squier has been at the centre of a fierce debate regarding the diagnosis of abusive head trauma in infants. She has argued, in publications and in Court, that there is little to no evidence that traditionally accepted views about the diagnosis of abusive head trauma are correct. Those who try to raise concerns about the quality of the medical literature on abusive head trauma are often met with accusations of misconduct, and the debate about  quality of this literature has become rancorous. It would be useful to have researchers with expertise in diagnostic test evaluation cast their eyes over some of this research, as the views of informed but neutral parties would be of great value to this discussion. 

Dr Squier focussed on recent guidance from The Royal College of Paediatrics and Child Health, who have endorsed a diagnostic model which they say can help clinicians detect the physical abuse of infants.  This model, called PRED-AHT, is presented as an "evidence-based resource available to clinicians across the UK and internationally to inform clinical practice, child protection procedures and professional and expert opinion in the legal system". This model was developed using cases drawn from the medical literature on abusive head trauma, and predicts that children with certain pathological findings, such as subdural and retinal haemorrhage, are likely to have been physically assaulted (Maguire 2011; Cowley 2015).

However, concerns have been raised about the quality this research. In particular, the criteria used to determine whether a patient had been abused relied upon court decision or the consensus view of a multidisciplinary panel, which use these same pathologies to diagnose abuse. The validity of the model appears based on a circular argument. This problem is common in the medical literature on abusive head trauma, and has been pointed out repeatedly, most recently by a comprehensive review by the SBU in Sweden. Despite this, many clinicians and researchers continue to claim that the specificity of certain pathological findings such as subdural and retinal haemorrhage is a well established fact, with a sound evidence-base. It would be useful to have a greater variety of researchers take a look at this literature.

In July's Test Club we heard from Dr Gareth Walker, a gastroentorologist at the Royal Devon and Exeter NHS Foundation Trust and Research Fellow at the University of Exeter. He will be talking about how his work on faecal calprotectin testing in primary care can reduce unecessary referrals and cut costs. The abstract of his recent paper is below, and the full paper can be found here:

Walker, G.J., Moore, L., Heerasing, N., Hendy, P., Perry, M.H., McDonald, T.J., Debenham, T., Bethune, R., Bewshea, C., Hyde, C. and Heap, G.A., 2018. Faecal calprotectin effectively excludes inflammatory bowel disease in 789 symptomatic young adults with/without alarm symptoms: a prospective UK primary care cohort study. Alimentary pharmacology & therapeutics, 47(8), pp.1103-1116.

Test Club in June was presented by Dr Brian Willis, GP and MRC Clinician Scientist at the University of Birmingham Institute of Applied Health Research (bio here). Brian presented his paper on the philosophy of science and the diagnostic process (available here) leading into a lively exploration of the philosophical underpinnings of diagnostic reasoning and why we come to the conclusions we do.

The abstract is below:

This is an overview of the principles that underpin philosophy of science and how they may provide a framework for the diagnostic process. Although philosophy dates back to antiquity, it is only more recently that philosophers have begun to enunciate the scientific method. Since Aristotle formulated deduction, other modes of reasoning including induction, inference to best explanation, falsificationism, theory-laden observations and Bayesian inference have emerged. Thus, rather than representing a single overriding dogma, the scientific method is a toolkit of ideas and principles of reasoning. Here we demonstrate that the diagnostic process is an example of science in action and is therefore subject to the principles encompassed by the scientific method. Although a number of the different forms of reasoning are used readily by clinicians in practice, without a clear understanding of their pitfalls and the assumptions on which they are based, it leaves doctors open to diagnostic error. We conclude by providing a case example from the medico-legal literature in which diagnostic errors were made, to illustrate how applying the scientific method may mitigate the chance for diagnostic error.

In May, we heard from Maria Olsen, on secondment from Amsterdam Medical Center with the Exeter Test Group and NICE. This topic forms a second part of her PhD project, which aims to propose a framework of preferred design strategies for the evaluation of biomarkers and medical tests. The talk was titled 'Improved labelling of diagnostic study designs: A classification'.

In diagnostic test evaluations, there is a lack of standards for describing study designs. For example, “case-control study” and “cohort study” are widely used, but they do not directly translate to diagnostic studies and their use can cause confusion and hinder several steps in research of diagnostic accuracy and evidence synthesis, in e.g. searching, reporting and quality assessment. We are currently developing labels that better describes diagnostic study designs.

In this presentation, Maria presented identified strengths and weaknesses of the currently used labels and following, present a preliminary framework for classifying a set of standard designs, important for the development of labels. The presentation closed with an open discussion about the preferred design domains and features that should be considered in the development.  

April's Test Club heard from Professor Tamsin Ford and Dr Tamsin Newlove-Delgado about their work on CAMHS standardised mental health assessments, with a talk titled 'Standardised Mental health assessment to speed Access to Treatment (SMART): developing a trial to test the effects of broad based diagnostic assessment at the point of referral to Child and Adolescent Mental Health Services'.

Emotional disorders (clinically impairing anxiety and depression) are common (4% of school-aged children) and less likely to be treated in child and adolescent mental health services (CAMHS) than other disorders. In a climate of limited resources, clinicians screening referrals to CAMHS need to be able to quickly and reliably identify children and young people who could most benefit from further assessment and treatment within their service. We recently submitted a full application in response to an NIHR Health Technology Assessment call for RCTs of standardised diagnostic assessments as an adjunct to clinical practice in CAMHS for children with emotional disorders. We will discuss our proposal and the challenges we encountered in terms of developing and justifying our design and in responding to the NIHR brief.

Test Club in March heard from Jennie Hayes on her work exploring "Autism Diagnosis as a Social Process: an exploration of clinicians’ diagnostic decision making".

Research suggests that diagnostic procedures for Autism Spectrum Disorder are not consistent across practice and that diagnostic rates can be affected by contextual and social drivers. This PhD project explores the factors that underlie clinicians’ decision-making around diagnosis of autism in both children and adults; and considers how social factors might influence decision-making.

February's Test Club was a different session, where we welcomed a guest, Prof Jeremy Grimshaw who was visiting from Canada. Jeremy was originally a GP, but now is a world-renowned health service researcher with particularly expertise in behavioural change interventions.

Jeremy and Chris Hyde have known each other for many years. Last year they were sad to lose a mutual friend, Fred Wolf - a very well known social scientist in the US. In the course of discussion about Fred’s achievements, Chris and Jeremy covered some interesting papers he produced exploring the nature of clinical decision making on tests.

These make very interesting reading, not least because they are over 30 years old and represent a type of research which has gone out of fashion. Chris was struck by the continuing relevance of the issues being discussed and to come across new things which may have been lost in the mists of time.

 The February session was therefore a journal club on one of these papers:

Wolf, F.M., Gruppen, L.D. and Billi, J.E., 1985. Differential diagnosis and the competing-hypotheses heuristic: A practical approach to judgment under uncertainty and Bayesian probability. JAMA, 253(19), pp.2858-2862.

In January's Test Club, we heard from Dr Nigel Oakes, a Pathologist at the Royal Cornwall Hospitals NHS Trust who gave a talk titled "Experiences of implementing a GP Ordercomms system for Pathology requesting".

Dr Oakes addressed why they decided to move to an electronic system for placing Pathology requests. There are numerous advantages but some pitfalls as well. He touched on how the system helps with demand management and alignment of patient pathways, and finished with a short discussion of how the system feeds into research projects.

In our December Test Club meeting, Dr Tristan Snowsill of the Health Economics Group presented on health technology assessments of testing strategies for Lynch syndrome:

Lynch syndrome is a hereditary cancer syndrome caused by mutations in the DNA mismatch repair system. Individuals with Lynch syndrome are at an increased risk of bowel cancer and other cancers. Testing strategies typically involve tumour tests in affected individuals to identify those at high risk of Lynch syndrome for constitutional mutation testing. Once a mutation has been identified in a family, it is simple to test other family members for the mutation.

PenTAG has conducted two assessments of the effectiveness and cost-effectiveness of strategies to test for Lynch syndrome in people with bowel cancer. Tristan describes the process, methods and results, as well as further work in the area.

November's Test Club heard from Dr David Llewellyn who presented some of his work titled "Detecting dementia: the pathway from clinical judgement to automation". A summary is below:

Dementia is not a disease. It is a heterogeneous, complex, multifaceted syndrome that typically occurs sporadically. How then are clinicians supposed to spot THE signs and symptoms that should trigger concern given this variability? NICE guidelines are vague, though no obvious alternative exists. That said, does it really matter if people with dementia remain undiagnosed? Conversely more people than ever who don’t have dementia are referred for costly and potentially worrying investigations at memory clinics that are struggling to cope. Given the current lack of disease modifying treatments for dementia the consequences of false negative misclassification are rather different from some conditions, such as treatable cancers. Nevertheless the current dementia referral pathway is not fit for purpose and the situation is in danger of spiralling out of control under the weight of all the ‘worried well’ false positives. I therefore instigated a program of work with the overarching aim of developing technology (DECODE) to enhance or eventually automate clinical decision making. Reasoning that we could teach computers to recognise hidden patterns in clinical data we have already developed algorithms that have the potential to match or even outperform human experts. In collaboration with Devon Partnership NHS Trust we have just received funding to conduct feasibility work using DECODE in primary care consultations and as a memory clinic triage tool with clinical oversight. We are also currently applying for funding to develop an online platform to pit clinicians against DECODE in a ‘man versus machine’ experiment. I will end by proposing an autonomy scale for intelligent clinical systems that raises a wide range of practical, ethical and regulatory issues (it opens a can of worms). There is little prospect of DECODE replacing clinicians altogether for some time, though this should ultimately be possible. In the meantime DECODE may change the way in which clinicians work and artificially enhance the dementia referral pathway.

In October's Test Club, Professor Andrew Shore discussed his work titled "High Frequency Biomarker Sampling Peri-operatively – a wine glass recovery".

High-frequency blood sampling peri-operative to the surgical insult shows the evolution of the biomarker responses such as CRP or the Complement cascade. The larger number of data points allow the trends to be considered almost as signals and processed as such. The surgical insult is an impulse to the system and the system responds sometime later once the de novo synthesis of CRP has occurred. The system then recovers with a first-order recovery – just like a wine glass ring-down – and recovery becomes predictable. The rapid sampling burden can be overcome with a point-of-care technology and we will show some first images of the our new technique.

In September, Pete Reader presented work on his PhD project “in vitro activation of the Complement cascade for predicting patient outcome post-operatively”. Pete's Director of Studies is Professor Andrew Shaw who presented in October's Test Club, and it was fascinating to hear about these two related projects.

Pete Reader's project is looking for classifiers of patient risk of complications following surgery. Classifiers may stem from observing the activation capacity of the Complement cascade - a fundamental component of the innate immune system.

We had a break in August.

In July's Test Club, we heard from two different projects linked by the common thread of modelling of test processes with the aim to increase diagnostic accuracy.

Firstly, Mike Allen talked about 'Dr James'. Can a machine learn which stroke patients should be given life-saving but risky clot-busting drugs?’

During a PenCHORD project looking at the acute stroke pathway in 7 regional hospitals it became apparent that physicians in different hospitals had different attitudes towards the risk/benefit of clot-busting ('thrombolysis') drugs. These drugs can significantly improve outcome, but carry an increased risk of intra-cranial haemorrhage. Hospitals with highly experienced stroke physicians were more comfortable in selecting patients to be given these drugs. As part of our research into the pathway we asked one experienced physician (Dr Martin James) to look at the stroke clinical audit record (which contains ~60 data fields) for patients at other hospitals to see if he could identify patients he was likely to give thrombolysis to. A question then arose, 'could a Learning Machine' do a similar job? Preliminary results into our first foray into Machine Learning are presented.
The potential use for such a machine (currently called 'Dr James') is to be part of the stroke audit process. The national stroke audit programme already highlights to all units how their performance compares with other units. We are currently working with the national audit team to implement a model predicting thrombolysis rates based on key process timings (and to help units see which process step improvements would lead to most overall improvement). We are looking at the feasibility that the next stage in this implementation could be the incorporation of 'Dr James' as part of clinical feedback (not decision making at this stage); a prediction of thrombolysis use would be made based on passing patient audit records through Dr James (which would be trained on decisions made in recognised centres of excellence in acute stroke care).

Exclusive use of open-source software and code is allowing us to port models directly to the national stroke audit team, and ensure any implementation of Machine Learning comes with full transparency of code, available to any and all who wish to interrogate (or improve) it. We are looking at how a dummy ('patient-like') data set may also be created and made available so that people can more fully test the operational of the model.
Michael Allen is in no way a specialist in Machine Learning. This work currently represents a hobby project carried out almost exclusively on the 7.19am train from Newton Abbot to Exeter.

Some links for those interested:

Make your own hand-writing recognition neural network from scratch

A broad introduction to machine learning in Python

Richard Stallman argues for Free ('freedom-respecting) Software

April’s Test Club was a discussion of research waste in trails of diagnostic test accuracy, using two papers as a starting point:
Research waste in diagnostic trials: a methods reviews evaluating the reporting of test-treatment interventions [full reference: di Ruffano LF, Dinnes J, Taylor-Phillips S, Davenport C, Hyde C, Deeks JJ. Research waste in diagnostic trials: a methods review evaluating the reporting of test-treatment interventions. BMC Medical Research Methodology. 2017 Feb 24;17(1):32]
Test-treatment RCTs are susceptible to bias: a review of the methodological quality of randomized trials that evaluate diagnostic tests [full reference: di Ruffano LF, Dinnes J, Sitch AJ, Hyde C, Deeks JJ. Test-treatment RCTs are susceptible to bias: a review of the methodological quality of randomized trials that evaluate diagnostic tests. BMC Medical Research Methodology. 2017 Feb 24;17(1):35]

In March’s Test Club, John O’Connor, consultant at the Royal Devon & Exeter Hospital spoke about a redesign of pathology reports with a view to patients receiving these directly and having greater control over personal management of long term conditions. Further reading can be found in his paper: O’Connor, J. D. (2015). Reducing Post Analytical Error: Perspectives on New Formats for the Blood Sciences Pathology Report. The Clinical Biochemist Reviews, 36(1), 7–20.

Sebastian Rachuba discussed his work on simulating the diagnostic pathway for chest pain patients in emergency departments. The abstract is below:

Patients presenting with chest pain at an emergency department in the United Kingdom receive troponin tests to assess the likelihood of an acute myocardial infarction (AMI). Until recently, serial testing with two blood samples separated by at least six hours was necessary in order to analyse the change in troponin levels over time. New high-sensitivity troponin tests, however, allow the inter-test time to be shortened from six to three hours.
Recent evidence also suggests that the new generation of troponin tests can be used to rule out AMI on the basis of a single test if patients at low risk of AMI present with very low cardiac troponin levels more than three hours after onset of worst pain. We demonstrate the use of a discrete event simulation model to assess the likely impact on the number of hospital admissions if emergency departments adopt strategies for serial and single testing based on the use of high-sensitivity troponin. Historic data from an acute trust in the South West of England is used to quantify the resulting effects.

In February’s Test Club, Dr Mitra Tavakoli talked about her work investigating the use of in vivo corneal confocal microscopy as a valid surrogate end point for human diabetic neuropathy. Download slides

 In January's Test Club, we head Professor Tamsin Ford and Dr Tamsin Newlove-Delgado about their work on CAMHS standardised mental health assessments, with a talk titled 'Standardised Mental health assessment to speed Access to Treatment (SMART): developing a trial to test the effects of broad based diagnostic assessment at the point of referral to Child and Adolescent Mental Health Services'.

Emotional disorders (clinically impairing anxiety and depression) are common (4% of school-aged children) and less likely to be treated in child and adolescent mental health services (CAMHS) than other disorders. In a climate of limited resources, clinicians screening referrals to CAMHS need to be able to quickly and reliably identify children and young people who could most benefit from further assessment and treatment within their service. We recently submitted a full application in response to an NIHR Health Technology Assessment call for RCTs of standardised diagnostic assessments as an adjunct to clinical practice in CAMHS for children with emotional disorders. We will discuss our proposal and the challenges we encountered in terms of developing and justifying our design and in responding to the NIHR brief.

2016

In August we had a break.

In July, Prof. Willie Hamilton talked about his research about cancer daignostics in primary care.

In March, Dr Antonieta Medina-Lara, Senior Lecturer in Health Economics, talked about planned evaluations of tests she had been involved with. We were also joined by Prof. Patrick Bossuyt, Visiting International Academic Fellow to the University of Exeter from the University of Amsterdam, where he is Professor of Clinical Epidemiology and Head of the School of Public Health.

In February, Dr Obioha Ukoumunne and Prof. Chris Hyde presented the results from an NIHR funded evaluation on school entry screening for hearing loss which had recently been completed. Chris talked about the overall plan of the project as an example of a complex evaluation strategy; Obi talked about the accuracy evaluation of two potential hearing screening tests – an interesting example of a comparative accuracy study.

In January's Test Club, Dr Jaime Peters talked about the use of decision models to evaluate testing strategies for monogenic diabetes (the UNITED study).

2015

In December's Test Club (8th Dec) participants presented their current work projects and the STARD guidelines were discussed.

The first meeting of the Test Club took place on 2nd November 2015. The following paper was discussed: Tsoi et al. Cognitive Tests to Detect Dementia: A systematic review and meta-analysis. JAMA Intern Med. 2015 Sep;175(9):1450-8.