Theme: Predictive Validity and Patient & Public Involvement

Download a PDF of the abstracts: Predictive Validity and Patient & Public Involvement

Parents as new Educators- the student perspective

Dr Roisin Begley, UCL Institute of Child Health and Whittington Hospital, rbegley@doctors.org.uk

Background

The General Medical Council have highlighted that the roles of patients in medical education are changing. There is greater consideration to what patients and/or relatives themselves can contribute to the educational process and what the benefits or harms are to them in doing so. It is common practice for real and simulated patients to be used in assessment and examinations and to provide feedback. Little research exists looking at their contribution to written formative or summative assessments.

Summary of work

Our study aimed to evaluate the possible benefits of using ‘expert’ parents to mark and provide text feedback on students’ written assessments. The students on UCL’s iBSc paediatrics and child health were asked to write a reflective essay on patient journey sessions. The purpose of this assignment was to improve the students’ skills in reflective practice: using their own and others’ experiences along with relevant literature to improve their future practice.
Building on our work that compares parent markers to medical (academic) markers and peer markers, we are now analysing student responses to parent feedback and exploring the usefulness and contribution to student learning. Students were given qualitative feedback and grades from a peer marker, medical marker and parent marker. We will explore students response to parent feedback in a focus group and they were encouraged to compare and discuss the usefulness of feedback for learning and the possible benefits or harms to using parents in written assessment of medical students.

Conclusion

We believe parents/patients have an increasing role to play as new educators but further research is required.

Take-home messages

Parent markers add a human perspective to clinical paediatric care that is much needed for holistic training of future doctors.

 

Crossing boundaries: working with adolescent smokers to improve the behavioural change skills of future doctors

*Peppas I, Moreiras J, Datt C, Sharma S, Fertleman C

Transition points in people’s lives represent an important opportunity for intervention in health-related behaviour. Adolescence constitutes a major transition point with unique challenges and opportunities for intervention by health care professionals. The significance of this is exemplified by the case of adolescent tobacco smoking, as 40% of adult smokers in the UK begin smoking before the age of 16. Evidence suggests that interventions involving motivational interviewing can be beneficial, but those delivering it must have been exposed to appropriate behavioural change skills.

The GMC has emphasised the need for new educational strategies to allow medical students to access real life-situations, but also to support the patients’ right to influence the attributes of newly qualified doctors. In this context, we organised a workshop with adolescent smokers and medical educators to identify which behaviours and attributes of doctors would facilitate their engagement with smoking cessation services. Together, we developed an OSCE to assess the communication skills of penultimate year medical students, in which adolescents played a central role from inception to the final assessment.

*Corresponding Author Dr Ioannis Peppas, University College London, ioannis.peppas@nhs.net

 

Using patient journeys - assessment really driving learning

*Macaulay CP, Powell P, Fertleman CR

Reflection and patient-centeredness are central to the GMC’s requirements of medical school curricula. UCL’s iBSc in Child Health (the first and only one of its kind in the UK) began in 2010. Central to the programme is the study and discussion of patient journeys. Students spend time discussing families’ experiences ‘their physical and emotional journeys’ using several different methods: they have small group sessions with families where parents or young people talk about their own experiences: they have fortnightly seminars facilitated by a child psychiatrist discussing journeys that have been collected by others using questions as prompts for themes to explore; they follow families with new babies along their own ‘journeys’ visiting them at home and accompanying them on medical visits.

At its inception, students were asked to write a reflective piece on one of the patient journeys they had discussed as part of their assessment. As the course developed, the value of this part of the course became increasingly clear in student feedback. Students are now required to collect their own patient journeys, from families they come in contact with, and write them up for future groups to learn from, thereby providing a bank of ‘journeys’ for subsequent years. This also forms part of their assessment. The collection of journeys involves many different skills ‘communication, distillation of information, reflection’ all required by the GMC. It also represents an innovative approach to assessment. These patient journeys have also been compiled into a book in press - for more students to study. Assessment driving learning in more ways than one!

*Corresponding Author Dr Chloe Macaulay, Evelina London Children's Hospital, chloe.macaulay@gstt.nhs.uk

 

Can A Formative Integrated Clinical Anatomy Spot Test Predict Students’ Performance in Summative Applied Medical Knowledge Assessments?

*Chinnah T, Leitner C, Brandom K, Rice N, and Devaraj V.

Integrating knowledge of anatomy with other biomedical sciences is very important in clinical practice. We developed a formative integrated Clinical Anatomy Spot Test (iCAST), which focuses on students’ ability to demonstrate clinically-relevant factual knowledge. This study evaluates its validity and predictive value on students’ performance in summative applied medical knowledge (AMK) assessments. The iCAST was first piloted with 36 year 2 students in 2012/3 and rolled out to year 1 students in 2013/4 (n=130) and 2014/5 (n=129). It is delivered termly to years 1 and 2 students and requires students to rotate through 50 stations of integrated short-answer questions. Immediate feedback is provided, scripts are peer-marked and scores are independently verified. In both cohorts, average scores in the iCAST correlated with scores in the summative end of year 1 content-specific knowledge assessment (2013/4: r2=0.46, p<0.001; 2014/5: r2=0.73, p<0.001), and with scores in the final summative AMK progress test (2013/4: r2=0.44, p<0.001; 2014/5: r2=0.57, p<0.001). Multivariable regression analysis adjusting for baseline AMK, shows that term 1 iCAST scores were associated with performance in the end of year 1 knowledge assessment (p<0.001). The students’ response to the exercise was positive with them indicating that the tool would them consolidate their learning and identify areas for further studies. Performance in the iCAST correlates with performance in the summative assessments of AMK. Term 1 iCAST and the end of year 1 knowledge assessment performances showed strong association. Our evidence suggests that integrated assessment is an effective approach for learning enhancement and promotion of knowledge integration. It also shows that the iCAST is a strong predictor for students’ summative applied medical knowledge performance. The iCAST is a very popular formative assessment that closely predicts outcomes in the progress tests and may serve as an indicator for identifying students needing support.

 

A GP curriculum within the medical school curriculum

*Professor Sir Denis Pereira Gray, University of Exeter, denis.pereiragray@btinternet.com

Medical schools have a responsibility for producing doctors for the country and in the UK this means the NHS. The NHS is clear it wants half of all medical graduates to enter postgraduate training for general practice. However, only 19% of medical students see general practice as their first choice career. Few medical schools teach the theory and principles of general practice apart from the diseases seen in general practice. Many medical schools have no teaching from academic staff on the principles of general practice in any of the five years. General practice teaching is effectively outsourced to general practices who have little or no support in illustrating the principles. There is therefore an urgent need for an undergraduate curriculum for general practice within the general curriculum of each medical school and the Health Select Committee of Parliament has recently recommended (para 144, Woolaston Committee) that general practice/ primary care be taught as a subject in each medical school and that the General Medical Council should hold schools to account for this. This presentation presents the first specific curriculum for general practice and therefore provides a basis on which learning about it can be assessed. These principles should be assessed in the final MBBS examination. This curriculum can be covered in 20 hours within the five-year medical course by medical school staff, but needs to be supplemented with educational support for the GP teachers, who should have adequate general practice libraries and files of key articles available in their practices. They can then illustrate these principles operating in practice during day to day consulting with patients. Reference Pereira Gray D. (2016) an undergraduate curriculum for general practice. Starfield Memorial Lecture, University of Exeter 2015 (submitted for publication)

 

Early identification of the struggling learner using learning analytics: Opportunities and Challenges

* Chatterjee A, Burr S, Zahra D, Gabe-Thomas E

In recent years, the conception of assessment as a summative function (i.e. assessment of learning) has been broadened toward the conception of assessment as a formative function (i.e. assessment for learning). Assessment is at the heart of formal higher education. Feedback is most effective when highly related to clearly identified learning goals so that effective formative feedback is not only based on monitoring progress toward the specific goals but also promotes students to develop effective learning strategies. These processes characterize formative assessment and are aimed at supporting learning. Within Clinical Education, a range of approaches are currently being used where summative assessment data sets are used for identifying learners who might require additional support and enhance scaffolding support. Siemens (2010) defines Learning Analytics as the use of intelligent data, learner-produced data, and analysis models to discover information and social connections, and to predict and advise on learning. Even though very promising the technology to deliver this potential is still very young and research on understanding the pedagogical usefulness of Learning Analytics is still in its infancy (Johnson et al., 2011b; Johnson et al., 2012). The authors will present a range of approaches, opportunities and challenges on how to best harness the state of the art learning technological advances. Siemens, G. (2010). What are Learning Analytics? Retrieved July 29, 2011, from http://www.elearnspace.org/blog/2010/08/25/what-are-learning-analytics/
 Johnson, M.W., Eagle, M.J., Joseph, L., & Barnes, T. (2011a). The EDM vis tool. In M. Pechenizkiy et al. (Eds.), Proceedings of the 3rd Conference on Educational Data Mining 2011 (pp. 349-350). Eindhoven, The Netherlands: International EDM Society. Johnson, L., Smith, R., Willis, H., Levine, A., & Haywood, K., (2011b). The 2011 horizon report. Austin, Texas: The New Media Consortium, Retrieved July 29, 2011, from http://net.educause.edu/ir/library/pdf/HR2011.pdf

*Corresponding author Dr Arunangsu Chatterjee, Plymouth University Peninsula Schools of Medicine and Dentistry, arunangsu.chatterjee@plymouth.ac.uk