Workshop details

An introduction to generalisability theory

Date: Friday 14 October

Facilitator: Lee Coombes, Centre for Medical Education, Cardiff University School of Medicine

Workshop description

Generalisability (G) theory is a statistical framework that is used for examining reliability in assessments. It is based on the idea that any measurable observation can have multiple sources of variance that can influence the score. Through statistical modelling, G-theory attempts to identify variance components that contribute error to a measurement estimate. It can then be used to investigate the impact of changes to an assessment on the reliability coefficient that it produces, and a strategy put in place to reduce error and increase the reliability.

This workshop aims to be a beginner’s guide to G-theory and its application, and will be a simple, understandable, and practical introduction so participants will be able to use what they learn on their own data. It will provide participants with two examples of G-theory being applied to simple knowledge and clinical assessment formats. It will also show how a decision (D) study can be used to investigate the impact on reliability of changing the assessment.

Objectives of the workshop, indicating knowledge, attitudes and skills that participants should gain from it

At the end of this session, participants will be able to:

  • Apply G-theory methods to simple knowledge and clinical assessment data sets
  • Calculate absolute (phi) and relative (g) standard error of measurement
  • Calculate absolute (phi) and relative (g) reliability coefficients
  • Explore changes to assessment reliability using a decision (D) study

Maximum number of participants

Room capacity