Professor Neil Johnson outlines why he supports the concept of introducing a Medical Licensing Assessment (MLA) in the UK.
I’m writing this as the Chair of the GMC’s Expert Reference Group for the proposed Medical Licensing Assessment. My day job is as Dean of Health and Medicine at Lancaster University (where I am Professor of Medical Education) and I have a clinical background as a General Practitioner.
Why do I support a Medical Licensing Assessment?
My interest in assessment goes back more than twenty years. One of the lessons I’ve learnt along the way is just how challenging it is to design good assessments. In particular, I’ve always been well aware that standard-setting in assessment is based on making a best judgement.
This was particularly highlighted to me when I was faced with a series of student appeals a number of years ago. How confident was I that the standards we set in our particular medical school were the right ones? Might they be too high, or could they even be too low?
Sharing the problem with an external examiner helped to some extent – but I could still only compare our standards with those of a very small number of other schools and through the lens of individual interpretation. This led me to begin to question whether a national approach to assessment might be desirable.
My interest in assessment led to my involvement in the Medical Schools Council Assessment Alliance (MSCAA) which I now chair. This is a group comprising representatives from all UK medical schools who work together to bring about improvements in the assessments we use in undergraduate medical education.
Through the MSCAA I became involved in a project in which medical schools share written questions. One of the findings of that project was that schools differ in the standards they set for the same questions (see the paper in full – I am one of the contributing authors).
To me this meant that I couldn’t say to a student or to a member of the public that all students were being passed using exactly the same standard.
I must emphasise that I’ve seen no convincing evidence to suggest that medical schools pass students who shouldn’t be passing. So I accept that my worry is based on a potential problem rather than a problem that has been proven to exist.
Nevertheless, this potential variability in the standards required of new doctors as they graduate troubles me and it’s for this reason that I very strongly support the concept of a Medical Licensing Assessment (MLA).
How can we turn the concept into reality?
My experience has shown me that assessment concepts are all well and good – but what really matters is whether or not they get translated into programmes of assessment that do what they should do, and do it well. And it is this focus on turning concept into a high quality reality that led to the GMC to establish an Expert Reference Group for the MLA.
The purpose of the Group is to advise the GMC on how best to develop a valid, reliable, feasible and acceptable programme of assessment to inform decisions about the award of a first licence to practise.
In forming the ERG the GMC has brought together people with a range of assessment expertise – some with a deep understanding of the design and testing of assessment, and others with particular interest in its purpose and what it needs to do.
We’ve now met twice. In our first meeting we began to identify questions that needed to be addressed when designing a new programme of assessment. In the second meeting we started to look in detail at some of those questions.
What is clear is that there are few ‘easy’ answers. As always with assessment, balances will need to be struck. For example maximising the authenticity of an assessment tends to make it complex, expensive and difficult to replicate.
Conversely, a high stakes assessment must be reliable – but achieving very high reliability can make an assessment unfeasibly long and inauthentic. This doesn’t mean that the challenges are insurmountable – but it will take time to resolve them.
I’m a strong believer that the more you listen, the greater the chance that you’ll make the right decision in the end. And that, for me, is the purpose of this consultation. We need to hear the fullest possible range of views so that we can both get a sense of the balance of views on key questions, but also so that we have the chance to hear something that offers a new angle on a knotty problem and which, in turn, could help us to find the best solutions.
So – please do respond to this consultation. I certainly see it as a genuine opportunity to shape what I see as one of the most important steps in medical education for very many years.