Measuring learning: from old exams to new technologies – Paul Kelley #educationfest


“Our exam system isn’t a gold standard; it’s a legacy system”

Our data says that we are getting better, with grades at A level increasing each year. However, PISA and OEDC studies show a different story. The external measure of the British system, Kelley claimed, say that literacy is going down.

Columbia, Georgian, Norway and New South Wales are using new technologies to drastically reduce costs and timescales associated with exams, as Ewan McIntosh reported some time ago…

Cost is one thing, but what about speed? When exam results come immediatly rather than in a brown envelop months later, could the summative assessments often accused of being of little use to learning actually become formative?

He shared John Hattie’s evaluation of the effectiveness of interventions. These results are suprising, and some teachers in the room obviously disagreed with some of them, in particular the evidence that ability grouping has no benefit to learning. Many of these findings don’t ‘feel right’ to teachers, both new and experienced. The question we need to ask is do with go with evidence to shape our practice or gut feeling? ;


Kelley argued that when you have rigerous, regular and quick assessment, you begin to move to place where educational practice can be specifically evaluated in terms of what makes a difference. At present, he characterised our educational system as where medicine was in the days of the leech; we are applying things based on a hunch they will work.

It strikes me that the biggest problem we have here is engaging school teachers with research, so that their criticism can move from it ‘not feeling right’ to more critical questions of methodology and validity. In our current system there is not the time or the culture of engaging with this kind of work in a sustained, embedded way, and until there is we will continue to work on our hunches despite being told the ‘evidence’.

Related posts:

Joined up thinking - Carl Gombrich #educationfest (@carlgomb)
Building aptitudes for criticality - Guy Claxton #educationfest
Change 3 numbers to change education - Mona Mourshed (McKinsey) #educationfest


  1. Great notes, thanks.

    Not sure if you attended Ben Goldacre’s talk – he implored teachers (and those deciding on public policy) to use randomised control trials to test ideas.

    This resonates completely with your point abut research, I think, and I would love to see a randomised control trial for the methods Kelly described…there are definitely other factors (lovely school building! Great headteacher!) that need to be ‘controlled’ out to prove this works.

    Goldacre co-wrote a guide on this for the Cabinet Office “..the paper is readable and straightforward, like the Ladybird Book of Randomised Policy Trials, and I really hope you’ll enjoy reading it. It’s a good primer on basic research methods, and on how to do a trial properly in any domain, with clear examples taken from the real world of medicine, business, teaching…”

    I missed the first part of the talk, so maybe a randomised control test has been done – I hope so, because I love the ideas Kelley showed – they instinctively ‘feel right’ to me, but ‘feeling’ is not trusted proof.

Leave a Reply

Coding across the curriculum - Miles Berry @mberryNicky Morgan: Technology key to reducing teacher workloadFlexible learning space opening at Westfield Junior SchoolLearning to swim - Eylan Ezekiel #wherenext @EylanEzekiel Creating a sustainable education system - Stephen Heppell @stephenheppell #eduict2013Tim Rylands - Back to their future #bett2013

Creative Commons Licence

Oliver Quinlan: Learning, teaching, technology by Oliver Quinlan is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported License. Based on a work at beyond the scope of this license may be available at