‘Everyone’s pretending…paying lip service to certain ideas or concepts that things that are supposed to be meaningful. And, again, my fear would be that because of that sometimes we [SLT] lose the ability to do other things that actually are meaningful.’ Deputy Head Teacher, Secondary School, Kent.
In Homo Deus, Yuval Harari points out that the invention of writing endowed scribes with the ability to record and share systematised versions of reality. This is of course mainly great. It’s in my top five of ‘reasons why I’m glad to be a human.’
However, in a warning which reminds me of that time when a weedy boy was bitten by a radioactive spider, Harari points out that with this great power comes great responsibility.
Sometimes pressures to succeed can systematically nudge scribes away from accurately portraying things. They record what they (or their bosses) wish things to be rather than how they are. Yuval Harari uses the example of Chairman Mao’s reforms which included the intention to treble grain production.
Pressure to meet this ambitious aim resulted in officials ‘writing in’ additional grain. They massaged or falsified the figures to make their region or district appear productive. Recorded grain levels looked great. The huge (apparent) surplus was sold to western powers in exchange for money/weapons etc. This was immediately followed by a devastating famine.
I did wonder if it’s appropriate to compare this tragedy to the pressure teachers feel to ‘show progress’. However something resonates around the idea that school leaders can be co-opted into creating a version of learning which meets external/political expectations but only at the expense of becoming detached from reality.
Here are four examples (mostly from a gallery of data nonsense wonderfully curated by
And here’s a window into the more complex individual stories which the above examples conveniently ignore. It helpfully indicates the range of outcomes which we could expect – of different individual students who scored similarly when they were in primary school. Essentially there is no ‘should’ about it. One particular grade may be slightly more probably than another based upon prior data. That’s it. Given a large enough group, we would expect many students to gain higher and (lower) grades than they apparently ‘should.’ (Thanks for this graph originally in this article)
Yuval Harari argues that when recorded and actual reality clash, the former often comes out the victor. Many school leaders’ jobs have become a desperate attempt to make sure that individual pupil learning falls in line with the ‘flight paths’ which they have created in an attempt to meet the perceived demands being made of their school.
Rather like with Mao’s grain reforms, there is considerable pressure on many teachers to write in data which matches policy (three sub-levels of progress anyone?!) rather than actual test results. It’s worth pointing out here that whilst the Mao’s officials probably had a reliable way of measuring grain – counting sacks say – teachers don’t have equally reliable ways of measuring learning.
We teachers are asked to generate grades from these answers ‘as-if’ it was the end of a course and pupils had actually been asked a larger number of different questions about a broader range of topics. This in itself is highly dubious before you consider whether this ‘grade’ aligns with the neat flight paths shown above.
As a result, many teachers and leaders go home worrying that deeply unreliable grades given months (or years) before a terminal qualification don’t match the grade which perhaps a quarter of students with the same starting point tend to get in the end anyway.
It’s pointless worry. Whilst finely calibrated in one sense – some schools can worry about kids by sub-grade (D1’s for example) rather than just whole grade – it’s also low on useful information. What (if anything) should a teacher, leader or student do differently?
The system requires particular behaviours – targeted interventions for underachievers for example- in order to help make data fall largely in line with the flight path. So this is how people behave. Often without adequate thought about why we do it or what we’re actually trying to achieve.
‘differentiation’ ‘intervention’ ‘demonstrating progress’ ‘targeting kids’ are recent examples of our attempts to bridge the gap between how we think things are and how we would like them to be. This sort of language emerges when we try to come up with ways of making one written version of reality (the grades we’ve just entered/pulled from the air match another (the progress lines above for example)
I suspect that teachers also (on some level at least) want these things to be real as by achieving them they would be doing something meaningful. Moving kids up a grade or two; closing the inequality gap; successfully intervening where a child (or teacher) was underperforming are satisfying things to achieve.
However when one digs deeply into the underlying principle that the data shows how things ‘should be’ i.e. John should be working at an A*, he is working at a B, this means he is underperforming, we find that this is a deeply inappropriate application of aggregated data for students with similar starting point to Johnny.
There is no should about it. Actually all we know is that John might be working around a B. For 100 students who scored similarly to him at GCSE, a number (30 say) gained an A grade or higher. Some didn’t. Either way none of them are him.
As a result we often worry a lot and sometimes work really hard trying to achieve things which are at worst nonsensical or at best ill-guided use of resources and time.
If I want to improve the quality of my teaching (and its impact upon learning) I don’t think that flight-paths blended with fortune telling is the best route I should take.
I am very much on board with the idea that schools can (and should) work hard to ensure that they teach students well as possible. I long ago jumped off the good ship Mid-Year-Intervention though. I try to teach as well as possible from the start of the year. I’m not keeping back extra tips and tricks back for apparent ‘under-performers’ in March.
Take the perennial case of ‘mock exam results induced Ofsted panic.’
I can confirm this does happen in multiple schools.
Teachers ask real students to answer actual questions. They then mark these questions. The marks (largely) represent the extent to which students correctly answered questions about specific parts of a course.
However the ‘systematic’ school reality demands that this is given a grade – a grade that is taken to indicate what students will achieve in 4 months time, in a different paper, with different questions, different markers and when different experiences (including but not limited to learning experiences) having happened in between.
In the particular example we’re interested in here, teachers have come back from the Christmas holidays a little grumpy and pessimistic about the future (it is Dry January after all). So enter grades which are lower (in general) than those which kids at that school regularly achieve.
Maybe they’re not grumpy and it’s just that as yet students have not learnt as much as they may have done in four or five months time…
Maybe the students are less worried about these ‘internal’ exams so prepare less immediately before hand and decide not to swing at the more challenging questions pitched their way (while they’d probably at least flail at them if it was more important)…
Maybe the students do actually know less, at this point than last year’s cohorts…
We don’t really know why, but grades have come out somewhat low.
Nevertheless, worryingly low grades have been entered. Now a systematic prediction of how the school will look relative to other schools in August looks really bad.
This induces panic and worry. i.e. “If they get these grades then we will get ‘the call’ and we won’t be good or outstanding anymore” (both as it happens examples of recorded realities in themselves but that’s another blog post).
These worries provoke action in the school. Normally they include whole-staff briefings; directions to focus on borderline kids; intervene; raise aspirations; help the kids who with a little boost could improve out P8 score for examples.
Staff in two schools I’ve recently visited said that this exact thing had happened to them recently.
In one school the worry in March/April turned out to be misplaced. The P8 scores for that cohort ended up being the best ever.
In the other lots of ‘work’ ensued following the immediate grade collection. SLT meetings, HoD briefing, staff briefing, agreeing lists of kids to ‘target.’ A week later. 5 working days. At most three lessons per subject. There was another data drop.
It was recommended that teachers entered another grade – perhaps one more realistic (but optimistic) for each student. The teaching staff, as a whole complied with this ‘steer’! In this version of reality the school A*-C rates looked good again.
One of the senior leaders informed me that he was feeling a lot better now that the issue had been resolved (!) What had actually happened was that a worrying vision of the future had been replaced with a better one. Learning was (as far as I could tell) unaffected by this particular squall of worry in the upper echelons of the school.
What about workload?
These sorts of nonsense events more than likely contribute to workload issues for staff. They are highly unlikely to lead to improved teaching and learning than. It helps here to look closely at what are commonly considered to be the three key features of burn-out.
- Low feelings of self-efficacy.
I will finish by pointing to some ways in which the nonsense behaviour outlined above might have a detrimental effect upon burnout….
- Being told that they am clearly not teaching well enough (why else would grades be lower than they ‘should-be’). Probably doesn’t help teacher’s often fragile sense of self-efficacy
Joan, a fantastic teacher of English and school leader spoke to me about her constant feelings of guilt and that she wasn’t doing enough or more precisely – being good enough.
‘I think most teachers feel guilt and responsibility…You’re never finished, you can never be finished. It’s never as good as you want it to be…There’s always someone out there who seems to be doing it better than you… that idea of guilt…that sense that there’s always something else that you can do to be better.’ Joan talked about the stresses of ‘of being a school and doing your best and not being good enough.’ (Joan, Secondary School English Teacher)
- The drive towards ‘targeting’ students who are ‘under-performing’ may contribute to de-personalisation. This is seen as unhealthily high where teachers start to see students as barriers to them achieving their aims rather than the individual people who they have entered the career to serve.
A popular data management package now includes a ‘what-if’ function. You can play around with the grades you allocate to individual students to find out what your ranking (compared to other centres) would be like if they scored higher or lower. Not only is this inappropriately conflating average performance of similar kids with the idea that we can know what any one child ‘should’ get, but for me it quite literally presents the lower achievers in my class as a barrier to me looking good at my job (or possibly even gaining a pay rise when my performance is reviewed).
- Being asked to complete futile tasks is a recipe for feeling overwhelmed and unable to continue. Exhausted in other words.
Marc, a primary school teacher spoke about what was most likely to make him feel overwhelmed and exhausted at work.
I think sometimes if you’re going through and doing things, a little bit like with your lesson planning or with your marking, for the sake of ticking a box of, ‘I’ve done it,’ then it will get very, very monotonous…as soon as you’re in that frame of mind then it can become overwhelming.’ (Marc, teacher, primary school)
In his famous Mcdonaldisation Thesis, Ritzer argues that where industries become obsessed with highly rationalised efficiency then irrational practices ensue. Ironically, highly rationalised systems in themselves can produce irrational behaviours. Identifying students who should be better for targeted interventions to uplift them by a sub-grade. Is probably one of these.
The systems above are nonsensical. Pretending otherwise may help with short term worries but is unlikely to help attainment or staff morale.
Here’s a big challenge for SLT teams. Sometimes we discuss the recorded version as if it is more accurate than – we know– it actually can be. This is a dangerous game to play. Here in a candid interview a deputy head teacher told me that:
We spend huge amounts of time looking at data that’s often meaningless, yet we all play along with the idea that somehow it’s meaningful. And I think everyone perhaps in the management team…. I think almost everyone does probably up to and including, you know, the principal of the school [knows this]… but again, it’s this idea that we have to use this information and we have to be able to present it in a certain way, so we do. And therefore we all pretend it’s meaningful when actually it isn’t.’
‘Everyone’s pretending…paying lip service to certain ideas or concepts that things that are supposed to be meaningful. And, again, my fear would be that because of that sometimes we [SLT] lose the ability to do other things that actually are meaningful.’
This then produces nonsense tasks for teachers.
Let’s cut it out out.
Ben White is a teacher and research lead. He has been research data use and workload for the past few years. This has included producing this research piece on data use and workload. He was a member of the data and workload group which produced this report.
This year Ben has been piloting a data use audit with school senior leaders. It is designed to identify practical ways of improving data workload and refining the way in which your school uses data. It is informed by the above research and the promising direction which the new Ofsted framework is heading in relation to data use.
‘The session with Ben was a great opportunity for the school to re-evaluate data practices in light of evidence and the new Ofsted curriculum. Evidence was provided around the validity and reliability of working at grades, target grades and flight paths that will drive a discussion at the school and across the Trust about their use and impact on workload. Solutions were provided in the discussion that will drive and define our data priorities for the 2019 academic year, whilst making sure teacher workload is driven towards purposeful practice.’
Jake Steers, St Paul’s Way Trust School, London.
If you wish to discuss these issues further , or are interested in the audit process. He can be contacted via twitter or at email@example.com