The Magnificent Seven: The compelling evidence on TA impact and where we go next
30 October 2016 by Rob Webster
At the end October, the TES reported findings from a research study that found primary-aged pupils made more progress in literacy when they followed a pen-and-paper course compared to those who took an online version of the same course. Compared to a control group, children who received online reading support made two months progress, whereas those receiving the offline alternative made a gain of an extra month.
The TES report [paywall] focused on the implications of pen-and-paper vs. online learning – all of which was relevant and interesting. Yet there was another, unreported, story that deserves attention. The ABRA study was the seventh independently evaluated trial of a structured literacy or numeracy intervention programme funded by the Education Endowment Foundation (EEF). The results of each one of these seven trials – all completed within the last four years – have reported positive findings. Across the trials, pupils made, on average, three months additional progress compared to those in the ‘business as usual’ condition.
The significance of these accumulated results can’t be overstated. Consider first of all, the rarity of consistency. To date, the EEF has published 60 independent evaluations of its funded trials. Fewer than 30% have returned a positive finding, indicating that the majority of interventions (on a wide range on topics) have no measureable impact on attainment. Yet the seven trials relating to TA-led interventions (12% of those completed) have a 100% success rate.
A requirement of the randomised control trial (RCT) design favoured by the EEF is that pupil samples are sufficiently powered to ensure that if an effect is found, the statistical probability that those results occurred by chance is pretty much ruled out. So to see consistent effect sizes lining up, one after the other, seven times in a row, within such a relatively short span of time is – in research terms – virtually unheard of. Add to this the fact there is compelling evidence showing exactly the same thing is happening in the American education system, and it's beyond doubt that there’s something unignorable about what we’re observing in relation to TA impact.
Secondly, there’s the significance of the ‘business as usual’ comparison condition. There’s good evidence (including from our own DISS and MAST studies) to suggest that the ‘business as usual’ condition isn’t no interventions – quite the opposite, in fact. The control schools in the RCTs were, in common with many schools, almost certainly running lots of interventions. So, these remarkable findings compare to that scenario.
Imagine if this happened on a different theme: say, for argument’s sake, synthetic phonics or ability grouping. You can bet the family silver on it being cited as the basis for an unstoppable policy proposal. But before we get carried away, the practicalities of a proposal to deploy TAs to lead interventions isn’t straightforward. Delivering interventions away from the classroom accounts for only around 40 minutes of a TA’s day*. The majority of TAs spend the majority of their time in classrooms, supporting lower-attaining pupils and those with SEN.
The numbers involved are pretty immense too. There is the full-time equivalent of 227,000 TAs working in mainstream schools in England. Based on actual headcount, it's actually around 343,000 TAs; that's comfortably more than the entire population of Iceland! A significant increase in the number of TAs delivering catch-up programmes, or the proportion of time they spend doing this, will have a knock-on effect for pupils: they would spend less time in the classroom with the teacher.
As my colleague, Jonathan Sharples, senior researcher at the EEF, says: the issue is not whether schools should do interventions programmes, it’s how they’re delivered in terms of precision and rigour, along with how the programmes are scheduled, coordinated and cohere with the wider curriculum. It’s clear there is a balance to be struck.
The story about TA-led interventions is, unquestionably, a good news story. But there’s still work to do. Evidence of the effective use and impact of TAs within classrooms is, in comparison, less mature.
This brings us to another recent announcement by the EEF: the awarding of a grant to implement and evaluate our integrated approach to maximising the impact and practice of TAs in classrooms. It’s a first for the EEF and UK research. Evaluating our programme will help to fill the gap in relation to TAs' classroom practice and how schools and teachers deploy TAs to best effect.
Our MITA project is a collaboration between UCL Institute of Education, the University of East London and the London Leadership Strategy. It will be conducted over the 2017/18 school year and involve 100 primary schools, spilt into treatment and control groups. As with other EEF RCTs, it will be independently evaluated. You can read more about it here and follow our adventures on Twitter @MITAproject.
Teaching assistants remain the single biggest spend of Pupil Premium funding. The essence of our guidance for schools on making best use of TAs is to deploy them in ways that supplement, not replace, teachers and high quality teaching. There now exists an impressive evidence base on the use of TAs to deliver curriculum interventions. The priority now is on developing a similar and secure foundation for what works in terms of using TAs in everyday classroom settings.
Our MITA project will formally evaluate the effects of changing practices at the school level in an important area of educational improvement. Our aim is to find good evidence that when schools invest time up-skilling their TAs and carefully rethinking their deployment, they can thrive in their role and ensure they make a positive contribution to learning in classrooms.
* Blatchford, P., Russell, A. and Webster, R. (2012) Reassessing the impact of teaching assistants: How research challenges practice and policy, Oxon: Routledge; and Farrell, P., Alborz, A., Howes, A. and Pearson, D. (2010) The impact of teaching assistants on improving pupils’ academic achievement in mainstream schools: A review of the literature, Educational Review, 62(4): 435–448.