Practical Work in Science Part 2: Beyond the blundering headlines

In this series of blogposts I discuss the state of practical work in secondary school science lessons. Last time I welcomed Ofqual’s decision to scrap ISAs (the current method of assessing practical skills at GCSE) and suggested some reasons why many Science teachers feel the same way. This is a rare moment of accord between a regulator and the teaching profession and one thoroughly spoilt by the press briefing ‘scientific’ members of the educational establishment. The doom-mongering ‘it is the end of practical work’ headlines are tiresome and unhelpfully endorsed by tedious education secretary Nicky Morgan. Yet I find myself equally impatient with partisan ‘what do they know' responses from some colleagues especially as some of the most prominent voices are members of the press and scientific community themselves. I think the sceptics know quite a bit and, contrary to popular opinion, they have listened to teachers.

The Gatsby Foundation and Wellcome (GFWT) Trust are amongst the most vocally opposed to Ofqual’s proposals yet a joint policy note from June 2013 (referenced directly by Ofqual in the consultation document) articulates no love for ISAs:

‘There is universal agreement among those we consulted that this assessment method is deeply flawed. It makes teachers focus on a narrow range of externally-set practicals as they hone students to do well in what constitutes 25% of their final grade. Students are internally assessed on their planning and analytical abilities (not on their technical skills) by their teachers who, under our high stakes system, are under enormous pressure to give students maximum marks.’

Despite this accurate assessment the GFWT were signatories to a letter, published in the press, sent to Head of Ofqual Glenys Stacey criticising the reforms:

‘We are deeply concerned by the proposal that practical skills will not be directly assessed, and the suggestion that such skills can be validly measured through questions in the written exam.’

To me this excerpt reveals more about the GFWT’s motivations than the headlines. Their inclination towards direct assessment of student’s practical skills, not ISAs, seems clear if a little short sighted given their genuine concern about validity and Ofqual’s reasons (see here) for rejecting direct assessment:

‘Sheer numbers are a particular consideration…teachers would need to observe directly and carefully a sufficient amount of each student’s practical work to judge whether or not that student had performed the wide range of skills to the required standard. If students’ skills were to be assessed in more than a few practical tasks, the assessments would become unmanageable, yet with too few tasks they would not be particularly valid.’

Briefing against Ofqual on this demonstrates a lack of understanding of departmental level systems on the part of GFWT. That said their letter does, once again, demonstrate a remarkable appreciation of the unintended consequences of our target driven accountability measures:

‘Without direct assessment, practical science may be devalued by head teachers and senior leaders who are under pressure from school accountability measures and tight budgets.’

Yet GFWT's appreciation only goes so far and I think it is contradictory, given their earlier positioning, to assume that Ofqual regulating in favour of direct assessment will get disinterested people to reverse their attitudes towards practical skills; if accountability measures encourage people to take short cuts then more regulation will likely lead to more of the same behaviour.

Despite the clumsy headlines the dividing lines between the educational establishment, Ofqual and the teaching profession are limited to direct assessment and solving the problems associated with accountability measures. The educational establishment need to let direct assessment go and realise that anything consuming money and time is an enemy of pertinent practical activities. There also needs to be an acceptance that, while current accountability measures remain, different regulation is unlikely to lead to a massive shift in values. With this in mind Ofqual’s proposed minimum of eight practicals per GCSE qualification with associated examination questions is a good way to nudge the reluctant while minimising regulatory burdens on teachers who genuinely want to offer a higher frequency of pertinent practical work.

 


Practical Work in Science Part 1: Three Reasons Why ISAs Must Die

Merry Christmas everyone! Christmas this year is extra special because, for Science teachers everywhere, it came two weeks early courtesy of Ofqual's move to consult on the way that practical laboratory work is assessed in GCSE Science – the entire report can be found here. This first in a series of blogposts focuses on controlled assessment and the impact it has on teachers and practical work. Ofqual have suggested that 'Individual Skills Assessments' (ISAs) are going to be replaced at GCSE level and it seems pertinent to discuss it given the hysterical press reaction to similar developments surrounding A level Science.

What is an ISA?

ISAs are how a student's scientific skills are assessed at GCSE, they are a form controlled assessment that constitutes 25% of a student's final GCSE grade. ISAs require students to carry out a practical investigation (one per award) from a small number set by the Awarding Organisation under controlled conditions supervised by teachers (the Gatsby Foundation and Wellcome Trust or GFWT, 2013).

Three reasons why ISAs must die

Reason 1: A bureaucratic nightmare

If Science is about observing the world systematically, by experiment, and thinking about those observations in an intellectual way then it makes absolute sense to try and expose students to laboratory work. Some critics of Ofqual's current thinking such as The Right Honorable Andrew Miller MP argue that removing controlled assessment 'could lead to schools neglecting experimental skills' (Sample, 2014). This sentiment demonstrates a misunderstanding of how ISAs and controlled assessment in Science work.

It takes about eight hours to get a class through one ISA (AQA or Edexcel), just one hour of this is devoted to laboratory work. Students devote the rest of the time to completing bureaucratic pro-formas or question booklets designed to assess what the GFWT describe as 'planning and analytical abilities' – actual laboratory skills are not and have not been assessed for years due to teacher malpractice (more on this later).

Once the work has been completed it needs to be marked by teachers, there is a lot of pressure to justify marking by annotating the work in the belief that annotated scripts are more likely to pass moderation. This is time consuming; it takes approximately 15 hours to mark a class set of ISAs. In other words a relatively small proportion of time spent on an ISA is devoted to laboratory work and even that isn't graded while the absurd amount of time it takes to assess the parts of the ISA that do count reduces the time available for teachers to plan effective laboratory experiences elsewhere. ISAs ultimately foster neglect of the very skills they supposedly measure – the extremity of this neglect brings me to my next point.

Reason 2: 'Slideshow Bob'

Because of ISAs a GCSE science student is effectively required to do just one hands on laboratory experiment per qualification. The purpose of this experiment is to gather data to analyse as part of the ISA and little else – most state secondaries do not have the resources for students to work alone so it is entirely possible for a student to let other kids in their group do all the work and not touch any scientific instruments at all!

Of course the majority of teachers run experiments with their classes that cover parts of the curriculum not addressed by the ISA. That said because things are as they are 'Slideshow Bob' style science teachers can get away with running two or three experiments a year with their GCSE groups. With this in mind it is quite amusing to hear the advocates of controlled assessment in Science talk as if it's presence is safeguarding practical laboratory work. This idea that specifications and targets equate to quality educational provision neatly brings me to my final point.

Reason 3: Predictability, gaming and cheating

The GFWT suggest that teachers 'under our high stakes system, are under enormous pressure to give students maximum marks'. Ofqual have noted that there is a huge discrepancy between the grades achieved by students in externally marked examinations and controlled assessments (see Figure 1).

Figure 1: Percentage of candidates achieving particular percentages of marks available in GCSE Science components (Ofqual 2014)

GFWT are actually quite diplomatic in their assessment of this situation because over-marking is picked up during the moderation of ISAs by examining bodies (hence the aforementioned pressure to annotate students work during assessment). The gap is there because a the majority of teachers are competent enough to exploit the repetitive nature of ISAs and because some teachers cheat.

A competent teacher will soon spot that the same (Edexcel) or similar (AQA) questions/tasks appear in each ISA and they use published practice materials to make sure students approach questions in the right way. Cheating ranges from teachers discussing model answers specific to live ISAs with the students before they complete them (this isn't allowed) to ensuring that tailored practice sessions occur by looking at the high control sections of the ISA and associated marking guidance before the work is completed (AQA specifically warn against this). The very worst forms of cheating that I am aware of involve teachers letting students draft their ISA so it can be checked and altered before marking and teachers who write the marking guidance on the board while the students are completing their work (see Figure 2).

Figure 2. Controlled assessment in Science is discussed on Twitter.
Conclusions
Education is an industry managed by specifications, benchmarks and compliance targets. The stakes are high and in high stakes systems ambitious charlatans, false prophets and desperate victims of circumstance will exploit loopholes or cheat in order to get money, power or to simply stay in the game. Science ISAs offer a lot of loopholes and a lot of shortcuts so it is great news, and about time, that Ofqual has moved to close them.

References

Ofqual (2014) Consultation on the Assessment of Practical Work in Science. Available from: https://www.gov.uk/government/news/ofqual-consults-on-changes-to-gcse-science-practicals [Accessed on 27 December 2014]

Sample, I. (2014) Science A level practicals face axe despite barrage of criticism, says MP. Guardian, [online] (last updated 23 December 2014). Available from: http://www.theguardian.com/science/2014/dec/23/science-a-level-practicals-mp-andrew-miller [Accessed on 27 December 2014]

The Gatsby Foundation and the Wellcome Trust (2013) Policy note: Assessment of Practical work in Science April 2013. Available from: http://www.gatsby.org.uk/~/media/Files/Education/Practical%20Science%20Policy%20Note.ashx [Accessed on 21 December 2014]