Going digital: could online exams level the playing field?

After navigating the early months of COIVD-19 and subsequent lockdowns, schools soon faced one of their biggest challenges to date: how to grade students in a pandemic. Such assessment needed to account for lost learning, the challenges of virtual lessons, the digital divide, and a lack of in-person interaction. The ensuing exam grading system in summer 2020 proved insufficient and controversial, with algorithms failing to do what was hoped.

School leaders believed that this might lead to serious consideration of the assessment system and signal a chance for change. Perhaps more holistic models of assessment or wider qualification reform were on the horizon. A system that might better gauge young people’s skills and knowledge, and best serve the most vulnerable pupils, could be a positive thing to come out of an unimaginably challenging time for schools.

One reform on the table is a shift to digital assessment which the AQA are trialling with a small group of schools for GCSEs.

The jury is out on whether this leads to a fairer process. Whilst many young people might be more comfortable with online assessment and it might well better mirror and prepare them for the world of work, there is a risk that wealthier pupils win again. Those who have access to their own digital devices and wifi network to practise at home are surely likely to be able to master what it takes all the more easily than those who rely on overpriced mobile broadband add-ons, when their families struggle to meet weekly food bills. And as for schools, those with well-resourced ICT suites are going to find the shift much easier and get their pupils into the habit with relative ease compared to those who struggle to provide IT equipment at scale. Our lockdowns have demonstrated the gaping digital divide, exacerbated by the reliance on online learning and associated disengagement and isolation, and a shift to onscreen assessments must be carefully thought through.

Ultimately with onscreen assessment being one of a very small number of proposed reforms to qualifications, then this announcement highlights the importance of working with school leaders in the future to better understand models of assessment.

We reached out to colleagues across the organisation for their expertise on how digital assessment will affect their areas of work.

 

Curriculum Development: Soumya Suresh, Curriculum Design Manager

The introduction of digital assessments isn’t something that would surprise many of us. It’s been expected for a while now and the pandemic has only proven to accelerate this. There are a few key considerations to keep in mind as we consider this shift:

• Is it perpetuating the existing inequalities in education (for example: unequal access to technology, differing accessibility requirements)?
• Is it true to what research tells us constitutes good assessment: clarity of purpose, validity, reliability and mitigation of bias?
• Given these are being considered for summative assessments: do they cover a large domain of knowledge, are standard tasks being completed in standard conditions, are we using a scale score?
• Will it enable us to maintain the same standards and level of rigour as paper-based tests?

There are obviously a lot of questions to be worked out as we consider using digital assessments on a large scale. However, they seem to hold potential and it would be interesting to see the results of this pilot.

 

Careers: Mike Britland, Development Lead

The changes in modern recruitment practices for medium and large organisations across the public and private sector has shown an increase in the use of assessment centres, proficiency tests and virtual interviews. As such, students need to be provided with experience and practice in how to function and then thrive in such environments and scenarios. The move towards digital assessment could potentially provide this much needed experience to do so.

We know that the digital divide in schools in disadvantaged communities is a real issue for young people. Improved IT infrastructure, which should come about if the pilot is successful, can only have a net positive on the levelling up that needs to be done for disadvantaged young people to have fair access to job or apprenticeship opportunities that are only advertised online.

 

English: Emma King, Head of English

AQA’s piloting of on-screen assessment in GCSE English offers us the opportunity to capitalise on lessons learned during the pandemic. In particular, the adaptive possibilities that online assessment offers our neurodiverse pupils, pupils with SEND and pupils experiencing mental health challenges all means we look forward to hearing the results of this pilot.

Digital assessment may sound new to the world of English teaching, but most English teachers will have administered online reading age tests to their pupils. We have therefore seen that adaptive digital assessment works for tests like single word replacement activities or multiple choice questions that assess comprehension – and how these can be powerful tools.

However, significant questions remain for what this means for the subject of English. Regardless of how sophisticated an online system is, how will it measure the nuanced and creative outcomes pupils create in extended responses? What about the pupil who experiments with breaking the rules of English to create something brilliant? Are we at risk of creating assessments that test what is easy to measure, rather than measuring what English is really about? The disciplinary nature of English is discursive, subjective, creative, analytical and dialogic – it’s much more than spotting where a comma goes or knowing synonyms.

 

SEND: Sarah Shreeve, Head of Early Years and Primary

The move toward digital assessment is an interesting and important one when we consider learners with special educational needs and disabilities (SEND). This is of particular relevance to our vision and mission at Teach First, as we know that communities experiencing poverty also often experience higher levels of special educational needs and disabilities (Shaw et al, 2016).

Done well, a move toward digital assessment could be an empowering change with a positive impact on learners with SEND. The most common assessment arrangements (AAs) for GCSEs and A Levels in the academic year 2020-21 were: extra time; use of a computer or reader; and scribe/speech recognition (Ofqual 2021). It is the use of computers/readers and scribes/speech recognition where the impact of digital assessment stands to be the largest.

Research into the use of specially designed digital assessments to support SEND learners in Scotland (Nisbet and Aitken 2007) replaced human readers or scribes with assistive technologies (e.g., computer readers, voice to speech software). This was favourably received by both schools and learners. Request for digital papers soon eclipsed requests for scribes/readers, with one teacher stating:

"I personally would like to see digital exams as the default choice for pupils with physical disabilities, and paper, helpers, scribes etc as options that would need to be specially requested, because I believe they [digital exams] are empowering, less difficult to administer and cost-effective," with a pupil noting: ‘It made me feel more confident." (Nisbet 2021).

Whilst approaches that empower learners with SEND should be celebrated, there are a number of issues we must be conscious of in order for digital assessment to remain an asset, rather than a barrier to, learning:

1. Digital divide. Access to technology is lower in communities experiencing poverty: ‘only 51% of households earning between £6,000-10,000 had home internet access compared with 99% of households with an income of over £40,001’ (Holmes and Burgess 2020). Combined with the comorbidity of poverty and SEND, the learners who most need access to technology are statistically some of the least likely to have this access (Shaw et al., 2016).
2. Familiarity with technology. Pupils need to be computer literate and have practiced with the technology before the exam. Neurodiverse pupils, in particular, may benefit from longer to get used to new ways of working or new technologies (Ofqual, 2019). Pupils are also recorded as having better outcomes with the technology when teachers are confident in identifying what technology best suits their needs, and supporting them to use it effectively. Visual impairment services have seen year-on-year budget cuts, reducing staffing and expertise in these technologies (RNIB, 2019).
3. Suitability of the digital assessment. Ofqual studies show that a lack of consistency in formatting with digital assessments can cause technological errors, reduced validity in exam scores and a more difficult, stressful exam experience for learners with SEND. For example, different screen readers will interact with PDFs in different ways, sometimes causing sub questions to be skipped altogether. A typographic error when read aloud by a screen reader can entirely change the meaning of a question. Digital papers without interactive ‘answer boxes’ can see pupils needing to use two devices simultaneously- one to display the paper and the other for word processing. Awarding organisation’s protocols for checking adapted papers are not in the public domain (Ofqual, 2019).

With this in mind, we celebrate Nisbet’s finding that digital assessments can empower learners with SEND, whilst calling on policy makers to continue to bridge the digital divide. To make the most of this trial, we must also better understand how teachers can be supported in using assistive technologies, and how to ensure the format of any digital assessment is accessible for all learners.

 

As our country begins to rebuild after COVID-19, it's time to break the cycle of inequality. Part of this means ensuring schools and pupils have the resources they need, and are supported by policies that give them a fighting chance to reach their potential. Our manifesto highlights our 11 commitments to eliminating inequality in education, including fairer digital practices in our school system.