fbpx
Saturday, March 6, 2021

Give automated marking a chance

Educators should have an open mind about computers assessing creative work

Reading Time: 4 minutes
Most educators use smartphones throughout their day. They communicate using email, utilise social media is various forms and drive cars that carry embedded computers. They also utilise spelling checks and grammar tools offered by Microsoft Word and other programs. It is clear to most educators that computer technology is developing, and adapting to it is necessary.
For those educators expressing discontent over computer automated marking, it might be wise to consider its benefits prior to disparaging the concept or making claims based on fear and anecdote.
Whilst it is true that adaptive technologies cannot yet wholly or accurately mark all forms of creative writing, educators should be open to working with technology in the interests of advancing educational outcomes, productivity and efficiency.
The context
Computers have been marking aspects of student assessment for well over a decade. Automated marking has become increasingly sophisticated. It is likely that every adult in Australia will have experienced exams, competitive tests such as the Australian Mathematics Competition, externally set school exams and even done their driving test, where computers were integrated into the process of marking. No one argues that the computer marking of so-called ‘objective response’ multiple choice questions is a problem, despite issues arising annually in every Year 12 exam featuring multiple choice questions where some of the questions may be open to interpretation and ambiguity. Nevertheless, educators have not stopped relying on computer marking as a result, even when there is confusion or disagreement with how questions are framed, and even the proposed ‘acceptable’ responses.
In terms of creatively written tasks, the notion of computers marking creative writing seems to have generated significant resistance from some educators. However, this resistance needs to be assessed in the light of what is currently done in external assessment. For written tasks on external exams (such as the Year 12 HSC in NSW) teachers will be briefed to mark to a standard from a pre-determined marking guideline and will then be allowed to mark from home at night after school, and on weekends. It is possible that a marker may be sipping on a glass of red wine while The Bachelorette or other television show may be on in the background, while reading and marking scanned scripts online.

Educators should be open to working with technology in the interests of advancing educational outcomes, productivity and efficiency

There is a requirement that teachers mark in the order of ten essays per hour, marking out of 20 or 25. Most items are marked once. If items are double-marked, which is occurring less frequently, then a margin of disparity between markers is allowed. The margin can vary as much as 15% above or below the first-given mark, meaning a variation of up to 30% can occur if three people mark a response. When it comes to creative tasks, it can be very difficult for subjectivity and discretion not to affect how a marker allocates marks, even on a predetermined criteria set.
 
Can algorithms mark creative written work?
Algorithms will be programmed to ‘recognise’ particular forms of writing, choice, use and order of words. There is academic evidence supporting the accuracy and efficacy of computer-based marking for written work.
 
Managing the transition
It is probable that algorithms will read ‘preferred’ forms of writing and, in this context, it is incumbent on authorities to disclose what forms of writing are acceptable. This should be publicly available to ensure equity and access.
Lexical, syntactic and semantic features are three important aspects of creative writing – but not all of them. A consideration for assessors and educators should be that the actual work has to be typed in order to be marked by computer and so a question arises as to whether the same creativity can be brought to typed tasks as hand-written tasks. Consider, for example, a creative piece that has been written in non-linear or non-rational (that is, creative) order as an aspect of the creative element.
Moreover, what if as an aspect of creativity, a child utilises SMS text and abbreviated speech that scores poorly on account of syntax but is actually valuable in the context of the piece?
The issue here is the ‘outliers.’ However, outliers at present still get penalised through the inaccuracies of the human-marking process. Having been a marker of externally state-run tests, I can state from experience that even amongst humans, there are significant disparities in marking, even when marking against a set of ‘objective standards.’ Therefore, a robust hybrid approach should be favoured with extensive human-marked sampling for validation. Included in this should be that all students not achieving requisite standards should have their work marked by teachers as well.
Educators should be interested to understand how adaptive technologies can ‘read’ creativity and the elements of persuasion. It is very common for people to be sceptical about the uncertainty that change can bring.  However, for the metrics that can be derived from clever design and application, the outcomes for students can be extremely beneficial, more targeted support and improved evidence for rich teaching helping educators to use their resources and energy wisely.

- Advertisement -
Avatar
Mohan Dhall
Academic leader, M2K Education and Advisory and CEO of Australian Tutoring Association and Global Tutoring Association.

Related Articles

LEAVE A REPLY

Please enter your comment!
Please enter your name here

Podcasts

Ep 9: What do young Indians want from love?

0
Growing up in Indian culture, most of us know that love has never been as popular as marriage. Even in the movies, the main...

Ep 8: Indian links in Indigenous Australian poet Ali Cobby Eckermann’s...

0
To celebrate NAIDOC week 2020 (between 8-15 November) I spoke to Yakunytjatjara poet Ali Cobby Eckermann about her time in India where she taught...

Ep 7: In the case of Sushant Singh Rajput

0
  The torrid and high-octane Sushant Singh Rajput case has been fodder for Indian people and press for the last few months. The actor’s tragic...

Latest News

Guess The Song rj ekta

LISTEN: Will you be the one to correctly guess this tune?

0
  Are you good with guessing tunes? Keen ear for rhythm and beats? RJ Ekta might've been able to stump you with this one. She recently...
Dr Michelle Ananda-Rajah twitter thread

Women, what would you tell your school age self?

0
  Ahead of International Women's Day on 8 March, Monash University's Dr Michelle Ananda-Rajah was asked to speak at a girl's school. The inspiring academic,...
women empowerment

WATCH: Hindi poem about women empowerment

0
  Ever have those days where nothing is going your way? You feel demotivated and wonder, "what's the point?". The last couple of weeks in Australia...
march 2021 shows and movies

Indian shows and movies to watch in March 2021

0
  Bombay Begums (Netflix) After the award-winning Lipstick Under My Burkha (2016), director Alankrita Shrivastava is back with another empowering series about women. Set in urban...

Japanese billionaire seeks eight artists for free Moon ride

0
  Japanese billionaire Yusaku Maezawa has invited eight people to join him for a free ride to the Moon on a SpaceX Starship rocket sometime...