Attend an evening with Andre Agassi
They are some of the most memorable and stirring words of the 20th century, but Churchill’s speech exhorting the British to “fight on the beaches” would fail if submitted as a school essay and subjected to a proposed computerised marking system.
The wartime leader had a style that was too repetitive, according to the computer being tested for the online marking of school qualifications. It rated Churchill as below average in the equivalent of an A level English exam.
His reference to the “might of the German army” lost him marks because the computer interpreted this as an incorrect way of writing “might have” rather than recognising “might” as an abstract noun.
Other authors, including Ernest Hemingway and William Golding, were also dismissed by the computer as not being up to standard in the American equivalent of an A-level English exam.
David Wright, chief executive of the Chartered Institute of Educational Assessors (CIEA), an umbrella body for exam boards and other organisations, said that Churchill’s speeches to the nation in 1940 had not impressed the computer. It criticised his repetition of the words “upon” and “our” and did not identify “broad, sunlit uplands” as a metaphor.
Graham Herbert, deputy head of the institute, said: “The computer was limited in its scope. It couldn’t cope with metaphor and didn’t understand the purpose of the speech. We also tried a passage from Hemingway. It couldn’t understand the fact that he had a very spartan style and [it] said he should write with more care and detail. He was also rated less than average.”
The institute tested an extract from William Golding’s Lord of the Flies, using a tense part of the story that contains a paragraph of just two words: “A face.” Mr Herbert said: “We, as readers, understand the horror in that but the computer marked it as wrong because it was erratic. It also described the opening of A Clockwork Orange as bizarre.”
The potential shortcomings of such a system were pinpointed at the Westminster Education Forum yesterday. Online marking of school qualifications is being tested by British exam boards and could be introduced in the next few years.
Computers are already used to mark some multiple choice GCSE exam papers and trials are taking place with technology that could assess essay-style answers. This system is already in use in America.
The computer program has been created using a range of comments given by human markers in response to exam papers. While the program can recognise sentence structure, it is not able to understand style or purpose.
Mr Herbert said that some children in American had “cracked the code” by learning to write in a style that the computer recognised. This was called “schmoozing the computer”, he said. “At the moment we do not have a reliable and valid way of assessing English language using a software package, although this is something for which there is demand.”
One education company is preparing to start trials of marking GCSE English papers by computer, although it will not use the scripts of British children. Experts expect increasing numbers of schools to adopt online testing and marking.
Sue Kirkham, a retired head teacher, told the forum that it was unlikely that today’s primary school children would, at the age of 18, sit their A levels by handwriting on exam papers. She said a recent survey showed that schools spend more on exam-related costs than on books and equipment. Another speaker at the forum said that £750 million was spent each year on testing children in schools.
Greg Watson, chief executive of OCR, of one of the biggest exam boards in Britain, suggested after the forum that A levels had become devalued. He said: “A levels, which started pretty much as a qualification to gain university entry, have become widened to take in a broader group of students. That has compromised the ability of A levels to be the ideal entry point for university courses.”
Written off...
“In the old days Hortons Bay was a lumbering town. No one who lived in it was out of sound of the big saws in the mill by the lake. Then one year there were no more logs to make lumber. The lumber schooners came into the bay and were loaded with the cut of the mill that stood stacked in the yard. All the piles of lumber were carried away. The big mill building had all its machinery that was removable taken out and hoisted on board one of the schooners by the men who had worked in the mill.”
— Computer says: Shows lack of care in style of writing and vocabulary
“Ralph drew his feet up and crouched. The stake was in his hands, the stake sharpened at both ends, the stake that vibrated so wildly, that grew long, short, light, heavy, light again. The ululation spread from shore to shore. The savage knelt down by the edge of the thicket, and there were lights flickering in the forest behind him. You could see a knee disturb the mold. Now the other. Two hands. A spear. A face."
— Computer says: Inaccurate and erratic sentence structure
“What’s it going to be then, eh?. There was me, that is Alex, and my three droogs, that is Pete, Georgie, and Dim, Dim being really dim, and we sat in the Korova Milkbar making up our rassoodocks what to do with the evening, a flip dark chill winter bastard though dry. The Korova Milkbar was a milk-plus mesto, and you may, O my brothers, have forgotten what these mestos were like, things changing so skorry these days and everybody very quick to forget, newspapers not being read much neither."
— Computer says: Incomprehensible
Apply to become a journalist at one of the world's top news organisations
Industry sectors news at a glance. Interactive heatmap, video and podcast
Everything the Business Traveller needs to know to make a better trip
Get ready for the winter sports season, with our resort guides and snow reports
We are backing British business, what is the confidence of the nation and what businesses are succeeding?
Growing demand for energy, oil that is harder to reach and the rise of carbon dioxide emissions. We examine the energy challenge
With rail travel in Europe on the rise, we review the benefits of travelling by train
In this special section we explore new food trends to help improve your dinner party and impress guests
Enjoy further reading from Travel to Fashion, Business to Sport, discover more
Shortcuts to help you find sections and articles
1998
£47,955
12 months for the price of 11 and a 5% discount.
Offer ends 31/11/09
Check your free Experian credit report before applying
Car Insurance
to £60K + bonus (OTE £90k)
Lord Search & Selection
Location Flexible
PwC’s Consulting practice helps businesses of all shapes
and sizes work smarter and grow faster.
£85k
CPA
Highly Competitve
Specsavers
Whiteley, near Southampton
Moments from Battersea Park.
For sale with Winkworth
Find out about shared ownership.
See your free Experian credit report beforehand
7nts - Penang £499; Borneo £699; All Inclusive £799 including flights, taxes, accommodation and private transfers
For your ultimate tailor-made ski holiday, click here
Get covered on your travels with a superb range of policies at great prices. Visit InsureandGo.com
World Class Golf, Spa and preferential Beach Club. Private estate overlooking West Coast
Villas from £275 per night inclusive of Golf
Contact our advertising team for advertising and sponsorship in Times Online, The Times and The Sunday Times, or place your advertisement.
Times Online Services: Dating | Jobs | Property Search | Used Cars | Holidays | Births, Marriages, Deaths | Subscriptions | E-paper
News International associated websites: Globrix Property Search | Milkround
Copyright 2009 Times Newspapers Ltd.
This service is provided on Times Newspapers' standard Terms and Conditions. Please read our Privacy Policy.To inquire about a licence to reproduce material from Times Online, The Times or The Sunday Times, click here.This website is published by a member of the News International Group. News International Limited, 1 Virginia St, London E98 1XY, is the holding company for the News International group and is registered in England No 81701. VAT number GB 243 8054 69.
Your Comments
Order By: