Many college teachers assign either opinion essays or argument essays to their English Second Language students. These highly structured writing assignments give students practice organizing their ideas and defending a point of view. The goal is to prepare students to go on and write research reports or graduate theses at university. However, when it comes time to score these opinion essays or argument essays, the correction load for the teacher can be enormous. Clearly, teachers need an automated essay evaluation system.
This article discusses automatically scoring argument essays, but there are other automated evaluations on the Virtual Writing Tutor if you are interested:
The impact of one hundred and fifty 500-word argument essays on a teacher’s time
If a college student submits a 500-word essay with structural problems and grammatical errors, the teacher will likely end up spending up to 20 minutes trying to provide feedback on how to improve it. With 150 students submitting 150 essays at once, one essay assignment will generate up to 50 hours of correction work for the teacher. The teacher will then spend the next two weeks doing little else.
Since many students depend on the teacher for guidance on how to improve the essay and maximize their score, the teacher often feels compelled to provide detailed and thorough feedback. Without collaboration from the teacher, ESL teachers believe the student might not be able to make the revisions needed to meet the high standards required at university. In this way, we teachers place a lot of pressure on ourselves to help our students improve their writing.
What is the impact of correction time on practice opportunities?
You should know that teachers limit the number of writing tasks they give their students because of the correction load each essay assignment generates. There is a physical limit to how long we can stay hunched over a desk correcting student essays. If we assign too many, we will burn out, not get the corrections done, or become so overloaded that we end up giving students low quality feedback and unreliable scores.
What if we could give students more essay writing practice without increasing our correction load? What if we could give consistently reliable, objective, and detailed feedback? And what if we had an automatic essay evaluation system?
An automatic essay evaluation system
For the past 7 years, I have been hard at work developing Natural Language Processing tools for the Virtual Writing Tutor to give feedback on English Second Language student writing. I have devoted most of my time to coding error detection rules for the Virtual Writing Tutor’s grammar checker. However, along the way, I have developed a number of feedback tools: a paraphrase checker, a word counter, a target structure checker, a vocabulary checker, a cohesion checker, and a sentence checker. While interesting in their own right, I realize now that they are of limited usefulness to students.
Indeed, students don’t always know how to interpret the feedback from a grammar checker website such as mine. Unless the teacher specifically trains the learner to use the feedback to eliminate errors, improve cohesion, extend their vocabulary, or improve their paraphrasing skills, students aren’t always sure what steps to take to improve their scores.
Recently, I have discovered that feedback on form is a lot more compelling to students when it comes with a score.
When the NLP tools listed above are all integrated into a self-scoring pen pal exchange project, IELTS academic writing tests, a film analysis essay checker or argument essay evaluation system, students know just what to do. They use the score and feedback to maximize their next score. They want to level up as they would in a game.
An argument essay writing module
I have been collaborating with with Dr. Frank Bonkowski at Cegep St-Laurent to create a new automatic scoring system. This one is for the argument essay writing task he has given this term. His students chose 7 topics that interested them, and we designed a system to identify their topic and give feedback on all aspects of their structure, vocabulary, cohesion, strength of their claims, and grammar.
This argument essay writing task and evaluation is free for anyone to use. Please share.
Before you try it with students, you should know that the argument essay task is rather complex. So, here is a step-by-step lesson that will help you teach students how to write a successful argument essay from scratch.
But first, what is an argument essay? An argument essay is an opinion essay that includes an opposing view, a concession, and a refutation. This discourse model requires some research, reflection, and a considerable amount of time to organize ideas into a coherent structure.
Don’t worry. If your students have access to a computer, they will quickly get the hang of it using the Virtual Writing Tutor. You’ll be amazed.
Step 1: Score a model argument essay
To begin, start by trying out the argument essay evaluation test with each of the two model essays below. Try the one that needs improvement first with your students to see if they can improve the score. Then, have them try it with the second essay.
Step 2: Outline an argument essay with the essay outliner
Ask students to use the model essays with the argument essay outliner to develop an outline. Once they are able to outline an existing essay with the easy essay outliner, they are ready to write their own outline. This will prepare them for the actual essay writing exam.
Step 3: Choose one of 7 Debatable Topics
Ask students to choose one of the 7 different topics below to research. They will need to find at least three articles. They should search for at least two articles in support of their point of view and one opposing view.
- abortion
- climate change
- body image
- animal rights
- gender equality
- internet censorship
Debate websites like iDebate with its searchable Debatabase are excellent resources for finding arguments for and against a point of view.
Step 4: Outline an essay on one of the topics
The students should now outline their own argument essay using the information they found during their research. Their experience with the previous steps will make it easy for them.
Step 5: Write an argument essay
Once students have written an outline, they will be ready to write their own essay. They should make sure that they follow the prescribed structure described below with a title and a “Cited Works” section.
Argument essay writing prompt
Instructions: Write a 450-750 word argumentative essay on one of these topics: abortion, climate change, animal rights, body image, feminism (a.k.a. gender equality), immigration, or internet censorship. Use the essay structure described below.
Paragraph 1: introduction
- Opening: Begin the essay with a provocative, thought-provoking question. You could begin, for example, with “Did you know that…?”
- Context: Establish the importance of the topic. Use phrases like these: a vital factor in, the leading cause of, widely considered to be, set to become, undergoing a revolution, is responsible for.
- Thesis: Include a debatable thesis about one of these areas of controversy: abortion, climate change, animal rights, body image, feminism, immigration, internet censorship.
Paragraph 2: first supporting argument
- Topic sentence: Begin the paragraph with a topic sentence that summarizes the main idea of the paragraph and includes the first of the two topics in your thesis sentence.
- Argue: Refer to facts, statistics, logic, what is reasonable, what follows from the evidence, and conclusions one can draw.
- Evidence: Give evidence from your reading and research using these types of words: according to, to quote from, tells us that, shows us that, referring to, argues that, stated, wrote, argued, discussed, expressed the concern that, as written. Use these quotation marks: “…”
- Cite sources: Properly cite quoted text using the person’s family name in parentheses (Einstein).
- Support: Give examples to support your claims using words and phrases such as these: for example, for instance, suppose that, take the case of, that is, to be exact, to explain, to illustrate, to put another way, to show what I mean.
Paragraph 3: second supporting argument
- Topic sentence: Begin the paragraph with a topic sentence that summarizes the main idea of the paragraph and includes the second of the two topics in your thesis sentence.
- Argue: Refer to facts, statistics, logic, what is reasonable, what follows from the evidence, and conclusions you can draw.
- Evidence: Give evidence from your reading and research using these types of words: according to, to quote from, tells us that, shows us that, referring to, argues that, stated, wrote, argued, discussed, expressed the concern that, as written. Use these quotation marks: “…”
- Cite sources: Properly cite quoted text using the person’s family name in parentheses (Einstein).
- Support: Give examples to support your claims using words and phrases such as these: for example, for instance, suppose that, take the case of, that is, to be exact, to explain, to illustrate, to put another way, to show what I mean.
Paragraph 4: counterargument
- Topic sentence: Summarize an opposing view with words like these: some people claim, some believe, others believe, some people object to, try to refute, discount, reject, it is often argued that, the opposing side will claim
- Concede: Show which parts of the counterargument have merit with words like these: concede that, granted, indeed, it is true that, while it is true that, naturally, to be sure, admittedly, one cannot deny that
- Refute: Refute the counterargument using words like these: nonetheless, nevertheless, regardless, whereas, although, and yet, in contrast, by contrast, despite, there is countervailing evidence, that said, however
Paragraph 5: conclusion
- Reformulation of the thesis: Restate the thesis in different words from the introduction.
- Build cohesion: Use expressions such as To sum up, To conclude, In closing, or For these reasons.
- Recommendation: Make a recommendation. Use one expression such as should, ought to, will, or going to.
- Prediction: Make a prediction. What will happen if the reader follows your recommendation? Use one expression such as should, ought to, will, or going to, or likely.
Works Cited section
- Heading: Identify this section of your essay with the heading, “Works Cited”
- List: Include at least 3 sources properly formatted, each on its own line.
Step 6: Get formative evaluation from the essay scoring system
Then, each student should use the automatically scored argument essay test to get formative feedback on their essay, make any changes suggested by the system, and turn in a final draft for the teacher to score.
They will need the teacher to coach and help them understand how to interpret the feedback. It can be a bit overwhelming at first. It is worth using a lab hour for this.
Step 7: Submit your final draft to the teacher for a score
In my opinion, the score generated by the Virtual Writing Tutor’s argument essay evaluation system should guide revisions, but the final score should come from a teacher. That means that the student really should submit the essay to his or her teacher.
In addition, it would be helpful if the the teacher were to receive the automatically generated feedback, also. The combination of automatic scoring and human scoring has been found to be more reliable than just computer scoring or human scoring, even with multiple human raters (Bridgeman, 2004). Ask students to print the feedback or generate a PDF to send electronically. It is easy to do.
Step 8: Repeat steps 5, 6, and 7 with a new topic
A teacher could build an entire writing course around this one page on the Virtual Writing Tutor. There are 7 controversial topics to choose from.
Simply ask students every two weeks to write an essay on a topic they haven’t tried before, get feedback and submit their final draft with the automated feedback for a score that counts.
To maintain student motivation, you could use the Mastery Model to score essays, telling students that only 4 of the 7 essay scores will count. The lowest scoring submissions will be considered formative. In this way, you can maximize essay writing practice for hardworking students who want to do it all, and even the most laid-back students students will get double the argument essay writing practice that they would get in a regular college ESL course.
Since the students will get a lot of feedback from the automated scoring system, you will be able to put down the red pen and use voice of the reader comments, reading your students’ writing in a way that wasn’t possible before.
Step 9: Report your findings
You didn’t expect all this to be your next research project or conference presentation, did you? But why not? Formative evaluation of writing assignments using an automated essay evaluation system is brand new. Your colleagues probably won’t know anything about it.
While it is true that the Educational Testing Service in the US has used an automatic scoring system for university admissions tests for years, nobody seems to have reported much on using automated scoring in the classroom for formative evaluation.
You can do it. I can help. I can create a test designed to score your particular essay writing task if this one is not quite what you had in mind. Just contact me. Not sure? Ask yourself, “If not now, then when? If not me, then who?”
Something that would really help me would be if you were to give feedback on the feedback the essay scoring system generates. If the system over-values a low-quality essay or undervalues a high-quality essay, let me know. There is a “Rate feedback” button you can use to let me know if the system is working as expected or not. Leave detailed comments. “This sucks” is not very helpful. Say what went wrong and what you would like instead. Complain to get a specific change.
Final remarks
There is clearly a pedagogical need for automated feedback. There are not enough teachers in the world to provide all the feedback students want and need. Automated scoring is the next step in computer-assisted learning.
(Notice my use of the word “assisted.”)
Teachers happily give practice tests to students with multiple choice questions, but essay writing tasks are rarely used in formative evaluation. Why? It’s a workload issue. Teachers cannot score essays fast enough. Obviously, if a machine can help with the task, we should get out of the way of meaningful practice and let the machine guide students through their early drafts.
Opponents of automatic essay evaluation will argue that automatic essay scoring is not ready yet. There are too many false alarms, not enough coverage, and the scores don’t match the scoring criteria and weightings that this or that department requires. While all that may be true, it doesn’t mean that we should cede the field.
It is better that teachers engage with this new technology to ensure automatic essay evaluation systems reflect what teachers care about. I, for one, believe that essay evaluation must be developed and reserved for formative evaluation only. We must ensure automatic evaluation is used for practice tests prior to the teacher assigning the final score, even if the teacher’s scoring decision is informed by automatic scoring metrics. Human teachers are needed to validate and humanize the educational system.
Some teachers will shudder at all this and say that automatic scoring will put teachers’ jobs at risk. Really? Quebec has a shortage of teachers at the moment. A colleague I admire very much is away on sick leave, burned out from overwork. The real victims of the lack of automation in our jobs are students who are denied meaningful practice activities with feedback and their teachers who are overworked trying to give students the meaningful practice and feedback they need.
Unreflective opposition to new technology is not how you build job security. Maintaining a pedagogical focus on what is best for students and developing a sound pedagogy using new tools will make expert teachers indispensable.
It’s one teacher’s opinion. Perhaps you agree with me.
Examples of automatically scored writing tests
Here are some examples of automatic essay scoring systems I have developed in the order I developed them. Please, give them a try and let me know what you think.
- Pen pal exchange project
- IELTS practice tests
- Film analysis essay evaluation
- Argument essay evaluation
Works cited
Bridgeman, B. (2004, December). E-rater as a quality control on human scorers. Presentation in the ETSResearch Colloquium Series, Princeton, NJ.
