Saturday, 19 January 2013

How we mark

It’s that time of year again. Students are getting assignments back and there is the normal mixed reaction of joy and despair. And the perennial question persists: how can you be objective about creative writing? This often materialises as students query marks when someone other than the person who has taught them assesses their work.
But let us be clear. We, and I suspect all other teachers of creative writing, mark against a clearly defined set of criteria. We also spend a lot of time and care on marking assessments, we mark formatively as well as summatively even on summative assessments and we apply a rigorous form of moderation.

The criteria and what they mean

Task Has the student fulfilled the task? Are all the bits and pieces required there? To get higher marks, has the student entered into the spirit of it as well as obeyed to the letter? This can also almost be a comment on effort.   
Expression Can we understand the text? Has the student clearly told us something? Is a story and / or premise clear?
Technique How good is the craft of the student, particularly in relation to the craft elements taught within the module? For example, in a fiction module, is there a sound structure within the piece? Is there a plot as well as a story? What about the other creative writing skills met earlier? Is the student able to show and not tell? Do they handle dialogue effectively? Questions for a poetry module may look completely different, of course.  
Writerly research Often we’ll look at an annotated bibliography here. What has the writer read? What have they gained, as a writer, from reading that? Or have they actually done some of the writerly research that all writers often do – looked at some primary resources, recreated an experience they are writing about, trailed their own memories or used their imagination to create a new world?
Style Does the piece have an appropriate voice? Is it consistent? Is it free from anything that jars it (poor grammar and punctuation can do this)?
Reflection  This often appears in the Self-Assessment and drafts that accompany our students’ work. We expect to see evidence of thought applied to their work.  
Language This is where we look for work free of mistakes: spelling, grammar, sentence structure and syntax. One or two – literally – typos may be forgivable but otherwise I absolutely refuse to award a first if the student does not know how to use an apostrophe – no matter how good the rest of the work is. Not for a degree that has “English” in the title!   
Presentation Is the text correctly formatted? Is dialogue set out correctly? Is referencing correct? Higher marks can be obtained for including a cover page, presenting work in a helpful way and where a creative piece allows it, using smart presentation to enhance aesthetics.        
We award comments that correspond to a range of marks to each of these criteria. The overall mark is to do with which category they fall into most often and then influenced up or down by the other comments.    
Most of our students present work that is fair, good or very good. We have a few excellents and outstandings, and a few adequate and unsatisfactrories. So for the bulk of students marks are between 50 and 79, with a few being above 80 or between 40 and 49.  It’s very rare indeed for students to be outstanding or unsatisfactory in enough categories for them to obtain 90+ or below 40. We do get fails (below 40) but often than not this is because the task is incomplete, not because what is presented is of a failing nature.        
The moderation process
A colleague not teaching on the module moderates the work. A proscribed sample of texts is taken. The moderators select these themselves, though a marker may ask them to look at any they found problematic. For example, if there are up to 100 students on a module we must select ten scripts, through a range of marks and markers. Often this means we are selecting ten scripts out of no more than thirty.
We have to comment on any we disagree with and often comment even when we agree. If the marks only vary a little, no action is required. However, we may find that generally a particular set of marks are:
  • Too high
  • Too low
  • Too spread out
  • Too close together
These four scenarios are relatively easy to adjust. We can alter the marks on a well-defined scale. We’ve only had to do this once, actually. And that was to put marks up.
Worst case scenario, a marker has actually been completely inconsistent. Then we need a total remark and at this point it might be useful to bring in a second moderator. Fortunately we have never had to do that.  
Then all goes to the external examiner who looks at similar sample.  

Electronic marking
We are now in our third year of doing this. I actually like it and I think I make more comments than I used to. There are some we can drag and drop – I routinely use “run-on” “formatting” and “overwritten”. The student can click on the comment and get more explanation. We can also add in extra comments either directly on the text or in a box which appears as a bubble on the assignment. The student clicks on the bubble to get the full comment. We also award them a category based for each of the criteria mentioned above and add a general comment.
This gets rid if piles of paper, means that the work is available as soon as students have submitted, the student can actually read the comment  - they are no longer in a lecturer’s handwriting – and students can pick up marks privately. They are still invited to discuss them with teachers.
How I form my comments
First I write about what the student has done well in terms of the criteria above and in terms of the market, mentioning also anything of interest that has not been covered in the course.
Secondly, I talk about what has been done less well within the same areas.