Saturday, April 20, 2013

Changing the Message: Grading vs Reviewing (Part 4 of 4)

Continuing the saga....


Rigid Grading Structures

OK, I recognize that this is a touchy subject, so I'll tread lightly (for now). The heck with that, let's get dirty!

"You, my friend, are going towards turbulent flows."
An Anonymous Friend
Today, after telling him about what I'm writing 

We follow rigid grading rubrics hoping to be "fair" to everyone. The problem is that these grading structures put us "fairly" off base.

"Grades tend to diminish students' interest in whatever they're learning. A "grading orientation" and a "learning orientation" have been shown to be inversely proportional...
Grades create a preference for the easiest possible task. Impress upon students that what they're doing will count towards their grade, and their response will likely be to avoid taking any unnecessary intellectual risks...
Grades tend to reduce the quality of students' thinking. They may skim books for what they'll "need to know." They're less likely to wonder, say, "How can we be sure that's true?" than to ask "Is this going to be on the test?"
Alfie Kohn, 2011 

What do we value?

Rate of Learning?
Imagine the situation in which a student learns the material at a slower pace than others in the class. The student earns a poor score on the first exam. This feedback inspires the student to try a different approach to learning and (s)he grasps the material in time for the final exam. Following the rigid rubric, this student's grade will be lower than another who learned the material faster, even though both may be leaving the course with the same understanding.

What is the message here? What do we really value? The rate at which a student learns? Rate of production is a crucial concept to efficient manufacturing.

Get it "Right" the first time?
Students are penalized for making mistakes. The result is that students are afraid to try anything unorthodox, or explore.

"I just want to know what formula to use to solve the problem the right way."
There is a high level of pressure on students to get things "right" the first time. Our RED PEN feedback system helps drive this home. If a student misses a question, points are lost and final course grades are affected. GPA drops and that homework problem or test problem just influenced the rest of my life. (OK, that's a bit extreme, but its the logical end to the thought).

The Root of the Problem

We have a need in education to determine how a student is doing. Students need feedback so that they may improve upon problem areas. Educators need to see how their students are doing so that they may assess the effectiveness of their teaching.

There is a long list of other reasons typically thrown up in the case for grades, but I think they are rubbish. One of the big ones is that "if we don't grade it, students won't do it." Yes, let's hold our students hostage with threats against their future success rather than providing true motivation.

Had I the heavens' embroidered cloths,
Enwrought with golden and silver light,
The blue and the dim and the dark cloths
Of night and light and the half-light,
I would spread the cloths under your feet:
But I, being poor, have only my dreams;
I have spread my dreams under your feet,
Tread softly because you tread on my dreams
W. B. Yeats

I think we may have started, long ago, with these two goals in mind. However, our "factory model" education system has polluted those goals, mainly due to the fact that we are trying to educate so many people. Call the EPA (Educational Protection Agency)!

Scaling Issues

With a large class, the task of providing feedback is not trivial. Graders must streamline the process by providing the least amount of feedback possible. They look over the work, think about what the student did, and try to scribble something down to let them know where mistakes were made. If only the student could hear the grader's thoughts! (In light of what I'm about to say, I think they call this foreshadowing)

Very rarely is any positive feedback given. For the most part, graders write down just enough to justify the score on the assignment. I don't fault them for this, they simply don't have the time to give meaningful, content-rich responses to each student's work.

The problem is that, from the students' perspective, this type of feedback is more a slap in the face than constructive criticism. Rather than a teachable moment, we offer up judgement and discouragement.

I have not failed. I've just found 1000 ways that won't work.
Thomas Edison 

Let's change the message.

Learner-Centered Feedback
(Stop Grading and Start Reviewing)

With tablets and pen inputs for computers we have the ability to record ourselves as we mark up a document. Imagine marking up a student's electronic homework submission. The grader no longer needs to write a detailed response to help the student learn from issues. The grader can record his/her thoughts in audio format while marking up the work.

Rather than grade the work we can review the work and provide meaningful feedback (audio) to the learner. I think if we did this right, it would actually save time for grader. Most people can speak much faster than they can write or type.

We could add "content area" tags to individual homework problems. Over time, the student's aggregated results would point out sticking points in a particular course and focus the student's study/improvement efforts. If a visual representation of this were available to the grader small bits of encouragement could emerge: "I see that the First Law is starting to click for you, good job!"

I can see clearly now....

Synced with the interactive Degree Path Sheet, a student would have a much better view of where they are. With our current approach most students can't see what's going on. All they see are a few "red x's" that leave them feeling less intelligent, or that "I just don't understand Thermodynamics." Truth is that's usually not the case. Maybe the real issue is a small part of it that creeps up in most problems.

OK, enough words...

Let me show you an example of what I am picturing, from an engineers perspective (sorry, I haven't graded anything else!):



If you kept track, I spent about 2 minutes providing feedback on this problem. In a class of 60 students, with an average of 10 homework problems per assignment, this translates to 20 hours of grading work. Interestingly enough, that's the exact amount of time that my grader logs while providing RED PEN feedback right now.

A few things to work out:

First, I'm laughing at myself for using a RED PEN while making this video. I imagine a better system in which I can use "cursor points" or a "focus bubble" to show what I'm looking at.

Second, the "tags" idea isn't a worksheet, but I'm not Java programmer. The check-mark image was supposed to represent clicking on tags.

Third, I DO use a red pen while grading currently. This idea is new for me this week, and I hope to implement a refined version of this system next time I teach a course.

Fourth, I DO collect the homework for a grade. I'll explain...

Some other things you might have seen:

Did you notice that the numerical answers were provided with the problem statement? This is a trick I use to maximize the self-learning for my students. With the answers to homework problems available, students know whether they've got it or not long before they turn it in. Assuming they start the work early enough, they can find the help they need before turning in the assignment.

Did you notice that I showed two submissions from the same student? I actually do this as well. In my course I try very hard to encourage my students to make mistakes and learn from them. This learning business is messy and most of what sticks in our minds comes from getting things wrong at first.

I allow my students to re-submit the work as many times as it takes, and they can earn full credit on every problem even if they don't get it until the last day of the semester. Effort and engagement is encouraged and rewarded. Based on their feedback, this is working as I hoped it would!

Closing: The Message

Once again, the "Medium is the Message." I hope that by taking advantage of the digital media tools available to us, we can shift the message that we send learners. Let's send the message that experimenting is a good thing. Let's send the message that mistakes are a good thing. Let's send the message that our students deserve more than some messy red pen graffiti all over their work.

At the risk of starting a riot:
STUDENTS: Like what you see? Are you tired of getting RED PEN all over your work? Demand more from your teachers!

Try turning your homework in written completely in RED PEN.

4 comments:

  1. Interesting post. However, I disagree with the idea that grading is negative for the student learning process. My own experience as an instructor has shown me otherwise.

    First, BLUE or RED marks on the paper are meant to serve two purposes. One is to allow the student to identify his or her weaknesses and correct them. The second is to allow the student to have a comparative idea of his/her performance in relation to everyone else in class.

    Second, I have never seen grades diminishing a student. Those who get high grades are proud of their work. Those who get low grades must now worry about doing better next time or risk failing the course.

    There are various other arguments to support the use of grades (RED and BLUE marks) but for the sake of space, perhaps it can be discussed elsewhere. However, I must say that using a RED Pen reflects very much what you will experience in real life. You will be constantly evaluated in everything you do. You will have to take standardized tests (which will not go away), you will submit your work to be reviewed and criticized by peers, and you will be selected to positions of power and knowledge based on your performance rank.

    Thanks for your article. I am sure you will appreciate an opposing view. Hopefully others will comment as well.

    Paulo Castro | www.fullminddesign.com

    By the way, you can check out my new blog at blog1.fullminddesign.com

    ReplyDelete
  2. I'm concerned that a single question is being used here to evaluate so many criteria at once. Perhaps a simpler, more modular set of exercises could build up understanding to a mastery level on individual points. How about something like this: http://slidespeech.com/s/s4vBG7Ymen/?autoplay=true

    ReplyDelete
  3. I appreciate your attention to the evaluative feedback narrative... so often - in school and in life/work - when expectations are not met, or we get something "wrong", that's all we're told. I see this in supervision and leadership all the time. People miss the mark, and are given "constructive feedback" about the fact that they did something wrong, failed to meet expectations, but often aren't given the valuable insight into how this is judged. Often because supervisors and managers haven't ever given thought to clearly articulating how they make their judgments - going instead on "gut" reaction, or generalizations. But in school, work, or life, if we find out simply that we've done something "wrong" - we are left to our own devices on a trial-and-error quest for improvement... when having some clearer understanding of HOW we missed the mark, some direction on how to proceed or where to go next would be infinitely more productive. (and by the way, builds a more solid relationship as well)

    ReplyDelete
  4. PS - at the absolutely worst end of the scale is something I've encountered several times with my daughter this year (middle school) - a grade shows up in the portal, and she has to pester a teacher to even see the test afterward. So I ask her, "What did you miss?" or "Do you understand what you did wrong?" or "What do we need to review some more?" and she has ABSOLUTELY no clue. Yikes.

    ReplyDelete