Saturday, April 20, 2013

Changing the Message: Grading vs Reviewing (Part 4 of 4)

Continuing the saga....

Rigid Grading Structures

OK, I recognize that this is a touchy subject, so I'll tread lightly (for now). The heck with that, let's get dirty!

"You, my friend, are going towards turbulent flows."
An Anonymous Friend
Today, after telling him about what I'm writing 

We follow rigid grading rubrics hoping to be "fair" to everyone. The problem is that these grading structures put us "fairly" off base.

"Grades tend to diminish students' interest in whatever they're learning. A "grading orientation" and a "learning orientation" have been shown to be inversely proportional...
Grades create a preference for the easiest possible task. Impress upon students that what they're doing will count towards their grade, and their response will likely be to avoid taking any unnecessary intellectual risks...
Grades tend to reduce the quality of students' thinking. They may skim books for what they'll "need to know." They're less likely to wonder, say, "How can we be sure that's true?" than to ask "Is this going to be on the test?"
Alfie Kohn, 2011 

What do we value?

Rate of Learning?
Imagine the situation in which a student learns the material at a slower pace than others in the class. The student earns a poor score on the first exam. This feedback inspires the student to try a different approach to learning and (s)he grasps the material in time for the final exam. Following the rigid rubric, this student's grade will be lower than another who learned the material faster, even though both may be leaving the course with the same understanding.

What is the message here? What do we really value? The rate at which a student learns? Rate of production is a crucial concept to efficient manufacturing.

Get it "Right" the first time?
Students are penalized for making mistakes. The result is that students are afraid to try anything unorthodox, or explore.

"I just want to know what formula to use to solve the problem the right way."
There is a high level of pressure on students to get things "right" the first time. Our RED PEN feedback system helps drive this home. If a student misses a question, points are lost and final course grades are affected. GPA drops and that homework problem or test problem just influenced the rest of my life. (OK, that's a bit extreme, but its the logical end to the thought).

The Root of the Problem

We have a need in education to determine how a student is doing. Students need feedback so that they may improve upon problem areas. Educators need to see how their students are doing so that they may assess the effectiveness of their teaching.

There is a long list of other reasons typically thrown up in the case for grades, but I think they are rubbish. One of the big ones is that "if we don't grade it, students won't do it." Yes, let's hold our students hostage with threats against their future success rather than providing true motivation.

Had I the heavens' embroidered cloths,
Enwrought with golden and silver light,
The blue and the dim and the dark cloths
Of night and light and the half-light,
I would spread the cloths under your feet:
But I, being poor, have only my dreams;
I have spread my dreams under your feet,
Tread softly because you tread on my dreams
W. B. Yeats

I think we may have started, long ago, with these two goals in mind. However, our "factory model" education system has polluted those goals, mainly due to the fact that we are trying to educate so many people. Call the EPA (Educational Protection Agency)!

Scaling Issues

With a large class, the task of providing feedback is not trivial. Graders must streamline the process by providing the least amount of feedback possible. They look over the work, think about what the student did, and try to scribble something down to let them know where mistakes were made. If only the student could hear the grader's thoughts! (In light of what I'm about to say, I think they call this foreshadowing)

Very rarely is any positive feedback given. For the most part, graders write down just enough to justify the score on the assignment. I don't fault them for this, they simply don't have the time to give meaningful, content-rich responses to each student's work.

The problem is that, from the students' perspective, this type of feedback is more a slap in the face than constructive criticism. Rather than a teachable moment, we offer up judgement and discouragement.

I have not failed. I've just found 1000 ways that won't work.
Thomas Edison 

Let's change the message.

Learner-Centered Feedback
(Stop Grading and Start Reviewing)

With tablets and pen inputs for computers we have the ability to record ourselves as we mark up a document. Imagine marking up a student's electronic homework submission. The grader no longer needs to write a detailed response to help the student learn from issues. The grader can record his/her thoughts in audio format while marking up the work.

Rather than grade the work we can review the work and provide meaningful feedback (audio) to the learner. I think if we did this right, it would actually save time for grader. Most people can speak much faster than they can write or type.

We could add "content area" tags to individual homework problems. Over time, the student's aggregated results would point out sticking points in a particular course and focus the student's study/improvement efforts. If a visual representation of this were available to the grader small bits of encouragement could emerge: "I see that the First Law is starting to click for you, good job!"

I can see clearly now....

Synced with the interactive Degree Path Sheet, a student would have a much better view of where they are. With our current approach most students can't see what's going on. All they see are a few "red x's" that leave them feeling less intelligent, or that "I just don't understand Thermodynamics." Truth is that's usually not the case. Maybe the real issue is a small part of it that creeps up in most problems.

OK, enough words...

Let me show you an example of what I am picturing, from an engineers perspective (sorry, I haven't graded anything else!):

If you kept track, I spent about 2 minutes providing feedback on this problem. In a class of 60 students, with an average of 10 homework problems per assignment, this translates to 20 hours of grading work. Interestingly enough, that's the exact amount of time that my grader logs while providing RED PEN feedback right now.

A few things to work out:

First, I'm laughing at myself for using a RED PEN while making this video. I imagine a better system in which I can use "cursor points" or a "focus bubble" to show what I'm looking at.

Second, the "tags" idea isn't a worksheet, but I'm not Java programmer. The check-mark image was supposed to represent clicking on tags.

Third, I DO use a red pen while grading currently. This idea is new for me this week, and I hope to implement a refined version of this system next time I teach a course.

Fourth, I DO collect the homework for a grade. I'll explain...

Some other things you might have seen:

Did you notice that the numerical answers were provided with the problem statement? This is a trick I use to maximize the self-learning for my students. With the answers to homework problems available, students know whether they've got it or not long before they turn it in. Assuming they start the work early enough, they can find the help they need before turning in the assignment.

Did you notice that I showed two submissions from the same student? I actually do this as well. In my course I try very hard to encourage my students to make mistakes and learn from them. This learning business is messy and most of what sticks in our minds comes from getting things wrong at first.

I allow my students to re-submit the work as many times as it takes, and they can earn full credit on every problem even if they don't get it until the last day of the semester. Effort and engagement is encouraged and rewarded. Based on their feedback, this is working as I hoped it would!

Closing: The Message

Once again, the "Medium is the Message." I hope that by taking advantage of the digital media tools available to us, we can shift the message that we send learners. Let's send the message that experimenting is a good thing. Let's send the message that mistakes are a good thing. Let's send the message that our students deserve more than some messy red pen graffiti all over their work.

At the risk of starting a riot:
STUDENTS: Like what you see? Are you tired of getting RED PEN all over your work? Demand more from your teachers!

Try turning your homework in written completely in RED PEN.

Changing the Message: Degree-Level View (Part 3 of 4)

Continuing the discussion on changing educational media to change the message we send.

Degree Path Sheets

ME Department

The degree path sheet is a very helpful tool for students. This single page summarizes all of the courses that they need to take to complete the degree, including links (connecting lines) to pre- and co-requisites. As I pursued my undergraduate degree I referred to this page very often. (Side note: In my day the form was in black and white! I've always wanted to say that...)

I think we can take advantage of the digital medium and do better.

Add more information 
(information that's already available elsewhere)

Couldn't this be an interactive web app? Imagine being able to click each course and find useful information. Linked to the scheduling system, times that the course is offered could be displayed, along with the number of open seats. Click the one you want to instantly add it to your schedule. The entire app could sync with the student's transcript, showing completed courses and grades. Courses could be "clickable" only if the pre/co-requisites are satisfied. Selecting a course could automatically select co-requisite courses.

Professor bios could be linked displayed, allowing students to pick their teacher based on research interests and examples used in class. We could add lecture previews, student reviews, links to course materials, and publications by the professor. Previous class projects could be linked, letting prospective students see what those who have come before learned.

As a student progresses, the system could "learn" her/his preferences and recommend certain elective courses or professors that align with the student's interests and learning styles.

The learning management system (even though I don't like the current one much) could be linked in, allowing the student to use this page as a launch to current classes. Information such as current grade and how much of a course grade remains could be available. GPA (which I also don't like) scenarios could also be computed.

If we got to a point where students could truly customize their education, a % Complete bar could indicate how much of the course they have completed based on the learning modules they have selected. The division between subjects and the need for completing a course in a certain semester could be eliminated. Imagine this entire sheet broken into smaller chunks of concepts that add up to form a uniquely-designed curriculum.

How ready am I for this course?

A preparedness index could be fashioned based on a student's past performance in prerequisite courses. Suggestions for important background material that the student found challenging in the past could be offered as study aids over breaks, before the course begins. If we changed the way we grade (see next post) the system could point out specific areas that the student should work on. Example: the introductory course I teach uses first-order ordinary differential equations, but that's about it from the "Differential Equations" course that is listed as a prerequisite. If a student struggled with that part of Differential Equations we could flag it for them.

Perhaps over time this is an answer to a challenge facing almost ALL courses: we spend the first 1/3 of the semester reviewing concepts from previous courses.

A Liberal Education

If you take a close look at the degree path sheet, you'll see classes marked "Area 1" and "Area 2" etc. As part of a liberal education, we have 7 Core Areas that students must engage with to complete a degree. As a student, I never took the time to read the description of these Areas (take a look at the link). The truth is that the idea behind each is beautiful and very exciting. Why can't these show up on the interactive degree path sheet as well? I wish I had seen the inspiring ideas behind these requirements, rather than just "toughing it out" through some required "useless" courses. They get at the heart of the difference between education and job training. The available courses that satisfy an "Area" during any given semester could also be suggested/shown.

My Story is Different!

One of the big drawbacks to the rigid degree path sheet is that very few people follow it. We all have extenuating circumstances (drop a class, change majors, take a co-op) that put us "off schedule". In a year of teaching, I've met very few students who are actually "on track."

This sheet serves as a constant reminder that students are off track, or that they are somehow "doing it wrong".

 Why not make each class "draggable"? Students could slide the courses around, and maybe even add a "solve my schedule" function that charts a path to degree completion based on current standing and course availability.

The New Message

The point here is that with some application of fairly standard web design, the message we send is completely changed:

"We recognize that you are a unique individual with a unique story, and we want to meet you where you are at and help guide you along. You don't have to fit our mold, because there is no mold to fit. Your education is important to us, and you are in the driver's seat."
ME Department (modified)

Kaizen Was Here (Part 2 of 4)

Continuing to think about "The Medium is the Message"

(Wiki, modified)

A recent experience confirmed in my mind what the Communications Department has been saying for a long time:
"The Medium is the Message"
The way that information is presented has a strong effect on how we view it, feel about it, remember it, and use it.

So here is my question:

What message are we sending our students?

Who's important in this room?

"Save your drawings for art"!????? Heaven forbid a student expresses creativity outside of art class! Nick, thanks for the beautiful image: "I'm a robot." I'm reminded of Ted Nelson:

"Material is dumped on the students and their responses calibrated; their interaction and involvements with the material is not encouraged nor taken into consideration, but their dutifulness of response is carefully monitored."
No More Teachers' Dirty Looks
Theodor Nelson, 1974


How well does this represent what you've learned about <insert any subject here>? What level of engagement was required to prepare for this type of exam?

These all-too-familiar images are a result of choosing the "easy solutions" to a problem that has plagued education throughout human history: "How do we scale this? How do we teach 50 people to be <insert a field here> with one teacher?"  Rigid classrooms built around the "Sage on the Stage", multiple-choice exams, and Scantrons are VERY convenient for the teacher/administration, not so great for the learner.

Welcome to the degree factory.

One Size Does NOT Fit All

The idea behind industrialism is that we can streamline a process to produce a large quantity of the same product. As students hoping to find a place in the world after graduation we spend our time in the factory attempting anything we can to come out different than everyone else. The very idea behind our efforts is to break out of the manufacturing model. That's the basic question asked in most job interviews: "What makes you uniquely suited for this position?" We want to have a good answer.

(Let me be careful here, this is an example. The true goal of education is much more than job placement)

What I've learned From Social Media:

Good or bad, we are enthralled with social media. Facebook, Twitter, LinkedIn, Vimeo, YouTube, Flickr, and Pinterest have permeated our lives. My theory to explain this is simple:


The overarching message of Social Media is that people care about me and what I have to say. I am NOT a number without identity, lost in the masses. I'm a member of a community that is concerned with what's happening in my life. My thoughts are important to someone.

For the last century our educational system has been built around a factory model. Input people and some (unfortunately limited) resources and out come "educated" graduates. For the most part we've been OK with this, having never experienced anything else.

Social media is awakening a higher value in uniqueness and individuality. We won't be tolerating the faceless and almost anonymous educational experience much longer. I hope the institutions of learning can prepare for it.

The Rigid System

I've already expounded on the idea of customizing education, and letting students take the role of designer. Education needs to move from a "factory" towards a "custom shop" mentality.

Improving education requires flexibility. Our current system is rigid. We give rigid lectures with little to no interaction on the part of the students. We assign rigid textbooks that are arranged according to the authors' view of the "best way" to learn a subject. And we march through the book in a linear fashion, moving down the assembly line. We give rigid homework assignments that have one right answer forcing students to figure out what someone else has done, rather than creating something new.

These rigid "material delivery systems" do not permit much exploration, learning at different paces, or different interests.

The message is clear:
Do it our way.

On the one hand, we need systemic change in our approach to learning. On the other, some fairly small changes can have a large impact. I'm reminded of the idea behind dimples on a golf ball. The dimples make small changes to the airflow around the ball, leading to HUGE improvements in drag. Sometimes changing the ball is the answer. Other times we can make intelligent small changes that give dramatic results.

The Medium is the Message

The examples above certainly streamline the process of teaching, allowing us to offer the same education to a large number of students. The problem is that these edifices also communicate a subtle, but deafening, message. The result: lack of creativity, exploration, and individual thinking on the part of the learners. We tell students that the ideals are important and then crank them through a system that penalizes or discourages these 21st Century Aptitudes.

Presented in the following posts are some thoughts I've had recently about small changes we can make that could have a far-reaching impact on our students. They are just some examples of the important idea:

We have the technology to redesign our educational media, completely shifting the way learners think/feel about learning. And it's not even that hard to do.

Wednesday, April 10, 2013

The Galaxy Reconfigured (Part 1 of 4)

(or the Plight of Mass Man in an Individualist Society)

I may have mentioned that I am participating in a seminar that is discussing the impact of the new forms of media made possible by the computer on life and education. We are working our way through a collection of "essays" written over the last century that discuss the ideas and possibilities that computers offer. One of these essays is entitled "The Galaxy Reconfigured, or the Plight of Mass Man in an Individualist Society", by Marshall McLuhan. The piece is followed up by what has become a communications theory standard: "The Medium Is the Message."

The engineer in me tried an experiment that I hope you'll find interesting. After I tell you about it, I'll give some more thoughts on education that it brought up (follow up post).

The Old Media Reader

If the book we are pulling these essays from can be called the "New Media Reader," McLuhan's essay ("The Galaxy Reconfigured") is certainly the Old Media Reader. Here McLuhan presents a history of media, ideas, and how we spread them and shape society. I made it through a whole page of the essay, filled with references to everyone from Quintilian to Arthur Rimbaud, before I gave up. I had NO IDEA what he was talking about. I take some solace in the fact that even people who study McLuhan haven't got a clue.

So I skipped to the next essay by McLuhan that was on the docket ("The Medium Is the Message"). This one clicked a bit, I think. One of the main ideas is exactly what the title says: the content (e.g. the words on the page) is not the main message, but rather the medium (e.g. book, newspaper, movie) communicates something much more important. Deep, my head hurts about the implications.

My Experiment: The Medium is the Message

I started out reading "The Galaxy Reconfigured" from the book. Its a standard textbook-sized volume, hard cover, no color. As I mentioned above, it didn't take too many references to 18th Century poets for me to lose track. I admit my literary background is a bit weak, and so the significance of the people he references was completely lost on me.

In this static, one-way-through-it book, I got the message loud and clear:


I tried something different. I scanned the article to PDF (hope Big Brother doesn't find out, but I'm not distributing) and tried to re-read the article on my iPad. 

Suddenly the whole thing felt different.

When I came across a reference I didn't understand, I switched over to the internet and searched for the person. Within a few seconds I had found articles, selections of their work, and biographies on these people. I found enough context to grasp some of what McLuhan was saying.

The message changed entirely:

It might take some work, but YOU CAN DO THIS, and it is BEAUTIFUL

By the end of the article, I saw how beautifully McLuhan unpacked the history of media and ideas, and actually appreciated his closing comments:

"Our most ordinary and conventional attitudes seem suddenly twisted into gargoyles and grotesques. Familiar institutions and associations seem at times menacing and malignant. These multiple transformations, which are the normal consequence of introducing new media into any society whatever, need special study..."
The Galaxy Reconfigured
Marshall McLuhan, 1962

Oh, I learned some new words too: parataxislinealsomnambulism, and chiasmus.

Ok, Ok, I know what you're thinking....

Obviously, sitting in a library or at my computer I could have done the exact same thing. The iPad didn't really do it for me. Fair point, but I don't typically read while sitting in front of a computer, and unfortunately I don't spend as much time in the library as I'd like. I take these types of articles with me, and read them as 15-minute mental "snacks" to break up a day filled with jet engines, pitot tubes, and thermodynamics. Lugging the full book around is cumbersome, and I typically don't carry my computer with me.

Can we do better?

Of course. Wouldn't it be nice if the iPad (or other tablet device) were capable of embedding those links for me? I'd like to be able to click any word, pull up a small summary on it, and go back to reading without ever leaving the "page" I'm on. We could embed other forms of media, links to commentary, current discussions, etc. My fairly linear march through the article could have been a much more valuable experience with a bit more flexibility that something like a computer is already capable of. 

By scanning and looking up the links myself, a word used frequently last week by Janet Murray comes to mind: Remedial. My little "trick" is a band-aid, barely scratching the surface of what the iPad/Tablet medium is capable of. Let's do better.

I will not cease from mental fight,
Nor shall my sword sleep in my hand,
Till we have built Jerusalem
In England's green and pleasant land.

-Jerusalem, William Blake

Continue the 4 part series: Kaizen Was Here (Part 2 of 4)