In the following conversation, Elizabeth Choe and Jaime Goldstein talk about how they approached grading in 20.219 Becoming the Next Bill Nye, a course focused on students’ creative work. They also discuss the challenges associated with and strategies for providing students with meaningful feedback about their work. Their conversation has been edited for clarity, length, and content.
ELIZABETH CHOE: Although this course was very much about students’ personal journeys toward digital media literacy, I didn't want it to be an A-for-effort class. I wanted students to develop a sense of self-awareness about their creative work, such that even if they didn't hit the mark, they would be aware that they didn't, and know why they didn't.
I had a conversation with Chris [Boebel], who also teaches another media class, about this topic. He said that one thing he routinely struggles with is deciding if students who are inherently talented, but throw together work the night before the assignment is due, should receive the same grade as students who do not exhibit intrinsic ability, but who work really, really hard, spend a lot of time on their assignments, and are aware of the ways in which their work falls short of expectations.
Elizabeth Choe confers with a student about the rough cut of the video she has produced for 20.219 Becoming the Next Bill Nye (Image by MIT OpenCourseWare).
Including a challenge aspect in our rubric (PDF) and assessing students’ relative growth helped us address this struggle in to some degree in 20.219 Becoming the Next Bill Nye. We pushed students to challenge their audiences and to challenge themselves. When we assessed students’ second drafts of their scripts or second cuts of their videos, part of their grade was determined by the degree to which they integrated the feedback we gave them on their first drafts and cuts. If a student wrote a really good first draft, and I only had a couple of comments, and he or she addressed those comments in the second draft, then, yes, that student totally deserves an A. Even if it only took them five minutes, they still deserve an A. I'm not going to penalize them for having only a few changes to make. Their growth trajectory is going to be different, but it doesn't make them any less self-aware, or less appreciative of media. If a student doesn’t integrate our feedback, his or her grade will be lower.
JAIME GOLDSTEIN: But it wasn't really about just taking in the feedback, right? Because we always made them feel like they were the authors, they got to decide what elements to include in their films. I think it was less about acting on the feedback, and more about understanding where the feedback was coming from.
ELIZABETH CHOE: Yes, it was about students’ abilities to justify their decisions, as well. We told them we were going to give them feedback, but at the end of the day, it was their product and their decision. At the same time, however, we told them they had to be able to articulate why they made those decisions.
The daily blogs were key in helping us understand students’ decision-making processes. These self-reflections pieces were informal—they weren’t long essays—but they really helped us understand how self-aware students were on their journeys. Also, we tracked students’ completion of the daily blog entries, but we did not assign them a score.
JAIME GOLDSTEIN: You did, however, assign grades for other assignments.
ELIZABETH CHOE: Yes. I was a hard grader. A lot of people did not earn stellar grades for the first few assignments. But because so much of the final grade depended on the final blog entry, it wasn't a devastating blow if you didn't do well on the first few assignments. Also, from the beginning of the course, we tried to be as transparent as possible about grading. We encouraged students to talk to us any point if they were worried about grades. However, no one voiced concerns.
I wanted students to develop a sense of self-awareness about their creative work, such that even if they didn't hit the mark, they would be aware that they didn't, and know why they didn't.
—Elizabeth Choe
JAIME GOLDSTEIN: Right. I think concerns arise when there is not transparency related to grading. I’ve always always found that students get frustrated when there's a veil obscuring grading practices and they don't understand how or why they're being evaluated. Frustrations also arise when students haven't really thought deeply about their own performance. Getting a letter grade without context can feel jarring.
When I was a teacher, I gave students many opportunities to get feedback before I gave them letter grades. That way, there were no surprises. Many times this involved having students self-assess their work based on an agreed upon rubric. I think this is invaluable.
In my role at MIT, I work with faculty members in a lot of different departments. I see many faculty members struggle to figure out how to give meaningful feedback without it taking a billion hours. I actually just worked with a professor who assigns a 15-page paper in his course in which 30 students are enrolled. I helped him scaffold the course, and structure it so that the students would be more successful. One of the things we worked on was strategies for being more transparent in his syllabus about what constitutes successful papers and how he grades them.
I saw him recently, and his students were nearing the point which they would be submitting their papers. He said, “I don't think these papers are going to be good enough.” I reminded him that he had one week to give feedback to students before they earned real whoppers for grades. I came up with a super quick six-point checklist based on what he wanted students to accomplish in their papers. Then I told him to put his pen away. I asked him to read a student’s entire document, without writing a single comment in the margins, and to simply ask himself, of the six most important things he wanted students to accomplish in this paper, which did he or she do successfully in this draft? Give a check for each important thing demonstrated successfully. If the student didn’t meet his expectation, tell the student he or she had exactly four days to fix that before a grade would be assigned.
Jaime Goldstein listens to a student during a classroom discussion (Image by MIT OpenCourseWare).
This is what I consider really effective feedback because it gives faculty an opportunity to say to students, “Here's where you are. Here's my vision of where you should be. There are lots of resources available to support you in getting from here to here. Figure out how to close the gap.” For younger learners, you develop a structure that will help them close the gap. Learners in higher education can close the gap themselves. You don’t want to hold their hands too much.
This strategy of providing high-level feedback works well for busy professors. They don’t have time to go through each student’s draft in detail, but they can skim the papers and let students know if they’re missing huge sections they believe are critical. It’s triage.
ELIZABETH CHOE: You mentioned that you don’t want to hold university students’ hands too much when giving feedback. As an undergrad, sometimes I would get feedback on papers from professors that would just say, “improve this general thing.” I would feel so lost as to what they meant, or what they wanted, or why that general thing was missing from my paper.
Having had this experience, I didn’t want to leave our students too much in the dark. But at the same time, there were definitely moments during office hours during which I would say to a student that an aspect of his or her work wasn’t really working and they would ask for so much specific feedback that I felt like I was writing their scripts for them. It’s very easy to go down the line, and accidentally get yourself to that point. I struggled with that with a couple of the students. I don't know if I ever fully resolved that, but what I learned was that providing students with a menu of potential options for improving their work—saying, for example, “You could do this instead”—was helpful. But there were moments where I really just wanted to say, “Swap this scene out for this one,” because that's what I do when we produce Science Out Loud. But it wasn't about the final product in 20.219 Becoming the Next Bill Nye. Do you have any insights about this tension?
I’ve always always found that students get frustrated when there's a veil obscuring grading practices and they don't understand how or why they're being evaluated. Frustrations also arise when students haven't really thought deeply about their own performance. Getting a letter grade without context can feel jarring.
—Jaime Goldstein
JAIME GOLDSTEIN: I train graduate students to be communication coaches, so I actually think all the time about how to help someone to take ownership over their own learning. The last thing you want is it for someone take a free ride on you.
ELIZABETH CHOE: I don't even think that it was students wanting a free ride. It was almost like they were frustrated as learners and saying, “I know I'm wrong, but I don't know how I'm wrong.”
JAIME GOLDSTEIN: Sure. I mean, I think that it's a little bit different in our case, because there were some very concrete solutions at hand. However, typically when I talk with my graduate students, I tell them our goal is to have a blend between open-ended questions that help the learner think critically about his or her own work, and very direct feedback. You want to have a balance between telling learners what you think they could improve, but also leaving room for them to struggle a bit to figure out how they can make that improvement. It’s in the space of that struggle to come up with the answer that the learning emerges.
ELIZABETH CHOE: I think I did end up finding a balance between giving open-ended and direct feedback. For example, a student was using some analogies in his script that weren't really working, so I was commented, “In general, instead of making some of these analogies, remember that your visual elements are going to convey these ideas.” Then I gave a concrete example. It’s a hard, but important balance to strike, because I think it can be frustrating for a student to feel that an instructor is just micromanaging him or her, or, alternatively, leaving him or her in the dark.
On a related note, because the class was so small, I noticed that I didn’t always need to give students written feedback. We just met with the students one-on-one. In fact, most of the learning seemed to happen during office hours. The class session ended at 4pm and there was never a day I left before 6pm. That one-on-one time was so valuable to the students. I know students would stay and talk to you, Jaime, about their scripts, as well, which just goes to show the much the elements outside of technical camera work matter in work like this. Your background isn’t in video production, but your insights were just as vital.
JAIME GOLDSTEIN: That’s interesting. I think the thing that was really hard for students—and the thing I could help them with—was designing the story they wanted to tell. The actual filming of that story was beyond my area of expertise, but, you’re right, feedback about communicating their ideas was a key element in the course.