Instructor Interview: Assessment and Feedback in Creative Contexts

• Download this transcript - PDF (English - US)

The following content is provided under a Creative Commons license. Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free. To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu.

ELIZABETH CHOE: Assessment for this class was probably the hardest thing for me, at least. How to assign a fair grade to students, because at the end of the day, they're a grade for this. And it wasn't going to be an A for effort type thing. It was actually really hard because anything that requires creative effort-- I talked to you about this for a long time-- you don't want to crap on people's creative efforts, even if it's just not very good.

CHRIS BOEBEL: It's very hard.

ELIZABETH CHOE: Jamie and I talked a lot about this. But initially, the rubric was very plastic elements based. Like, is this well lit, can you hear the audio OK? And we just kind of scrapped that, and really went with-- there were four sort of principles to the class that I talked about on the first day, which was spark, does it foster curiosity, and that was like a big theme that went through every single assignment, and how it manifested change assignment to assignment.

You can see on the rubric. Kill your darlings. Like, is it concise enough. There are a couple others. So that's how we approached grading. And we assigned them raw numerical scores, and we didn't have like, an A is 20, a B is 10, or 15. It was in progress was from 0 to 5, and steps moving forward was like five to 10, and then mastery was like 10 to 15, for instance.

So they would get numerical grade feedback. We gave a lot of weight to their daily reflections so that not everyone in the class would end up with a C.

GEORGE ZAIDAN: I have a question. Do you think grading should have been A for effort? Because in real life, and I didn't tell this to you when we were planning the course, and I'm sorry. But in real life, it almost is A for effort. If I show up to do voice over and I'm late and I'm sloppy, and I don't do a very good job, I'm not going to get hired again.

And part of that is my technical ability to do voice over. And part of it is showing up 15 minutes early, being professional, all that sort of stuff, which does kind of seem like A for effort. I don't know. What do you think?

CHRIS BOEBEL: Well. So I've had to think about this a lot for the course I teach at MIT. And it's a huge, huge issue because, so who should get the better grade? The person who has a huge amount of talent and experience as a filmmaker and video maker, sluffs it, just like lays back throughout the whole course, but does a really nice piece at the end.

Or the person who never has done it before, doesn't necessarily have a whole lot on the ball, when it comes to media production, but just works and works and works and comes out with something that's kind of mediocre. Just because by the nature of it having never done it before. That's really the issue. When you're dealing with non-professionals-- because I completely agree with him, it's the Woody Allen thing right? Like most of life is showing up.

And it's totally true. But when you're dealing with this huge, huge learning curve, it becomes very, very tricky. So what we do, we don't do quite A for effort. But we do kind of allow an out if your final project, which is a big part of your grade, goes completely haywire and just falls apart on you, you can write a fairly substantial paper that describes what your intents were or what your thought process was, what went wrong, why it went wrong, what you would have done differently.

And so, essentially, you do have a way supplementing or making up for this innate problem you had, which was that you didn't know what the hell you we're doing. Which is a problem.

JOSHUA GUNN: It's fairy innate for me.

ELIZABETH CHOE: For all of us. Well, two of the course values were challenge, and a lot of that manifested in the rubric in the final project and the graded assignments, as is their improvement from the previous iteration? Is this person going outside of their comfort zone? And then thoughtfulness. Can you tell that this person has integrated some of the things we talked about in class, whether it's through their blog posts, or just like office hour reflections, things like that.

So that was built into the final grades, but then I would also say that the graded sort of quality projects, and the daily reflections were weighted pretty equally so that if you had a student, which we did, who had the skills, but didn't do the daily blogs, their grade would suffer a little bit more. Just as much as someone who did the daily blogs, and just could not host for instance.

In the end it worked out by doing it that way. All the students who I thought deserved great grades, because I knew they worked really hard, got good grades. And all the students who were very technically skilled still got decent grades, but it wasn't like they got 100 or anything. It's always a tough challenge, though.

GEORGE ZAIDAN: My math teacher used to say-- we'd be like, how are you going to grade this test. He'd be like, well I'm going to blindfold myself, turn around and throw darts behind my back. And we were like, good. That's good.

CHRIS BOEBEL: I just which I could have written a paper in chemistry class in college to make up for the--

GEORGE ZAIDAN: My experiment went south. And here's what happened.

ELIZABETH CHOE: It's hard because MIT students, they want to know what do I need to do to--

CHRIS BOEBEL: Exactly. People are very focused on achievement and doing well.

ELIZABETH CHOE: Which is why I was very conscious of by creating as specific of a rubric as I can.

JOSHUA GUNN: That makes lot of sense, especially as you said before, because this is a form of art, and there is no right or wrong answer, so how do you--

ELIZABETH CHOE: But you can tell someone, look you didn't change anything from your first draft. So you're going to get docked for points on challenge, as the rubric says. Or you straight up didn't do a blog reflection so you're getting a zero on that section. But at least I do think that there has to be something tangible to go back to that you can't just sort of arbitrarily assign a grade.

I will say that, at the end of the day, an assessment, a great assessment, should be a reflection. Or should be there to help students improve. And I do think that the value in assessment and feedback came from just sort of one on one, when you all worked with the students directly.

And that's something that's not captured on the OpenCourseWare class. But I would stay with Jamie and our TA Siri, for at least two hours every day after class, just talking to students. And you work with them.

And when you and John came to do the workshop, you'd just have direct interaction with the students. I really think that's where the most value came from. I don't know how you all feel about that.

CHRIS BOEBEL: I thought that workshop was great, by the way. I didn't get a chance to tell you, it was really nice.

JOSHUA GUNN: I mean, I thought it was just great to interact with the students. I mean I wasn't immediately sure how engaged they were, but I had some sense of it when I would actually just sort of sit down with them and they would directly ask me questions and there was that sort of that direct engagement.

And I guess one would question how this course could be done electronically, or via some kind of distance learning, or that kind of thing. Because that seemed invaluable to me. That kind of personal contact.

ELIZABETH CHOE: If you are teacher in Missouri, for instance, how can you teach this course without a Chris, without a George, and without a Josh.

GEORGE ZAIDAN: I mean, there are versions of us in Missouri.

ELIZABETH CHOE: There are.

JOSHUA GUNN: I have an alter ego in every state.

ELIZABETH CHOE: You said something nice about my home state. It's on camera, too. But really, again, you can watch the OpenCourseWare stuff to get a sense of just, I mean, they're just fundamental facts, I guess, that you should become aware of. But I gave a midpoint survey to the students of what lectures have you found most helpful, and they were all like, all of them.

Like, Chris' was really helpful because I got his perspective as a filmmaker, like as someone who was, and still is, in the industry. And then working with Planet Nutshell, they were like, oh I had no idea how to storyboard. And it was waving more helpful than just watching a video of someone do it. We were actually interacting with them.

Your hosting workshop, like how do you give hosting feedback not in real time, not live. I don't know.

JOSHUA GUNN: I just remember sitting down with a couple students and I think students were running their ideas by both of us. Maybe me individually, and John, just giving sort of direct creative feedback. Well, I think that could work, but have you thought about doing this? And then that would spur them to further sort of creative thinking. That kind of interaction, that relationship, seemed invaluable.

CHRIS BOEBEL: And the one on one interaction. I've thought about that too. I mean maybe the equivalent of crowd sourcing or something. But most of the valuable experiences I had as a student, and that I think I've had with students as a teacher, are really not even in the classroom, but one on one.

JOSHUA GUNN: Yeah, I mean, most of the experiences I remember most powerfully are being at office hours and having a teacher asks me like, so do you really think you're going to be an English major?

CHRIS BOEBEL: And particularly in media or video, it's like you have this work, it's a piece of crap, and someone is going to sit there and help you turn it into something else. And just to learn what can be done to a script, to something that's been shot and not edited the way it could be. It's a really powerful experience.

JOSHUA GUNN: Yeah definitely. And so I wonder, sorry to interrupt you, I just I just wonder if the balance-- I don't have a complete picture of how the course was balanced, because I wasn't there the whole time, but I wonder if there could be more opportunity for that kind of a one on one interaction.

CHRIS BOEBEL: It's very short. That's the challenge.

JOSHUA GUNN: Yeah.

ELIZABETH CHOE: Yeah.

CHRIS BOEBEL: It's such a short class.

JOSHUA GUNN: Like some kind of critique or workshop that maybe we all attended and gave thoughts on?

GEORGE ZAIDAN: And for the students who worked with us in production week, that was a lot of one on one. A lot of one on one, or like, three on one, in that case. But there again, for production week it was like you don't spend two hours like, here here's why your hosting was not optimal. You just tell them what to do to fix it quickly so you can get the next shot.

But that still is valuable because you still see how a professional does it. You're there and you're doing it. You're not just sitting there watching. Without you, the show doesn't run. Without the student, the show doesn't run. So they have to do it.

ELIZABETH CHOE: A couple of the students who did Science Out Loud, they waited to write their final blog reflection until after they shot. So they were like, I wanted to wait to see how different it would be from the class. And even in that one day, I learned so much. Like, more than I did in the two weeks of the class. Just like being in that production setting. Sorry go ahead.

GEORGE ZAIDAN: That was it.

ELIZABETH CHOE: Something I wanted to mention, again, this kind of thing isn't scalable. We can't spend one on one time with 20 students. It was manageable because there were only seven or eight of them at any given time, thank goodness. Initially were like, yeah we're going to blow it up, make it like 24 students. Oh my gosh, no. It was a struggle to keep up even though our teaching team is huge, just because it is such an intensive process to work with them one on one.

But what I did want to make use of their peer-to-peer learning and support. And somehow, I don't know exactly know how this got fostered, but we really had a nice class environment. The students were very supportive of each other, which was really nice, maybe because it was such a small class. But during the second week, they would go off in groups to shoot each other, and they would give advice to each other. Which was awesome.

I would overhear are some really great directorial advice, but I would also overhear some advice that would be like, no. Don't make them have that intonation. I wouldn't say anything because I think it's important for their learning experience. But that's what I mean by there's only so much you can do with people in the same sort of learning sphere teaching each other.

JOSHUA GUNN: It's a very short-- I mean so much of what this class is like, is characterized by the fact that it's so short. Because the cohort thing is so important. And, I mean, as someone who, I'll confess, went to art school. I know, sorry MIT people. So I went to NYU, the graduate film program, the three year program.

So the first year of that program, is kind of like the first couple of lectures in your class. It's all class based, you feel like your head is exploding with all this stuff. And you're hanging on your professor's every word, and it's like kind of transformation. After that, it's kind of pointless to go to class. It doesn't really matter what happens in the class.

It's all about the cohort, and it's all about what you do outside of class, and the help and support you give each other as you make these iterative projects. So it kind of takes over and becomes the thing, the only thing. The relationship of the students with each other, critiquing each other's work, working on each others projects.

The rest of it, once you kind of have the basics, the basic conceptual framework--

GEORGE ZAIDAN: It's like learn by doing.

CHRIS BOEBEL: Yeah, it's all learn by doing after that.

ELIZABETH CHOE: The problem is because it's a two week, or three week, class. Or even a semester, or even a year. At MIT, I mean, these students are nowhere near experts. Not saying that we are, right, there's only so much that they can be aware of with each other, that it was a weird balance to strike.

Free Downloads

Video

• iTunes U (MP4 - 39MB)
• Internet Archive (MP4 - 39MB)

Subtitle

• English - US (SRT)