Lecture 6: Teaching Interactively in Large and Small Groups

• Download this transcript - PDF (English - US)

The following content is provided under a Creative Commons license. Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free. To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu.

PROFESSOR: OK, interactive teaching. Here is a famous graph produced by Benjamin bloom and his students. So here on this axis is student achievement. So after a course is over, students are tested on the material. And here's their score. And this is the percentage who attained each score.

So here, they compared three methods of teaching, but I'll just compare two. So one method is standard one-group instruction, say one-to-30, or today, maybe one-to-90. And here is the histogram, the distribution of students' scores, nicely spread out.

Now, by itself, it doesn't say much. But what's really interesting is the distribution base after the same amount of teaching, but done in a tutorial form, with one-to-one or maybe one-to-two. You can put some quantitative figures on this from Bloom's results. The average-- so this is one-to-one tutoring, one-to-30. The average tutorial student is two sigmas better than the group students. In other words, the average tutorial student is at the 98th percentile of the ordinary teaching group.

So the question is-- the natural question is, how can we make group instruction as effective as tutorial instruction? As Benjamin Bloom says, he says, finding such methods would be an educational contribution of the greatest magnitude. Because, for example, it would change our notions about human potential. We'd actually realize, oh, actually, there's not that much innate in what people can do. Most of it is how well they've learned, how well they've been taught.

So the great question-- and it's the title of Bloom's paper-- is the search for methods of group instruction as effective as one-to-one tutoring. To that end, let's think about the following question, which is, what makes tutoring so effective? So I'm going to ask you that, to consider that question in groups of two and three, whatever's sort of convenient in your region of the room.

So, what makes tutoring so effective? After you discuss, we'll take reasons, put them on the board, and then I'll show you methods of group instruction that may not quite get us all the way to tutoring, but that answer many of the good characteristics of tutoring and bring them into group instruction. OK, so please find one or two neighbors. Introduce yourselves. You never know. You might find a teaching colleague. And think about what makes tutoring so effective.

OK, so take 30 seconds and collect your thoughts together. OK, who would like to suggest something that your group thought of? You don't necessarily have to believe it, just something you discussed and think we should share. Yes? Could you tell me your name?

AUDIENCE: Elena.

PROFESSOR: Elena.

AUDIENCE: For one-on-one tutoring, I imagine there's less time for misconceptions to get ingrained in your brain.

PROFESSOR: OK, you can nip misconceptions in the bud. Whereas in, for example, group instruction, you may not even know the students have a misconception until the final exam. Then what do you do?

OK, let's see. This group, could you tell me your name?

AUDIENCE: So we thought--

PROFESSOR: Could you tell me your name?

AUDIENCE: Eric.

PROFESSOR: Eric.

AUDIENCE: We thought in a group, there's probably a lot of just dissemination of information. And in one-on-one tutoring, it's probably not one person just telling person two a bunch of stuff. It's going through exam problems.

PROFESSOR: Right, OK. So the contrast in this is what does the teaching situation look like? In a typical lecture, someone may just talk at you for 50 minutes, whereas it's hard to imagine a one-on-one tutorial situation where you walk in for your tutorial, and they just talk at you for 50 minutes. That would seem so odd and so counter to all conventions of human discussion.

It would take a really, really disciplined and well-trained person to sit still for 50 minutes for that kind of treatment, so not initially a good kind of training either. OK, so I'll say you cannot yammer at the audience for 50 minutes straight.

Yes, can you tell me your name?

AUDIENCE: Sorry?

PROFESSOR: Could you tell me your name?

AUDIENCE: I'm Whitney.

PROFESSOR: Whitney

AUDIENCE: I think it follows from this discussion that arises between two people that the student experiences validation of what they've understood, and it helps build their confidence.

PROFESSOR: OK, because they're talking in a small group, so they can get validation in a safe setting. Is that? OK, so it's safe. It's a safe setting for raising ideas maybe that you're not sure about.

OK, there was one more. Yeah, can you tell me your name?

AUDIENCE: Martin.

PROFESSOR: Martin.

AUDIENCE: Yeah. With tutoring, it's much easier to identify exactly what the student has trouble understanding, and then tailor the teaching according to that.

PROFESSOR: OK, so you can tailor the teaching according to what you find out. So sort of like the new electronic SATs. According to what you don't know or do know, in each question, they give you questions based on your previous answers at the right level of difficulty. So you can tailor.

Yeah, can you tell me your name?

AUDIENCE: Azrael.

PROFESSOR: Azrael, yeah.

AUDIENCE: I think related to this, it's very valuable to be able to tailor your teaching style to the student [INAUDIBLE] in general, not just specific points [INAUDIBLE].

PROFESSOR: OK, you can tailor the teaching style to the student. If, for example, the student is-- for example, suppose you're teaching math to somebody who's an architect. Well, you can use lots of examples that are particular to something they're totally fascinated by. You can tailor a teaching method.

OK, let me get something from somewhere back there, in case I've been neglecting people way in the back. Oh, go ahead. You guys had your chance. I'll be back. Can you tell me your name?

AUDIENCE: Graham.

PROFESSOR: Graham.

AUDIENCE: Just the level of engagement, it is impossible to sit there in a one-on-one setting and think about the music you like [INAUDIBLE] the subject matter at hand.

PROFESSOR: Right. OK, so the level of engagement is much higher in a tutorial. So that's the complement of this one. So the student engagement is much higher.

Yeah, just as it's hard for a teacher in a tutorial setting to just talk for 50 minutes straight, it's also hard for a student in a tutorial session to surf the web for 50 minutes straight, or use their BlackBerry or read the newspaper or whatever it may be that the standard methods of lecture distraction are.

Yes, can you tell me your name?

AUDIENCE: Question about the conditions in these experiments. So normally in a university setting in this country, tutoring is something that students seek out for themselves. It's not something that's provided in lieu of lecture. Was that the case in this? Was it really just that the same material was covered in the same semester and they just did it two different ways, and everybody had to go to tutoring?

PROFESSOR: Yeah, so what they did was-- I don't think these were university students. I think these were high school and even grade school students. Yeah. But what they did was they took the same number of teaching hours, and they either taught them in lecture or they taught them in tutorials, one-on-one, or I think sometimes one-on-two.

AUDIENCE: So in both cases, they were required to go to class, in a sense.

PROFESSOR: Yeah. I mean, I don't know if they actually went to the big class. But I'm sure they went to the tutorial class. That's partly of one of the differences. That's number six. Yes, way in the back.

AUDIENCE: From experience, I've found that you know students don't understand something. You don't why until you sit down. Usually you find there's some gaping hole that's quite often easily plugged.

PROFESSOR: OK. And how do you find that out?

AUDIENCE: You start asking them, what class did you have? What did you not understand? And sometimes they'll just bubble [? right out, ?] I never got this one key concept.

PROFESSOR: OK, so you can ask them questions, specific questions to figure out what they don't know, to diagnose what they're confused about.

OK, let's take one more, and then I'll show you some methods that hopefully address what these characteristics-- these missing characteristics of group instruction. Yes?

AUDIENCE: On one-on-one tutoring, for the student who might not be as into the effort [INAUDIBLE] added pressure, just feeling that there are other people [INAUDIBLE] student, there's no feeling of why I need to learn much more, [INAUDIBLE]. You don't really prepare yourself for anyone else.

PROFESSOR: Right, so the method of teaching automatically adjusts to whatever level the student is. So they're not comparing to other people. They're are actually just comparing to what they don't know right then. So let's just call it intrinsic comparison with the knowledge, rather than extrinsic with other students.

OK, so there's less emotional churn as a result of that. OK, so we could continue this list, but this is an excellent start. So now, if we look through that list, there's several themes that come out. I mean, it's hard in a group setting to figure out what students don't know. It's hard to ask them questions. It's hard to keep people engaged. It's easy to just talk at people as a teacher.

So what can we do in group teaching to capture as many characteristics of this as we can? Well, I'll show you methods that I divide into four time scales, ranging from short to long. But before I do that, let me ask if there are any questions on anything so far. OK. You were first.

AUDIENCE: How is student achievement measured? Is it just test scores?

PROFESSOR: Yeah, that was a test score. So what they did is, actually they gave them a test. Like, if it was algebra, they gave them an algebra test, and then tested them at the end, and at the beginning too, just to see where they were to start with.

AUDIENCE: So were the lectures more tailored to just teaching them algebra, and the tutoring was specifically tailored to doing problems in algebra?

PROFESSOR: Well, the tutoring was whatever they needed to do well on the test. And the lecturing was hopefully designed that way too. But it had all the flaws of, you want to teach so students will do well on your final exam. But what do you do when you have 30 diverse students in there? You had a question too.

AUDIENCE: I was just wondering, do we know which of these particular suggestions are the most important, and which ones might be good?

PROFESSOR: We have some idea. Good question. The question is which of these is most important, and which is less important. Actually, they're all getting at something similar. But there's a middle group in there, actually, that they studied, which was mastery learning.

So mastery learning, it's just like a programmed instruction. So the idea is that-- Keller Plan is another word for it. So mastery learning is midway between tutoring and group instruction. There was one sigma better, so 84th percentile.

So mastery learning, you don't go on to the new topics until you pass the mastery exam on the previous topic. So you're always building on a solid foundation. And so you work until you can pass the exam, and then you continue on.

And just by adding that to group instruction, you get one sigma better. So right away, diagnosing misconceptions, you can see, is quite important. But there's still another one sigma to get. And so that's, I think, what all the rest of these items are about.

And we do know a lot of stuff about how people learn. One of it is questions. Questions are essential. So I'm going to talk about that. But let me just ask again, any questions?

OK, good. This time it worked. So now, what did I do? So this is actually the first method. So how long did I wait after I said, any questions? Half a second? One minute? No, five seconds.

So the first time, I didn't wait five seconds, because actually a question came up a little before five seconds. But this time, the second time, I waited five seconds. Now, why is that?

So that is the first method. When you ask a question-- for example, "any questions" or "what do you think about this"-- wait five seconds. And I'll explain why in a second. But, methods shortest to longest. The reason I divide them by shortest to longest is the shortest ones are the easiest to incorporate into your teaching. So anytime you teach, if you ask a question, wait five seconds, so the five-second time scale.

OK, so why do I wait five seconds, and why not two seconds? Well, the reason is, think about your audience. Suppose you're talking away at them, and all the sudden you ask a question. Now, what are they hearing?

So everyone's seen Charlie Brown? So what does Charlie Brown hear when he goes to school? Wah wah wah wah wah wah wah wah wah wah wah wah, and all of a sudden that changes and there's a question. It takes about a second to realize that the wah wah wah has stopped, and you're being asked a question. OK, so one second's gone. Let's call it that, the thumb-- one second.

Now, it takes about another second to even figure out what the question is. Now you say, oh yeah, there's a question being asked. What did they say? OK, maybe now let's say it's one and a half seconds. So now we're at somewhere between two and three seconds.

Now it takes maybe one or two more seconds to figure out what you're going to say about that. Do I have any thoughts about it? Do I actually have any questions? Do I have a thought about the thing they're asking me? Do I know what the band gap is in silicon, whatever the question may be? So now we're somewhere between four, maybe near five seconds. And then it maybe takes another second to get the courage up to raise your hand and be ready with a sentence.

So now we're at least five seconds. So if you don't wait those five seconds, what you'll get is, you'll be getting a few questions, but you'll get questions from people who already thought about the material, in other words, the people who are already at the 98th percentile. But if you want to bring everybody in, and get everybody thinking about it, you really need to wait, so you don't get questions just from the usual suspects. So wait five seconds.

Now, the mechanism is quite important. If you just say, OK I'm going to wait five seconds, you'll go, any questions. Oh, done, five seconds is up. And you'll think five seconds is up in about one second. And why is that? Well, you're up in front of a whole bunch of people, and the adrenaline will be flowing. And adrenaline speeds up your time sense.

All those stories about people who-- like mothers who saw their kid about to get hit by the car, they race out and grab the kid and the car keeps going. Maybe it screeches to a stop. They've managed to rescue their kid. Well, they could act much faster because adrenaline makes the time sense much sharper. So your idea of five seconds will be more like one second.

There's another reason, which is there's this mistaken idea that if you're not talking, you're not teaching. So you're being paid to teach. And oh my god, you're not saying anything. That means you must not know something. So you feel insecure, and that increases the adrenaline and only speeds up the time problem more.

So to wait five seconds, you actually have to count it -- one 1,000, two 1,000, three 1,000, four 1,000, five 1,000. And I actually sometimes just do it behind my back with my hand-- one 1,000, two 1,000, three 1,000, four 1,000, five 1,000. And it's not like you have to be watching the audience for the whole five seconds. It's actually better to wait the whole five seconds, and then choose one of the hands that has gone up rather than the first hand that goes up at two seconds. You say, oh, thank god, somebody asked a question. OK, now my waiting is over. Well, again, you'll be back in the usual suspect soup.

So, first time scale for bringing in questions, in other words, spreading the engagement through the class. Because when people are asking questions, they're by definition engaged. They're struggling with the material. They're trying to make it their own. Wait five seconds.

OK, the next method. So the next method is short time scale, one minute. And that is to use a feedback sheet at the end of every teaching session. So you actually have an example of a feedback sheet that we'll ask you to fill out at the end of this session, which we handed out at the beginning. And the blank versions of that sheet will also be on the TLL website, so you can actually download it, print it, and use it in your own teaching, as well as my notes on how to use it.

But the main point is that at the end of the class, you give the students one or two minutes to fill out a feedback sheet. And what I've converged on is the following three questions on my feedback sheet, which is, what was most confusing? Basically, what question you have that still wasn't answered. Second question is, what was the most useful or most useless example or demonstration or teaching method that happened today? And the third question is, any other comments? So now let me explain why I use those three questions.

The first question, these sheets are all anonymous. And it's a chance for someone, if they felt a little bit shy asking questions in front of people-- so here in the tutorial, one of the characteristics that was mentioned was that you don't feel nervous about asking a question, because there's no one there, or maybe there's one other person there. Whereas in a class of 30 or 80 or 20, you may feel a bit nervous. Well, if you feel a bit nervous, you just put your question on the feedback sheet.

And so then at the beginning of the next session, I answer all the questions on the feedback sheets. So it takes me more than one minute, but actually not that much longer. Because you could flip through 100 sheets in about 10 minutes. You just glance through. You'll see a bunch of questions are repeated. And you'll say, oh yeah, I do really need to explain that.

And at the beginning in the next session, just explain all the important points. And that has another benefit, which is that it links back to the previous lecture, or the previous teaching session. So now you've made a thread between the previous session and the current one, and you got everyone up to speed into the space where you want them to be. So that's why I use the first question.

The second question is, what worked? What didn't work? And I also had to add, and tell me whether it worked or didn't work. Because I used to get examples saying, that example. And I didn't know whether they loved it or hated it. So then I said, please tell me the sign of the effect as well.

And then I would know. I would start to build a model of how students think and what works for them. So every time, basically, I learned that I brought in a demonstration, there was universal love on the second question. So I said, OK, that really connects with students. And I would make sure to do more of that. So it was reinforcement for me and practice for me at being a better teacher.

And what you'll find is that as you use the sheet more and more, you get better and better at predicting how students think. So for example, you're doing some discussion on the board, some examples on derivation. And the first time you do it, maybe you don't know it's not going to go well. And then you read the sheet and you see, oh my god, I was a disaster. 30 questions out of 80 all said, I didn't understand x.

So now you understand that students didn't understand the way you did it. Well, now you've built a better model of how students think. So next time you're doing a derivation, what you'll find is-- at least what I've found is in the middle of the derivation, I'll realize, oh, this is not going well. I'm going to get a lot of questions about this on the sheet. Oops, but it's to late. I'm in the middle of the soup. I can't un-drink the soup. So I just finish the soup and just wait for the questions.

But then after maybe another week of using the sheet, you actually have quite a good model of the students. And now before you start the derivation or when you're planning the lecture, you realize, oh yeah, that's not going to go well. Let me do it the other way, because I know that works better. So you'll find your model of the students developing more and more refinements and more and more accuracy. In other words, you're becoming a more expert teacher.

And then the third question on the sheet is, any other comments? The reason for that is, it's just a free-form space. And I've actually had students say, oh, actually, when you explained that thing, you actually explained it wrong. I was like, oh that's good to know.

So then in the beginning of the-- because we did it properly in this other class, and here's the exact reference that you maybe didn't know about. I was like, oh great. So at the beginning of the next class, I say, hey, one of you guys pointed out a mistake I made, and here is the corrected version.

And people might not be willing to say that personally, because they say, oh my god, I don't want to correct the professor and make them feel stupid. But it's much easier to do it on an anonymous sheet. So I get really useful information on that. And sometimes you can figure out who did it, so you could send an email saying, oh thanks, that was useful. All right.

[LAUGHTER]

PROFESSOR: And it is. So you may think, oh no, you don't want to be corrected. But actually, after you tell the students the following enough, so it's again back to this. In a group, students feel like, oh my god, I don't want to make a mistake. So I give them the following speech quite often.

Because if you don't make mistakes-- I tell them, look, how much did you know when you were born? Not a hell of a lot. You couldn't talk. You couldn't hardly walk. Well, how do you learn stuff? Did you just all the sudden start walking and talking? No. You mangled your grammar. You put "-ed" at the end of every past participle, even though some don't have that. And eventually you sorted it all out and you stood up straight and you walked.

So basically, how did you learn? By making mistakes. So the only way to learn is to make mistakes. So if you don't make any mistakes, you will learn-- you'll know as much at the start of your class as at the end of the class. So you want to make mistakes.

So I tell them that a bunch of times. But if then I'm really embarrassed about making mistakes, then I've defeated the whole purpose of telling students that. So I actually thank them whenever they point out something that I didn't do right.

OK, so that's the second time scale. Now, the third time scale is not that much longer. It's about, let's say, four or five minutes. And it's a question which I'll call a conceptual multiple choice question. And I'll give you an example.

OK, so here is the question that I'll use to illustrate it. Here is a pyramid with a square base. And it's B by B. The length of one side of the base is B. And the perpendicular height of the pyramid is H. OK, so what's the volume?

OK, so does everyone understand the question? OK. Give it a try. Find your group or one or two. Any questions-- any confusions about the question? Don't give me any answers just yet, but any confusions about the question? OK, so find your groups of one or two other people, and we'll discuss that question in a couple minutes.

Anyone want to say anything about this? Is there a section I haven't heard from? Yeah, go ahead. Can you just say your name?

AUDIENCE: Brian.

PROFESSOR: Brian, OK. So you like D but not the others? OK, can you tell me, for example, why you don't like A?

AUDIENCE: Well, because all of them just have the wrong proportionality. If B increases, then the square is going to increase in area as B squared--

PROFESSOR: Ah, so you would like that to B squared.

AUDIENCE: Right.

PROFESSOR: Whereas it's just a single B and a single H. OK, so you don't like that because the proportionality is wrong. It doesn't scale correctly. So let me put here, bad scaling. What was another problem?

AUDIENCE: It also has the wrong units.

PROFESSOR: It has bogus units. OK, so it has bogus dimensions.

AUDIENCE: [INAUDIBLE] dimensions.

PROFESSOR: No, they're pretty much interchangeable. I use units to mean meters or feet or something, and dimensions to be things like length. But they're pretty much interchangeable in usage. So bad scaling, bad dimensions. OK, anyone have any comments about B?

AUDIENCE: Same.

PROFESSOR: Same problem. In fact, it's even more horrendous. Well, it starts out OK, I mean, at least as far as dimensions are concerned. But then what does it do? This has the wrong dimensions. That has the right dimensions, and then you add them. That's terrible. OK, so the dimensions are just bogus. Ah, OK. Can anyone say anything about choice C? Yeah?

AUDIENCE: If you look at the extremes, like if you make H very small--

PROFESSOR: So if you stop on this pyramid and step on it until H goes to 0, what happens to the volume here? It goes to infinity. Yeah, it's terrible. So the extreme cases are just bogus. So it fails that test. OK, now we're back to D. So who likes D? So we have a few people who like D.

[STUDENT CALLS OUT]

[LAUGHTER]

PROFESSOR: OK, go ahead. You don't like D.

AUDIENCE: No, I don't like D.

PROFESSOR: OK, why not?

[LAUGHTER]

AUDIENCE: It's the volume of the box, which has a size [INAUDIBLE].

PROFESSOR: So this is the volume of a box, in other words, if I had vertical sides. So let me draw vertical sides here. Right, so whereas the pyramid has actually got to be less. Now, it's kind of mysterious what this factor here should be, but it's definitely less than 1.

So this is comparison with a box. So it has a good properties, and it's a good answer. It's just not correct. But that's OK, because it's important to realize what's good about it and what's not good about it.

So it satisfies the extreme cases. It satisfies dimensions. It scales correctly. Like you pointed out, you want B squared, because it's the area down there. So it's good. It's not all good. Let's call it good mostly. But as they say, a miss is as good as a mile except in horseshoes and atom bombs. So good mostly, but box comparison.

OK, so let me cross this guy out and this guy out and that guy out. So what do you conclude from that? They're all wrong. So here's another multiple choice question.

[LAUGHTER]

PROFESSOR: OK, so who votes for A? One of these is definitely correct.

[LAUGHTER]

PROFESSOR: No votes for A? Who votes for B? Oh, I'm hurt. I'm hurt. Well, B is in fact correct.

[LAUGHTER]

PROFESSOR: So every once in a while, I throw the students a curve ball and make all the choices wrong. Now, why would I do that, such an evil, dastardly thing? Well, it's good every once in a while to do that. In fact, I found that out experimentally, because the first time I used this question with the students, I was already using the feedback sheets.

And on the feedback sheets, I got back tons of love on the comment sheets. The students said, oh, we love that, having all the answers wrong. Don't do it too often, but do it every once in a while, maybe once a month or so. And that way, they never know when it's coming, so they always have to check all the answers. They can't just do everything by elimination.

So it is a very useful thing to do. There are other reasons for it too, and if you're curious, ask me in the questions, and I'll explain more reasons to do it. But that's already a sufficient reason.

OK, so that's method three. And you see you get a bit of discussion. You get multiple choices. And what does that allow you to do right away? Well, what was one of the problems? Is, you can find out what misconception students have early.

Point one up there, in a tutorial, you can ask students questions. Well, why not ask some questions in a big lecture too? So that's what we just did. So you can find out. If 90% of the students get it like that, you know you don't have to say much more about the topic.

But if the answers are distributed at the monkey line, 25% for all the answers, then you know there's going to be trouble. And it's worth actually expanding on this point and the theme that question gets across right then in lecture. So you could make it much more tailored to the group of students you have.

And the students also-- what happens in their small groups? Right, so remember we mentioned that in a tutorial, that it's a safe setting for raising questions, whereas a big group might not be. Well, what have we done by letting everyone work in groups of two or three? We've created a whole bunch of safe settings for questions.

So the students are free to discuss in their groups of two or three. And if they can't resolve it there, well then they feel pretty confident that it's worthwhile sharing it with the whole class. Maybe the whole class can learn something from that as well.

And that has another benefit, which is that it gets around the gender imbalance. So to illustrate this problem, I was once in a bookstore in New York. And I was standing in the computer section and this couple walked up the stairs. And the woman asked the man, oh, there's some big fat book on Java. And she said, what's Java? And within a millisecond, not batting an eyelid, he said, oh, it's the fastest search engine on the internet, which is completely rubbish.

[LAUGHTER]

PROFESSOR: It's a programming language. So it has absolutely nothing to do with a search engine. I'm sure no search engine is even written in Java. So it has nothing to do with search engines. So it's a complete lie. But he said it so fast.

So there's a dichotomy there, which is-- generally speaking, it's called actually male answer syndrome. And the women--

[LAUGHTER]

PROFESSOR: I see all the women laughing, or I hear all the women laughing. Well, what you'll find is generally, men are very willing to talk about stuff they know nothing about, speaking crudely in extreme caricatures, whereas women are reluctant to talk about stuff they even know about.

So if you just ask a question to a group just flat out, even with the five-second wait, you're more likely to get responses from the guys than from the girls. So the way to try to mitigate that imbalance is to give people, first, a safe setting for asking questions. And then people actually calibrate to the correct level about, well, is this a reasonable idea? Is this not a reasonable idea? Could the people around me have good ideas about it? Is it resolved now? If

Not, well, it's probably safe to raise in a bigger setting. So you'll actually fix the gender imbalance to a large extent, and balance it out. So it has lots of benefits that you get from tutorial teaching.

So now I'll show you the last one, which we won't actually do for half an hour. But I want to sketch out how you could do a question for half an hour. I'll show you the kind of question that works in that time frame.

So this could be multiple choice. Doesn't have to be, but some kind of conceptual question with lots of discussion. And often, it may have a demonstration. So let me show you that particular question. So I'm going to use-- I have these guys too, but I also have the wood blocks which I like so much.

So here are two wood blocks cut for me by the machine shop of the Cambridge physics department when I taught there. So they cut them out of the same piece of pine, same grain, direction, everything. The only difference is this is 2 centimeters thick, and that's 1 centimeter thick. But They're both a foot long and 5 centimeters wide.

So what I'm going to do is I'm going to tap this block, and you'll hear a note.

[TAPS A NOTE ON WOOD BLOCK]

PROFESSOR: Did everyone hear the note? Great. OK, so now the question is, when I tap this block, is the note going to be higher, the same frequency, or lower? OK, so does everyone understand the question? OK, so discuss with your neighbors. We'll take a vote in about a couple minutes. I won't give you the 10 minutes I would normally give people to discuss and argue. We'll take a couple reasons, then we'll do the experiment.

OK, time to vote. So, just to sketch out how this could actually turn into a 30-minute question, normally I give students about a couple minutes to think on their own, then take a vote, and then set everyone to argue with each other. I say, in particular, find someone who has a different answer than you, and try to convince them of your answer. So then you get a big argument going.

And so then I let that go for about 10 minutes. And then we take another vote. And then we discuss. So let's just imagine we've skipped forward to the end of the group discussion, and we're going to take the second vote.

So who thinks the thick block will have a higher frequency than the thin block? About maybe 20 people. Who thinks they'll be equal? About 25. Who thinks the thick block will have a lower frequency? About 50.

OK, so we have a good spread of opinion there. So now, again, in the scenario of the 30-minute question, I would ask for reasons, just like I did here, and I would just be the scribe. I would just write down people's reasons for A, B, or C, for greater, equal, or less than.

Just make sure I understand the reason and then write it down. I don't have to agree with the reason. Just make sure I understand it, and the person who says it says, yeah, that's what I'm trying to say. Write it on the board.

Then give people the opportunity to say a reason for any of the others or against any of them, or comment on any reason already on the board. And that can go for another 10 minutes or so, sometimes even longer. And then I ask people, OK, the test of science is experiment. Should we do the experiment? Should we do the experiment?

AUDIENCE: Yes.

PROFESSOR: OK. So just to remind you of the earlier tone.

[TAPS A NOTE ON WOOD BLOCK]

[TAPS A HIGHER NOTE ON WOOD BLOCK]

[LAUGHTER]

PROFESSOR: So the ayes have it. OK, so it turns out to be roughly almost a factor of two higher in frequency, almost a full octave. It's not quite. If they were infinitely thin, and this were twice infinitely thin, it would be exactly a factor of two. But because they have a finite thickness, it's not quite true.

But that's how xylophones work. In fact, many people have reasoned their way to the correct answer by thinking about xylophones. And many people have reasoned their way to the incorrect answer by thinking about guitar strings, which brings up a very interesting discussion you can have.

So what did you notice about here? When I was about to tap the second one, could you hear a pin drop?

[LAUGHTER]

PROFESSOR: Right. Student engagement is much higher, because people are involved. But the act of voting commits people to a public view. And they want to know, is this right or is this not right? And the act of voting has another benefit, which is that you get a much better error signal.

Whereas, suppose you didn't vote. And suppose you thought, well, it could be higher because it's stiffer. On the other hand, it's bigger wavelengths, so it's lower. And you sort of both reasons in your head. You're in a superposition state. And then the experiment happens-- bing bing bing-- and it's higher, and you say, oh yeah, I knew that. It was higher because it's thicker and stiffer.

So you think actually you understood everything, but you didn't actually figure out, well, which of these two reasons is right and why? Whereas if you have to publicly commit to one answer, then you get a clearly defined signal back about whether that answer was right or not. So it's very important to make sure people vote. And at the same time, it's very important to make sure people are not punished for voting.

So does it matter in my classes whether people get the vote right or not? I just want them to vote. And so then I have to give my speech about, it's better to be wrong than right. Question?

AUDIENCE: So I've been in lots of classes where the professor asks a question and waits a few seconds for people to raise their hands. And by the time he's done or she's done, there's a third of the class raising their hand, and those the people who had their hands up first. So how do you actually take that step?

PROFESSOR: Sorry, could you tell me what they're doing? They ask a question.

AUDIENCE: They ask a question--

PROFESSOR: Do they let the whole class discuss it?

AUDIENCE: Not necessarily. So that's what you're saying is key here?

PROFESSOR: Yeah, it's really key to let the whole class discuss-- so the normal structure for a long question like this one I'll do is I'll make sure everyone understands the question. And then I'll let people think by themselves for a minute or two, and take a vote after that, a straw poll, I call it, so people don't feel it's too weighty.

The reason for that is that people have different learning styles. And some people want to think longer for themself, and some people want to just start talking right away, so I want to give something for everybody. And then after I take that vote, that gives me an idea where the class is, and it gives the class an idea of where they are, and who has different views. And then they can start arguing with each other.

And so now they've had a lot of time to actually check what they all think. So people will have had arguments in their group that they can't resolve, so they're very eager to actually share the argument. Whereas if you just ask the question without giving people time to struggle with it, you're basically wasting the effect of the question.

AUDIENCE: So how do you decide what level of difficulty for discussion?

PROFESSOR: This is a hard question. But it's so rich. It has so many things. I don't expect most people to get the question right. In fact, when I ask this to faculty in physics department, when I give physics teaching seminars, most people get it wrong.

I won't say all the places where that's been true, but it's been at many universities around the world. So I don't expect people to get it right, and I don't care. The reason for it is that it introduces so many ideas-- about, for example, spring models of solids. We can use musical instruments as analogies-- that I think it's worth spending even a whole class on it.

And in fact, one time somebody said, well, but a xylophone, all the things are the same thickness and they're just longer. How does that work? And I said, oh, I'm glad you asked that. And I pulled out my daughter's xylophone-- she has a baby xylophone-- from my backpack. And so we analyzed the xylophone right there. So it leads to so many interesting things that I'm willing to spend a whole class on it.

So that's how I decide how much time to allocate to something. If it's a really core idea, spend more time on it. And if it's peripheral, tend to spend less time on it or do a really short question on it, if at all.

And then the difficulty-- I like to get questions, ideally-- I mean, this one's different just because it's so rich-- but ideally where students are sort of at 60% correct thinking by themself, maybe 65%, even 70%. Because even though they may be right, that sounds like oh, they're mostly fine, but they may not know why they're right. So then in the discussion they can solidify their reasons and maybe get up to 80% or 90%.

Whereas if you've got a question where they're 30% correct to start with, and that doesn't have other redeeming qualities like the richness of it, you'll just propagate noise throughout the system, and just basically wrong ideas will just spread all over and you'll spend all your time correcting those. So I gauge the difficulty based on trying to hit that mark.

OK, so hopefully you've seen, first of all, that tutorial teaching-- the main points from today-- is much more effective than regular teaching as normally practiced because of these characteristics-- generally questioning, feedback, instant feedback and corrective feedback right away. But through a series of methods, you can bring many of the benefits of tutorial teaching to your own large group teaching, which is the kind of teaching that most of us will do, just for cost reasons. So I wish you best of luck in all your teaching.

AUDIENCE: Can I just [? get ?] one interesting statement?

PROFESSOR: Sure.

AUDIENCE: It's something to think about, and faculty often have difficulty with this. There's probably a pretty high correlation between the amount of information that you want to get through to students, and the time that it takes to do it. The more questions you ask, the less you can get through the material. And this constant questioning--

PROFESSOR: This constant battle.

AUDIENCE: What should I do? Should I just try to get this much information because I want to ask lots of questions? Or do I want to get that much information through, then I better not spend too much time asking questions? It's a conundrum. I think asking questions is just a phenomenal way of learning, but [INAUDIBLE].

PROFESSOR: Yeah, basically we teach too much stuff. And that's the fundamental problem. And so one solution to that is to say, look, all the details, get it from a book. And in class, we're going to do what you can't easily do from the book, which is struggle, argue, reason, and correct those reasons. That's hard to do from a book. So you should use the class time for something that's value added that you can't do elsewhere.

So good luck with that. Please take a minute to fill out the feedback sheets. And as long as no one kicks us out, I'll be here as long as people have questions.

[? COMPUTER: Answers ?] from lecture seven to questions generated in lecture six.

PROFESSOR: OK, so questions from before. What are your suggestions for someone teaching a course that they are not expert in at all, you know, for the first time? So I'm a practical man. I know most people, when they're teaching a course, teaching is not going to be their life. I mean, it is for me, but that's unusual.

And if you love teaching, that's great. But probably, your job will be dependent on how much research you do or other activities like department committees. And so you don't want to destroy your whole career on your first course. By which I mean, just try to survive your first course. Because what you could do the first time you're teaching it-- so the answer is, you need to somehow become an expert. If you want to teach it differently and creatively and interestingly, you somehow need to become an expert.

Now, you could do that by spending the whole summer before really figuring out the material from scratch. And that probably in the long term is a good thing. You'd probably find useful results in your research from it. But probably you don't have that summer to do that.

And so what you need to do is kill two birds with one stone and become an expert by teaching it, in which case the best plan I would recommend is just to teach it pretty much the regular way, with some interactive changes in lecture. But just follow the regular plan, as much as it sticks in one's craw. Because in doing that, first of all, you will be rocking the boat.

So people will accept it even if it's not great. And you'll become an expert by the end. And then the next time you teach it, you'll be OK. Then you you've done that summer of thinking, but you've done it during the class. So it meant the first class didn't go as well as it could. But it also meant you survived and you could order material for your lab and all the other things that you'd do in the summer.

How do you-- so I like it when you emphasize that it's OK to get the wrong answer. It's like with the wood blocks, I mentioned how, mostly when I ask this of physics professors too, people get it mostly wrong. And so I had another example of that in Colorado, because I gave a talk using the wood blocks.

So how can you incorporate that "it's OK to be wrong, as long as you learn" attitude into class? One of the best ways is to minimize the amount of grading, and the emphasis given to grading. Grading, basically, as a first approximation-- so if I just add two letters, it's hard to avoid that.

So grading, when you mark someone and you tell them, bad, A B, C, D, well, the people who are getting B's, C's, and D's, whatever the quote bad grades are, you are actually shaming people. And in some ways, you are degrading their psyche. So it's not healthy. So that is one of the political barriers to educational change is the importance that we give to grading. And I will talk about that in the final session.

You may think, oh what about the people who get A's? Well, if you're giving A, B, C's, and D's, the people who get A's. They may have struggled and were thinking, oh my god, will I get an A? Will I get an A? Then they get an A. So they go, phew. But still, they were nervous about their learning the whole time instead of enjoying it as much as they could.

And second, there's also studies, many studies, which show that when you reward people for things they already enjoy-- this is totally fascinating-- so when you reward people for things they already enjoy doing, their enjoyment drops. So if you start paying people for things that they would like doing anyway, then they don't like it as much.

And you [INAUDIBLE] everyday examples of this. You know if you know anyone who's a symphony orchestra player, they say, oh yeah, the best way to start hating music is to be a professional musician. They love music, and then they had to do it for pay, in all of a sudden they don't like as much anymore, not all of a sudden, but slowly. So there are many, many studies about that-- motivation.

So what you want to do really is to find intrinsic motivation. And so the difference between extrinsic and intrinsic motivation is fundamental. And what you want to try to do is create the environment always where the motivation is intrinsic, so that the questions are interesting.

So what that means is you don't want to use bad questions that people don't have an interest in, and then force them to be interested because you give them extrinsic motivation. That's a fundamental bad choice, and then it's being solved with the fundamental wrong approach.

What do you need to do is rather than do both of those, you say, OK, let me find interesting questions that people care about. And then the intrinsic motivation carries people through, whether they're right or wrong. So that's why I like questions like the wood block so much, because people are just actually interested. They want to know. They say, oh, that's a good question. I should know that. Oh, maybe I don't. OK, well maybe I'll learn something.

So the other part of that is, if you notice, I didn't use clickers when we did this question. Partly, it's because clickers are expensive and you have a bunch of logistics to do, and then you need probably some ludicrous Windows program to receive the signals, and I haven't written the Linux version for it. So there's all of those issues.

Those are, I would say, not the fundamental issue. The fundamental issue with clickers, why I don't like them, is that they make people think they're being spied on. So even especially if you live in a surveillance society, where actually all your phone calls are being spied on, people might think, well when you have clickers, you're being spied on to know whether you got the right answer or not, and it's going to count against you at some point.

So I'm very happy with people to just raise their hands. And that's, in a way, more anonymous than a clicker that belongs to you. So there are ways of getting around that with clickers. You can to have people trade clickers so they know that the clickers are random. So you can do things like that.

But still, I like the informal feel of, yeah, just raise your hand. Don't worry. It's a safe space in the classroom. And I have to say that a few times. But just raise your hand, and don't worry. And I'm not spying on you. I don't care whether you get the right answer or not.

For example, with the wood blocks, I expect most people to get the wrong answer. That's OK. We'll learn something about it afterwards. So you do have to work at it, but there are things you can do to keep the class [? safe. ?]

There are also things to do, which is that never-- it's hard to remember how much power you have as the instructor, and just as someone in a position of authority. So I might have told you the story from when I was a graduate student, but it's worth telling again, if I haven't, which is that I was giving a talk in a group meeting when I was a graduate student. My adviser got frustrated with me because I wasn't understanding some fundamental point, actually about springs.

So to demonstrate it, he kicked the table at me and said, is this acting like a spring? And I was so shocked. I was just mortified for like two days. I just couldn't believe my adviser had done that in front of everybody else, and thought that I must be idiot and showed it. So then eventually my fellow grad students said, well, actually he does that only when you really cares. So that's better than people just switching off. So I sort of consoled myself with that.

But it did make a real strong impression on me, how much even a small push from someone in authority can do. So it's the same thing in lecture, if when someone says something wrong, if you say, oh, why would you have said that? Even the slightest hint of that, people magnify and will pick up and be very, very sensitive to. So you have to be really, really, really careful not to let that happen.

So one way is the poker face. So I practiced the poker face. So when people are making suggestions, for example, about the wood block, why they would act one way or another, I just make sure I get in a different mental state. The mental state is transcription. So I just want to make sure I understand what people are saying. And then I just write that down.

Now, it's even there you have to be careful. Because you'll understand the correct answer quicker than you will the incorrect answer. So you have to make sure when people are explaining the correct answer, you don't complete it for them. And then in contrast, when people are explaining a wrong answer, you ask lots of questions, because that too is a signal.

So the poker face is tricky to do, but once you master it, that's one good way of giving yourself a pause and not criticizing people for their answers. So there are various ways, but you just have to be careful. And if you do that regularly, you'll find in a week or two, you've overcome the presumption people have that it's dangerous to ask questions. But you do have to work at it.

Is it possible to incorporate mastery learning at the college level? So actually, there is a very interesting experiment on that. And it was done at MIT in the 1970s, I believe. It was called-- so it was called the Keller Plan. I don't know if they did it for all courses, but the physics courses were taught this way. I don't know if only that way, or that was an option.

So it was in the 1970s, which was a time of-- as people who were here then said, it was a very turbulent time. And the turbulence actually enabled lots of educational experiments to happen, basically the '60s and '70s. So if you look at, for example, the Feynman lectures were in Caltech in the '60s, the Berkeley physics course, 1960s, things like the Keller Plan, the free schools at the school level-- tons of experiments happened in the '60s and '70s.

So the Keller Plan was one of them. It was mastery learning. Students worked on problems, and when they were ready for the next piece, they continued. And from what I can tell, it worked pretty well. But then eventually it got killed off by basically faculty who said, no, I'd rather teach and do lectures. Because what's the point of the faculty if the students can learn really well from the books?

So I think the Keller Plan was a really good idea. In school, I learned for five years of high school math in three or four weeks using the Keller Plan, compared to the normal pace in school where you spend a whole year doing trigonometry. So the Keller Plan is a really, really excellent way, if you have reasonable books.

So now, what are the full reasons it died? I'm not really sure. I need to check into that. But people who were around then, some of them said, yeah, just people who wanted to perform in lecture were really sad about the Keller Plan and wanted to go back to the old way. And so when the political currents changed, so did the Keller Plan.

How do I come up with these rich and interesting examples like the wood blocks, and how often do you incorporate demos? Well, we'll talk about that in Walter Lewin's lecture. So if you saw Walter Lewin's lecture, the one on the pendulum, he must have had four or five different demonstrations. They were short, but he had many demonstrations.

And questions like this, if I had 20 of these, I would use 20 different ones. So usually I have four or five of them that are a good long time demo that are suitable for, say, half a lecture or longer. And I use them. So wherever I can find them, I use them. But not every single demo is worth spending 30 to 40 minutes on. So you have to think about whether it teaches core ideas, how it fits with the goals of your course.

So for example, let me talk through why I use this one. Well, one of the goals that I want is I want students to be able to make physical models themself. I don't want them to just be able to solve equations that I've put in front of them.

So normally what you do is you'd teach them the bending beam equation, and then maybe you'd show them this, and then you'd do this in two or three minutes. But actually a much more interesting question is not, what's the solution to the bending beam equation when the beam's twice as thick. The question is, what physical model is responsible for making sound? When you hear sound, where is it coming from? When you tap this guy-- [TAPS A NOTE ON WOOD BLOCK]-- why is it making sound, long before you're even discussing the bending beam equation.

So that question is, to my mind, much more interesting. And so I'm very willing to spend more time on that, and having students develop that feel, and have them argue and realize that there's different physical models. It could be a standing wave in the block. It could be that the block is bending. How do you decide, and how do you decide which one you're going to hear more? And then for each one, how do you figure out how the frequency's going to change? So that, because it fits so well with the course goals, spend more time on it.

Why is the stiffness a factor of eight bigger? So that one, anyone who wants to know, I will explain after lecture. Just come to office hours. So there's a simple picture using wood atoms. Don't ask what wood atoms are.

How can you incorporate aspects of mastery learning into regular teaching? Ah, so that's what interactive teaching is all about. So the reason mastery learning works so well is that you're not building castles of knowledge on sand, roughly speaking.

So normally what happens-- so the extreme case is Cambridge in England. So the teaching system there is you have an eight-week term, and then you have a six-week break, and then an eight-week term, six-week break, eight-week term, and then 16-week break. So the eight-week term, about what we would do in a semester is compressed into eight weeks, just basically straight dictation and lecture. And the students have a hard time really understanding everything at that pace, so they just take dictation. And then you're supposed to figure everything else out, integrate it in the six-week break.

So the problem with that is that it's very inefficient. It's very inefficient because you haven't understood A. And now B uses A, but A hasn't really been understood. So you're not going to understand B. So now you have A and B not really understood, so you just write them down. Now C comes along using A and B, but A and B aren't understood, so the problem compounds.

Whereas with mastery learning, only when you understand A do you go onto B. And also when A and B are solid, then you go onto C. So your knowledge castles are robust and strong. And that's not true with most teaching.

So how do you remedy that problem? Well one way is you ask questions in lecture. So here, plan interactivity and build it into your lecture.

So by asking students questions you figure out, oh, if only 20% or 30% or maybe even 40% of the students knew the correct answer, and even after discussion, you think, oh, OK, well this is a fundamental concept. Presumably that's why you asked the question about it. But it wasn't mastered. So better fix it, before you go on. So that builds mastery learning into the regular lecturing process. And that's one reason that interactive teaching is so effective.

So one of the other questions later in the sheets is, is there evidence that interactive teaching is good and helps people learn? There is actually good evidence. Maybe I should put some of those papers online.

One is by Richard Hake. He's now retired. He's physics professor from Indiana University. So what he did is he did a sample of physics courses across the country, intro physics courses that included a total of 6,000 students. And he asked the instructors to describe how they did everything. And then everything was grouped into either interactive teaching or regular teaching.

And just the scores that students got on one particular exam, which was a force concept inventory, before and after-- they were given the force concept inventory before the class started and at the end-- they saw how much the students improved. OK, so if you improved 100%-- in other words, suppose you started at 20% correct and you got to 100%-- that was considered a 1.0 gain. If you went from 20% percent to 60%, that mean you got halfway to the maximum possible score you could have gotten, so half of the maximum improvement, that was a gain of 0.5. So he measured the gain for everybody.

So this is the gain on the force concept inventory. And this is the number of courses in each area. So it was roughly like that. So this was the regular.

So the regular teaching, by the way, included fantastic teachers, people who were rated highest lecturer in the university and had won lecturing awards. And the interactive teaching was something like this. I'll put the paper online so you can see the exact graph.

But basically, the worst interactive teaching was better than most of the regular teaching. And the regular teaching hardly got better than 0.2 or 0.3. So if you wanted any more gain than that, you really needed to do interactive methods. And I think that is because you've incorporated mastery learning into the curriculum.

OK, what do you do when the students are really quite and won't answer your questions? Or flip side is, what if only the quote 5% usual suspects are the ones answering the questions? What do you do? Well, so that is related to the question of, are there bad ways of incorporating interactivity? And there are. The worst is the following, which is-- I'll sort of parody it.

So we're doing relativity, or nuclear physics. E equals MC--

[LAUGHTER]

PROFESSOR: Squared, yeah. OK. So you'll find a lot of people who normally wouldn't teach interactively are now, they're told or they hear that it's good to teach interactively and ask students questions. But if you don't have any practice in how to do that, the easiest questions to ask are just things like this.

And the problem with that is not that they're asking a question. It's good to ask a question. But the problem is that this thing, this two, is just a memorized fact. So it's possible to answer it just by memory, whereas if there was actually some debate about what it should be, you could ask the students the question. But now, if you just ask people question like that, and you give people no time to think, yes, you will get answers just from the quote top 5%, the usual suspects.

So to avoid that, what you need to do is give people a chance to think on their own. So that's one of the interactive teaching techniques, the idea of give people a minute or even 30 seconds to think on their own, so that you'll get answers from a wider spectrum of people, because they've checked it out with their neighbor. And they feel safer in volunteering and answer or asking a question about it.

So that's how you avoid two things, two problems at once. You avoid the quiet students, because you've given-- those students are being quiet because you've given people a chance to think. And you also avoid the problem of just the usual people answering, because you've given everybody a chance to think.

OK, if we don't really understand until we teach, should we have students teach each other in presentations, or is that not worth the time? My feeling is that huge big presentations and having students present to each other is not so much worth the time. I mean, it's worth the time of the one person who's doing the presentation. But that's the main person who learns in that teaching session.

So there are several graduate seminars, many I took as a graduate student, that are run the following way. The professor just sits back and each time-- they assign a list of topics, and each student volunteers for one topic to present to everybody else. Now, that is a fantastic way to learn one topic in the entire course. And it does work for that. And part of my Ph.D. Came out of one of the topics that I learned from one of those courses, because I really wanted to figure it out.

So it can work. And it does work in some ways. But it's not a very efficient way, because all the other people are just basically passive. Yeah, they read the stuff. But now they're being taught by someone who's just barely mastering the field and is not likely to teach interactively with a large margin of safety. So it's much better to have students teach each other by having them argue with each other on small time scale.

So for example, if you remember in the wood blocks, I said, OK, are you with your neighbor about your answer? Well, now the students are trying to teach each other. Some students think, oh yeah, it's due to the bending. OK, well that turns out to be correct. But can you convince your neighbor?

Well, in trying to convince your neighbor, you are doing teaching. You're trying to figure out, what's a way that you can do it without writing down 50 equations. That's exactly what you want to do in teaching. So I think it's a much better way, is to build the interactivity into the class for everybody, rather than having it as a rotating seminar where just one person at a time learns.

In organizing big ideas into a tree, there are often multiple ways-- for example, you can do current understanding versus history, or experiment versus theory-- and it's often good for students to know more than one of these organizations. How do you do that in a course design?

Yeah, so unfortunately there's only one time dimension. There's three space dimensions, but only one time dimension. So the course has to evolve in one dimension somehow. So you need at least one organization. But you're right. You need other organizations as well.

So the way I square that circle is as follows. So let's say time goes that way. So this is unit one. Let's say it's one technique, one mode of reasoning, dimensional analysis. Let's say this is another mode of reasoning, conservation, and so on. So these are units.

And now you have topics. Well, what I like to do is I want to show other organizations. And why is that good? Well, what you're doing is you're connecting. You're building threads across. And you're right. You definitely want to do that.

So what I'll do is I'll find problems that can be solved multiple ways. Suppose this is, for example, a problem about a pendulum. Well, you can solve a pendulum with dimensional analysis. You can also solve a pendulum with conservation laws. So I'll do it again. So this way, this method is sufficient to solve it mostly. This method is sufficient, but now we're revisiting the old problem. So I try to find problems that thread through, and maybe then reappear over here. So that gives you a cross cutting organization.

So you can do that not just with problems, but with, for example, other principles. For example, the history-- suppose you want to incorporate history versus current understanding. Well, every time you mention history and the importance of history, you say, oh look, there's yet another example where we learned from history. We learned it here. We've learned it in this unit. So now the students have yet another structure to bind the course together.

This is hard to do the first time you teach the course. What you'll do is this first time you teach your course, well, you'll probably do it the regular way. The second time, you'll maybe reorganize it this way. And the third time you do it, you start building all the bridges across it. Because that's the only time you actually see them too.

What are my thoughts on grade inflation? I wish every grade was an A. So I think there isn't enough of it, because of the studies I said before. So I'll talk about that in the penultimate lecture.

I think most of these, yeah, I think I've now answered with other comments or questions that are there, except for one, which I'll save for next time.