Recitation 2

Flash and JavaScript are required for this feature.

Download the video from iTunes U or the Internet Archive.

Instructor: Prof. Gilbert Strang

The following content is provided under a Creative Commons license. Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free. To make a donation, or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu.

PROFESSOR STRANG: So this is the first true review session in 18.085. The last Wednesday, the first Wednesday afternoon, was a brief review of topics in linear algebra. But now we're into the course. We've done four lectures on the first four sections of the textbook and one homework problem in and back and a second homework set for next Monday. So I'm ready for any questions, including questions that are on the homework if necessary. But anything at all. Or maybe I'll just ask whether the pace, so this is really informal. Is the pace of the course, now today's lecture had a lot in it as I realized when I saw that the board was still full of 18.085 and there was a little more still to do because we didn't finish the matrix part. But are you okay with the sort of speed of the course? So that'll be one question. And then, what about specifics? Somebody start off if you will. Anybody. Yes, thanks.

AUDIENCE: Well, actually I had a question about the lecture earlier today.

PROFESSOR STRANG: Okay, go ahead.

AUDIENCE: I was just going to look it up in the text, but.

PROFESSOR STRANG: That's alright.

AUDIENCE: But haven't had a chance to. But, okay, so I'm not really sure how you take the initial conditions and apply them to the ramp function to actually get a solution.

PROFESSOR STRANG: So that's what luckily happens to be still here on the board, right? We've got these boundary-- I would call them boundary conditions. So this is definitely, we're in a part of math that's about boundary value problems more than-- time isn't in the picture. I mean later time will get into the picture. So with this particular example, the general solution is a standard ramp at the point a where things are happening. Plus the C+Dx, the usual. So that's a particular solution. That particular solution has the right behavior at the impulse. By the right behavior, I mean that the second derivative of the ramp is a delta and when I put the ramp at a, then the delta will show up at a. And when I put on that minus sign it'll mean that the second derivative is minus the delta. So the slope will step down. So that's a particular solution. But that by itself, what does that equal at zero? And what does that equal at one if you remember the ramp?

So let me just draw that ramp again. So the ramp was really based on, centered at the point a. And I'll put it with a minus sign. So it came along there and down there. And now suppose this is one end of our interval and that's the other end. So is that ramp the answer to our problem? No. Well it happens to satisfy this boundary condition, happens to start out at zero. But it doesn't end up at zero. So just like every particular solution we need a little more. We have to include more solutions because that was only one. All those are equally good solutions. If I add C+Dx, the second derivative of that is zero, so it doesn't spoil anything at all. On the contrary, it adds more solutions. I mean the great thing we're using here is that our equations are linear and when zero's on the right-hand side-- notice there's no arbitrary constant. I'm not putting an arbitrary constant on that particular solution. Is one particular solution plus a subspace if I use that language. A lot of solutions to the-- All the solutions to the problem with zero on the right-hand side. And these are the solutions, these are the null space guys, the ones that have zero on the right-hand side.

Now we need those. And the effect of those will be to move that ramp. And effectively, what are they going to do? They're going to swing that ramp around, it'll stay a ramp. But instead of being level here, it'll go up. And instead of going down there, it'll go down there. It'll be the same ramp, with still that slope dropped by one. Slope went down by minus one and that's not going to change here. The slope will still go down by minus one. But now I've just adjusted it so that it goes through the-- it satisfies the boundary conditions. And in the second problem with the free end, again, here's my ramp. But now I'm going to adjust it. And what happened? It just needed adjustment upwards. Because this was the zero, this was the u=0 fixed guy. And now if I'm doing u'(0)=0, the free guy, I can lift the whole thing up. So you see, I just lifted it up to the point where it came out right there. So in this case I just needed a C. And in this case I just needed a Dx. And in other cases I might have needed both. A little bit of C and a little of Dx.

Anyway that's what was-- maybe that's sort of repeating what we did today a little bit. But mechanically it's just, we've got a particular solution and we've got the complete solution and then we just have to choose C and D and we've got two conditions to do it with, the two boundary conditions. Could have other boundary conditions. Now, what about, here's a, yeah. I guess what will often happen in these review sessions is I get on a roll and I just keep, carry on with it. You know, you start me with a question and I can't stop. So I'll go a little longer but then I really will stop, ready for the next one. So we haven't discussed the free-free. u(0) equal-- u', sorry, free is u'. That's free at the left and free at the right. What's up with that? What's the solution to that? Again, I'm looking for -u'' equal an impulse. With those two boundary conditions, free-free. And do you know what's going to happen? No solution. Now it'd be interesting to see why not. Why no solution? Well one way to do it is try. Other right-hand sides could have a solution. So it's not just that this is-- something's the matter here. Specifically, that right-hand side and most right-hand sides will fail.

But let's just see it with this one. Why does that fail? Well you can see I can't, the slope here is zero and the slope here is minus one. If I adjust those I can't get, this is asking me to get slope zero at both ends. I can't do that, right? Yeah, just not possible for me to-- This will add a straight line but it's the same straight-- I can't get a ramp that comes flat at both ends because there's-- once I say what it's doing at the ends, I've got it. And you see what I mean? I'm looking for one that starts flat and ends flat and that's not in my family of solutions. That problem just doesn't have a solution. So that's one way to do it, is look at the solutions and realize you can't satisfy both boundary conditions.

Another way might be this. This is a little bit deeper way and it leads to something better. Suppose I take the integral. So this is an equation that's supposed to hold. Let me integrate both sides from zero to one. So there's the idea I'm putting in now. To go a little further, to discover when this has a solution. Or let me take more generally, -u''=f(x). So some other load. Not necessarily a point load, not necessarily a uniform load, but maybe some other load. And now my boundary conditions, I'm trying to do free-free. And usually no solution. But let's just see why and when there might be a solution. The key idea is integrate from zero to one. What do I get on the right-hand side? Well, I get the integral of f, whatever it is, and I would call that the total load. Fair enough? The total load of-- If it was a delta function, the total load would be one and it would be all in one spot. If it was uniformly over the whole interval, well I guess that would also integrate to one, so the total load would be one spread out. But it could be a mixture of the two, could be a few delta functions, whatever.

What happens when I integrate the left side from zero to one? Can you do that one? The integral from zero to one. There's that dopey minus sign. u''dx. What do I get? Why do I say that's a good idea? If I integrate the second derivative I get the first derivative. The integral of the second derivative will be the first derivative with the minus. So it's minus the first derivative. And what do I do now? I plug in the end points, right? You integrate. I'm integrating zero to one. So I've found the integral. I've put zero to one in there. So what is that? That's minus the derivative at one plus the derivative at zero. And now, what's that? That's zero. By my boundary conditions, that's zero.

So what have I found? I've found that if these are the boundary conditions, then when I integrate the left side it's going to give me zero. So when, what loads could be okay? What loads f(x) could allow me to solve this equation? The condition will be I need, what do I need for the total load to be able to solve this equation? The integral of the left side was zero so the integral of the right side had better be zero. So that's the condition. If I have these boundary conditions, then my problem is singular. Usually no solution. It's like having a singular matrix. It's like having this particular singular matrix, of course. Whoops, not that one. Let me get the plus sign in the right position. That's a plus. -1; -1, 2, -1; -1, 1. Right? This is the discrete version with a zero slope at both ends.

It's our T matrix. No, what matrix is it? B. It's our B matrix, both ends. I'll just come back here and then I'll do the discrete one. So tell me a load that we could handle? A load we could handle. So the integral has to be zero. So suppose my load has a delta function at a. Well that integral is one. So can you fix that, change that load or do something, maybe put on another load to get a total load of zero? What shall I do? Add another guy with a minus sign. In other words, maybe this, a delta function at some other point b. Well, that would do it. I believe we could solve that problem. Even with these bad boundary conditions. We could solve that problem. Because the total load would be, one from that, minus one from that, the total load would be zero. In other words, what would are solution look like? It has to start with zero slope. So it would buzz alone to a and then after b. And what does it have to do here? If I graph the solution to this guy from zero to one it starts with, it's free, so nothing's happening until I get to a. Then what has to happen? Let's see, if I'm graphing u, it'll be ramp. Right? It'll be a ramp, yeah. Because I've two derivatives. And it has to ramp down by one, so it'll ramp whatever it does.

I don't know where it stops. Where does it? Wait a minute. I haven't practiced this. So I start from the other end. The other end is flat. What's up? They gotta meet here. Oh, the other end is flat, but not at zero! Dumb, stupid. Right. Okay. Yes, the other end is flat, right. And it's, oh yeah, look! Oh, wonderful. You see. That slope dropped by one and the slope there increased by one back to zero. Slope was zero. It dropped to minus one because of that load. Now it increased back to zero because of this load with the minus sign and there's a solution. So that's a solvable problem. Well, you say, okay, that was a little surprising to get an answer for a singular problem. And no, it can happen. If we have a total load zero it'll happen.

But, there is still a but, that's not the only answer. That picture is a solution. But not the only one. So what my point is going to be, that when the problem is singular, if there's an answer, you say great. But then something has to go wrong and what goes wrong is too many answers. So tell me some more, what other graphs would draw solutions to this problem. I could shift, I could lift the whole thing. Here I've got a plus C that I haven't used. I could just do the whole thing higher up. Any of these. These would all work. It's like temperature. I don't have an absolute temperature here. All I've got is, I would have to-- It's not determined because there's a plus C that-- the plus C satisfied everything. A plus C, a constant has zero slope, it has zero slope, its second derivative is zero. So it's like, unseen by this equation.

And similarly can I just make the analogy as I always like to do with discrete stuff, so suppose I-- Tell me a right-hand side that we think would probably, is this going to be the same story for this guy? Yes. If I add those, where I integrated there, here I would add and I get zero, zero, zero. So this has to add to zero if there's a solution. So let me put for example, [0, 1, -1]. That would be kind of like our delta function in one direction and our delta function in the other. I believe I can solve that problem. So I'm just carrying, because I always want you to see the discrete one as well as the continuous. Continuous involved this integration. The finite one just involves adding. The left side adds to zero so the right-hand side better add to zero. That right-hand side does. Tell me a solution. Well, let me start out with a seven there. What's the next guy going to be? See, I want seven. Whatever I put there, I better have a seven there, right? Seven, seven, good. Minus seven, plus 14, oh geez. I didn't know this was going to happen. No, I want to get the answer one. What number goes there? Six, is it six? It's six, yeah, good. Minus seven, 14, minus six is that.

And now my claim is that we'll come out right on the third equation. So far I've just matched the first two. Now this one gives minus seven, plus six, that's minus one, good. So there's a solution. And I'll leave this problem alone if you tell me the rest, other solutions. That [7, 7, 6] was a solution to a singular problem with a right-hand side that had total load zero, so it was okay. But now that's a solution, but there are more. Tell me another one. I can shift it, right? I could make it [17, 17, 16]. I could add ten to everything. Right? That's the plus C that I could do over there. That can't change because 17 - 17 is still zero. -17 + 16 will still be minus one. All good.

So actually that just like helps our intuition, and physically my intuition is this. That I've got this bar and nothing's holding it. So if I put a weight on it, nothing to hold it, it'll just, rigid motion will take it out of sight, no good. But if I put another equal weight on it, no it's not a weight. What do I call it? If I lift it at that point, that's the other delta function that's going the other way, then it will sit there. But it would still be in equilibrium if I just moved it up to there or moved it as I like. I don't know if that is kind of a dumb picture. But it's saying what we've said from math. Well, you see where your question led. Yeah, thanks.

No, the integral, it was-- Watch what we integrated. We integrated u''. So that's not the area. We integrated u'' and got its integral was u'. So that just told us that a difference in slopes at the ends, yeah. Good, because our intuition automatically is if we're integrating something, we're finding an area. But here, if it was u , then I'd be finding the area under u, but we integrated the second derivative. Right, good. Now let's change the subject. Yes, please.

Yeah, I guess so. I'll try. Let's see. So my discrete equation was like -u... Yeah, so let's back up to the beginning. We've got this minus sign and we're using a second difference. So second differences have coefficients one, minus two and one. Now I'm reversing the signs because of my minus. So I have -u at some point. Let's take that as the point to the left. Two u's at what I'll think of as the center point, minus u_(i-1). Is the load at that center. That center point is i times delta x. That's where I would be looking. So now I'm using subscript. It's a little bit of practice then to take subscripts, take this way of writing the equation and convert it to a matrix way. It's usually clearer once you see it as a matrix. Now this is happening at all the points. At i=1, let's say, I have -u_0+2u_1-u_2 is f at, agrees with the load at the first mesh point. That's the center, the point h delta x.

And then if I want to back up further, I would have -u_(-1), but that doesn't really exist, plus 2u_0-u_1 should match the load at zero. And so on forward. But now I want to put in the boundary condition. That's what you want me to do, right? Put in this boundary condition. So what am I going to take as boundary condition? It has to be some approximation to u'(0)=0. Maybe I'm never going to get to minus one. Maybe I don't need minus one.

That's right, yeah, exactly. We did. That's what we knew about it. Sending it forward, we knew about forward difference, so I chose to do it. But then I think better of it. I chose to do it because it made the point that we, that at that boundary we were introducing a higher order error, first order error that's going to wreck things. I mean, it's going to spoil the, this is second order accuracy. And, but let me do that first order. So what shall I take? I'm going to approximate that by u-- Shall I take this one as I did in class?

Yes. Ah, plus one, thanks, plus one, right, thank you. Thank you, good. Okeydoke. Alright. This is how we got to that equation. If I now bring in this boundary condition-- I guess I don't have to, let me take your eye off of that guy for the moment, I think. We're getting beautiful music here. Is it coming out of this box or? No. Anyway. So I'm going to use this boundary condition to say, well okay, if u_1 is u_0, I'm going to replace this u_0 by u_1. This is the direct way. I replaced that u_0 by u_1 in that first equation. And then what I have is -u_1 and 2u_1, so that's the one and I have the -u_2. So do you see that that equation, when I put those together into a one, is going to-- If this is u_1, this is u_2, this is u_3 onwards, that first equation is u_1-u_2 and that's what I've got. This is u_1-u_2.

So I did it. I got to that matrix. The matrix is actually quite an important matrix. But from the point of view of accuracy in solving this differential equation, it's not the greatest. It's lost accuracy at that point. But the way to recover it turned out to be just a small adjustment at the boundary, so not a problem. Thanks. That's good. Yes, thanks.

Sorry? When the boundary-- sorry. Two boundary conditions at the same point? That's a good question. So when would we have two? So instead of a boundary condition at zero and a boundary condition at one, you're putting them, is that what you mean? Put both boundary conditions at the end. Okay. So that would be, that would happen, I would think that would be more, it would be very typical in a, let me see if there's some space here, yeah. That would be very typical and we will do it, can I change x to t? Because that's what, if I have some. What does this problem look like? And u(0)=0 and u'(0)=0. Both at the same, at the start equals zero or whatever. So what kind of a problem is that? Now these are, I would say, initial values. Initial values instead of boundary values, I now have initial values. And can I solve it? Yes. So I'm starting at time zero. This is t=0. I'm starting at rest. No velocity and actually no displacement and just going forward in time. So I could solve that differential equation. I'd be interested in the corresponding difference equation. All fine. It's a different category of problem. This is an initial value problem. It's like tracking some mass that's, some satellite. So that's what you're doing in tracking a satellite or a planet or something. Yeah, tracking a planet or a satellite. You're solving equations like this. Forward in time. You know the initial position and you know the forces acting on it. Probably gravity. And you go forward in time. Yes.

What would the matrix be? Good question. What would the matrix look like? So an electrical engineer would call a problem like this, and the kind of matrix that I'm going to write down, I think, would be called causal. That word just popped into my head, so let me mention it. You know, part of science and engineering, a big part of it is learning language, learning words. And you have to learn sort of the math language and the engineering language for whatever you're focusing on. But it's good to also to know a few other languages. Electrical engineering languages of filters and causal and other things that we'll see are important. What would the matrix look like? Here's what I think it would be. I've made this a plus there so I'll have to remember that. I think, so I'm looking at u_0, u_1-- No. Well, u_0 I actually know, so let me start with u_1, u_2, u_3, u_4. What would a typical-- Equal sum, right side, f_1, f_2, f_3, f_4. What do you think, what kind of a matrix am I going to get? Before I put it in there. This is a good question. What's the shape of this matrix? It's going to be triangular. Instead of being symmetric it's going to be triangular. I'm going to find, let's see, a typical value would be, say, u_3 because I've used a plus sign, oops! I can't make myself do it right. 1, -2, 1. That would say u_3-2u_2+u_1 would be the new force. This is the kind of thing we're going to get. One, maybe one something. I don't know what this is. This is up in the boundary, in the initial values. But from now on it'll be below the diagonal. It'll be 1, -2, 1.

Do you see? We're marching. We're marching forward. We start by knowing these and then the equation tells us the next one. Then the equation tells us the next one. That's what initial value problems do. You're told how you begin and you take a step, you take a step, take a step every time. And the new value just needs to know the older values. Do you see the big difference between that and our problems here? Our problem is looking left and right looking for back and forward. Back for one condition, forward for another. We start with one, but we're, it's more of a, it's like a hitting problem. We start forward, marching forward in our problems, but we have to hit the other end correctly. We don't know the slope, we don't know the starting slope, we know what we want to hit. Whereas these problems, we're told how we start and we just follow it in time. So that's the difference here.

Yeah, sure, okay. That's true. So this'll be known. Yeah, that'll be known. Yeah. u_1 will also be known. Yeah, and really, maybe I should have got, let me put even the other known one. So we know this, we know this. So those are sort of not in our, yeah, that shouldn't be in our problem somehow. No, I think, what would we get in the end? You're always looking backwards. That's the point. Lower triangular matrices are always looking, they only look backwards for earlier values and then they give you the current value. So that's why lower triangular matrices are so easy to invert. No problem. If it's lower triangular, you just, like, march forward. And if it's upper triangular, which way do you march? So if you have an upper triangular problem, suppose I gave you the problem, let me make it upper triangular. So x+y+z=7. 2y+3z=12 and z=17. So that's upper triangular. Where do we start in solving that one? From the bottom. From the right-hand end, the bottom. And we march backwards in time.

And what I was saying about A, well L times U, yeah, this is worth seeing. What I was saying about A=LU, it was, you remember that? Those letters? What that was saying was that this matrix that's looking both ways can be written as a product of a matrix L that looks behind for old values and you can go forward with it. And a matrix U, like this one, this upper triangular, 1, 1, 1, zeroes below that diagonal, that you go backward with. Somehow that's appealing. That's like aesthetic to break up a two-way problem into a problem that marches one way and then the other. And of course, that's what elimination aims for, is this problem that it can solve by, the words would be, back substitution. When you've started with your original problem, got to this one, then you just have a back substitution, you go backwards. Oh, so much, I'll mention the Kalman filter. That's a similar process of going forward, that's called prediction. Going backward, that's called smoothing. And so, Kalman had the great idea that he could break these problems that were fundamental in space computations into prediction and smoothing.

Once again, we've got off. Yes? Oh, the beam. Let me help you even more before the question. I said it's better to draw the beam this way. I like the beam better this way. Because the point of the beam problem is loads are acting -- and we'll see this, of course, later -- loads are acting perpendicular to the direction of the beam. That's why the beam bends. So it'll bend a little, right? And that is what leads us, its bending moments and other stuff. If you haven't met beams, well, it'll be great to just have a very, half a lecture about, or maybe a lecture about beams. That gives a fourth order equation that I'll write down again. Fourth derivative equal the load. Now, ready.

Yeah, now here I don't have the negative sign. Because once I've got second derivatives twice, so the second derivative is, in some way, negative. I'll complete that sentence in a second. Somehow the second derivative, which is the guy that has the 1, -2, 1, somehow that's a negative thing. But fourth derivative is second derivative of the second derivative. Yeah, do you want to tell me what the numbers would be? As long as we're wildly looking forward to fourth derivatives, just, it helps. Do you want to guess what will a typical row of the matrix B when I go to finite differences, fourth differences? Probably you've never seen a fourth difference. You may not have seen second differences before. That was a big deal, then, to introduce second differences. Those 1, -2, 1's. That was second differences. Fourth? Yeah, 1, 4, 6, 4, 1 with minus sign. 1, -4, 6, -4, and 1. In some way, I would get that by squaring this guy. So that would be a fourth difference.

Oh, what's the deal with boundary conditions? What are you figuring, on beams, beam problems, for a fourth order equation and a matrix that's stretching out further. What's going to happen at the left-hand boundary? I guess my specific question is, how many boundary conditions do I now need? Four. And the typical is two at each end. That's the balanced way. That's the way that would make this matrix sort of symmetric. So maybe at this end I say it's held at zero and maybe it's just sitting on a log there. Right? That boundary condition I would call simply supported. That boundary condition says that u(0)=0. Because it's sitting there. And but the slope doesn't have to be zero. What does have to be zero there? Yeah, sort of the bending moment. Nobody's here twisting it, right? So the other condition in that picture would be second derivative equal zero.

Maybe my point is that now you see what I said before, that getting the boundary conditions into the problem is often the hardest part. Because I have to replace u(0)=0, that shouldn't be too hard to do. But I have to use this other condition somehow, it's going to screw up the 1, -4, 6, -4, 1. I'll have two boundary rows at the top, two boundary rows at the bottom. I don't want to go further today. But I think maybe just mentioning this gives you the picture of sort of the how things fit together. We would still have some nice constant diagonals in the middle, but now we'll have two boundary rows at each end. So that's something to come. Yes, now back to reality, which is, any questions?

Lower triangular guy, yeah. What do I mean by marching forward? So let's see. I'll replace this. Let's see it better. I would replace this by maybe u, I'll use a different letter, n+1. At u_(n+1)-2u_n+u_(n-1) is some right-hand side if there's a force acting on my thing. So f_n maybe. By marching forward, I just mean that this equation-- that I can go in order. I can start with u_0 and u_1. They come from the boundary conditions. Then this equation will tell me u_2. I use the equation. With n as one. This says u_2, some u_1's, some u_0's, f_1's, all that I know. In other words, once I get started, I'm on a roll. If I have two boundary conditions to get me started, then the equation tells me u_2. And then the next time, u_3-2u_2+u_1, I can find u_3. So I can get those, I can go forever. If you give me enough to start on, two things to start, then I march forward. Whereas in our problems, we've only got one thing to start on and we've got one goal to hit. And that's why we have to solve the whole system together. This is, we can solve it step-by-step. This is way faster of course. To be able to just go forward in time. I'll mention that the topic of initial value problems and finite differences for them, we can't get to that. So we're seeing a little bit here, but it's done properly in 18.086 in the second semester is the initial value problem start part. And that has it's own interesting questions. Somehow we've talked about fourth order equations, initial value problems. But no homework problems. So I'm ready for, or even related. But that's fine with me. Is there a question? Yeah, thanks. Or it doesn't have to be a homework question, another question.

Oh, good question. You mean I should just send the homework out to Natick where MATLAB is. Do you know that MATLAB is just 15 miles away? I almost get there, I live 2/3 of the way there. Yeah, so we could just send the whole thing out there and get it back. That would save a lot of work. I suppose, I'm okay. Why should I say no? Anything MATLAB can do and you can make it do, I'm okay with that. I don't see that you have to do things by hand if you've got a better way. That's okay. And then probably the answer gets printed and you can graph it. So that's fine. So I mean, somehow a course like this has got two parts to it. Applied math has two parts to it. The modeling part, set up the equation, think, what is it you're supposed to do. And then, step two is do it. The numerical part, the computing part. And that's where MATLAB, Python, Fortran, whatever, is going to do a lot of the heavy lifting. Was there another question? So that first homework was certainly very general intentionally. Because I'm hoping you will read the book. The lecture, you'll be able to match the lectures with the book even later on when they separate a little or separate more. You'll see what we're doing. And those, the homework problems, you should look at some of the others just to see. Do I know how to do that? Right. Let's stop here for this first review. I'm sure we'll have more-- questions will build up for the second week.