Lecture 4: Heaps and Heap Sort

• Download this transcript - PDF (English - US)

The following content is provided under a Creative Commons license. Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free. To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu.

PROFESSOR: One of the cutest little data structures that was ever invented is called the heap. And we're going to use the heap as an example implementation of a priority queue. And we'll also use heaps to build a sorting algorithm, called heap sort, that is very, very different from either insertion sort or merge sort. And it has some nice properties that neither insertions sort nor merge sort have.

But what I want to do is get started with motivating the heap data structure, regardless of whether you're interested in sorting or not. So the notion of a priority queue, I think, makes intuitive sense to all of you. It's essentially a structure that implements a set S of elements. And each of these elements is associated with the key. And as you can imagine, a priority queue is something where you queue up for something, you want to buy something, you want to sell something. You have certain priorities assigned to you, and you want to pick the maximum priority or the min priority. You want to be able to delete it from the queue. You want to be able to insert things into this queue. You want to be able to change priorities in the queue.

So all of these operations are interesting operations that should run fast, and for some definition of fast. Obviously we are interested in the asymptotic complexity definition of fast. In that case, we'll be saying does this operation run an order n time, order log n time, et cetera.

So in general, I think for the next few lectures, you're going to see a specification of data structure in terms of the operations that the data structure should perform. And those of you who have taken six double O five, you'll see that it's basically an abstract data type that's associated with these operations. So it's a spec for the abstract data type.

In six double O five, you had really spent a lot of time on asymptotic complexity, or the efficiency of operations on the abstract data type. Here, in double O six, you'll specify this ADT, and specify the set of operations or methods in the ADT. And we'll talk about whether these are order end complexity log end complexity, and compare and contrast different ADTs.

So today's ADT is a heap. And what is the set of operations that we'd like to perform on a priority queue? So we can use that to motivate the development of the heap. And those are, insert s x. So you have a set of elements s, and you want to be able to insert element x into set s. You want to be able to do max of s, which is return the element of s with the largest key. And different from max of s is extract max of x, which not only returns the element with the largest key, but also removes it from s.

So you have a queue, and the person in the queue was serviced, or the element in the queue was serviced, and then removed from the queue. And finally you can imagine changing the priority of a particular element x in the set s. And this priority, there's an associated key as we have up there with each element. And that key is called a k. And increase key s x k would increase the value of x's key to the new value k.

And k could correspond to, it's just called increase. Most of the time, you're increasing the value in maybe a particular application. You could have suddenly a decrease key, and you would have to know what the previous value was. And is just a matter of exactly what operation you want to perform. You could call it update, or increment, whatever you like.

I'm going to spend most of the time here talking about how you maintain a rep invariant of this data structure called the heap, that allows you to do these operations in an efficient way. And we'll talk about what the efficiency is, and we'll try to analyze the efficiency of these algorithms that we put up.

So let's talk about a heap. A heap is an implementation of a priority queue. It's amazingly and array structure, except that you're visualizing this array as a nearly complete binary tree. And what does that mean exactly? Well, the best way to understand that is by looking at an example. We got 10 here, so. 1, 2, 3, 4, 5, 6, 7, 8, 9, 10.

So here's my array of 10 elements. And the elements are 16, 14, 10, 8, 7. So some set of elements that are in random order, clearly not sorted, and I'm looking at the indices, and I'm looking at the elements. I'm going to visualize this as a nearly complete binary tree. Is not a full binary tree, because I only have 10 elements in it, and it would have to have 15 elements to be a complete binary tree. And we want to be able to do the general case of an arbitrary size array, and so that's why we have nearly complete here.

So what does it mean to visualize this as a tree? Well, index one is the root of the tree, and that item is the value is 16. And what I have are indices 2 and 3 are the children, and 4, 5, 6, and 7 are the children of 2 and 3. And 8, 9, and 10 are the children of 4 and 5, in this case. And so that's the picture you want to keep in your head for the rest of this lecture. Any time you see an array, and you say we're going to be looking at the heap representation of the array, the picture on the right tells you what the heap looks like.

And so that I'm not going to fill in all of these. You can, but I'll do a couple. So you have 10 here, and 8, 7, et cetera. So that's a heap structure.

So what's nice about this heap structure, is that you'll have tree representation of an array, and that lets you do a bunch of interesting things. What do you get out of this visualization? Well, the root of the tree is the first element corresponding to i equals 1. The parent of i is i over 2. The left child of i is 2i. And the right child of i is 2i plus 1. So that's essentially what this mapping corresponds to.

Now on top of that, this is just what a heap corresponds to. We're going to have particular types of heaps that we'll call max-heaps and min-heaps. And as you can imagine, max-heaps and min-heaps have additional properties on top of the basic keep structures. So this is essentially a definition of a heap. Now I'm going to define what the max-heap property is. And the max-heap property says that the key of a node is greater than or equal to the keys of its children. OK, that's it.

It's obviously recursive, in the sense that you have to have this true for every node in the tree. And when you get down to the leaves of the tree, they're not children corresponding to the leaves, So that's a trivial property. But at higher levels, you're going to have children, and you have to check that.

So if you look at this example here, maybe I should fill this whole thing out. A have eight and seven here, and six would be nine. And I have three over here, and then two, four, one. So we can look at this and check whether it has the max-heap property or not. Does it have the max-heap property? This heap? Yeah. All you have to do is look at these nodes. one, two, three indices, index four, five, six, but you don't have to look at six and seven, because they don't have any children. But you could shop with five here, and you look at the children, and there you go. To the parent is greater than or equal to either of its children, or its only child, in the case of node five. And so you have the max-heap property. So fairly straightforward property.

And you can imagine defining the min-heap property in an equivalent way. Just replace the greater than or equal to, with less than or equal to. So right off the bat, what operation is going to be trivially performed on a max-heap? This is kind of trivial question. Yep. Just finding the biggest element. Exactly right. The max operation.

Now, what about extract max? Is that trivially performed on a max-heap? No. What do I mean by that? When you say, max is trivially performed, what it means is that you can return the max, you can find the maximum element, or a maximum element, and you obviously don't modify the heap. And the heap stays the same, so it stays a max-heap. In general, when we talk about data structures, and this goes back to rep invariance, which I've mentioned already, you typically want to maintain this rep invariant. And so the rep invariant of our data structure, in this case, is a max-heap property. OK. So we want to maintain the max-heap property as we modify the heat. So if you go from one heap to another, you start at the max-heap, you want to end with the max-heap.

It makes perfect sense, because in one of the simplest things that you want to do in a priority queue, is you want to be able to create a priority queue, and you want to be able to run extract max on the priority queue, over and over. And what that means, is that you take the max element, you delete it, take the next max element, delete it, and so on and so forth. And there you go.

It's a bit of a preview here, but you could imagine that if you did that, you would get a sorted list of elements in decreasing order. So you see the connection to sorting, because you could imagine that once we have this heap structure, and we can maintain the max-heap property, that we could continually run extract max on it. And if you could build extract max in an efficient way, you might have a fantastic sorting algorithm.

So, the big question that really remains, is how do we maintain the max-heap property as we modify the heap? And the other question, which I haven't answered is-- this array that turns out it was a max-heap, but it's quite possible that I have a trivial example of an array. In fact, let me make this one. That is not a max-heap. It's not a max-heap, it's not a min-heap, it's neither. Right? it's just a heap.

So if I just transform, or visualize I should say, this array as a heap, I don't have a max-heap, I don't have a min-heap. So if I'm very interested in sorting, and I am, there's this another thing that's sort of missing here that we have to work on, which is how are we going to build a max-heap out of an initially unsorted array. Which may or may not turn into a max-heap. This trivially happened to be exactly the right thing, because I picked it, and it turned into a max-heap just by visualizing it. But it's quite possible that you have arrays that are input to your sorting algorithm that look like that.

OK, so let's dive into heap operations. I'm going to have spend some time describing to you a bunch of different methods that you would call on a heap. And all of these methods are going to have to maintain our representation invariant of the max-heap property. So what are the heap operations that we have to implement and analyze the complexity for? Well, we're going to have build-max-heap which produces a max-heap from an arbitrary or unordered array. So somehow I got to turn this into, for example, four, two, one. Which is in effect, sorting this array. Or changing the order. Maybe not fully sorting it, but changing the order. So that's what I have to do, and build-max-heap is going to have to do that.

In order to do build-max-heap, the first procedure that I'm going to describe to you, is called max-heapify. Heapify. Sounds a little strange, but I guess you can -ify pretty much anything. So you correct a single violation of the heap property in a subtree, a subtree's root. And I'll explain what I mean by that in just a minute.

So max-heapify is the fundamental operation that we have to understand here. And we're going to use it over and over. What it does, is take something that is not a heap, not a max-heap. When I say not a heap from now on, pretend that I'm saying not a max-heap. We're only going to be talking about max-heaps for the rest of this lecture.

What max-heapify does, is take something that is not quite a max-heap. It can't take anything arbitrary. It's going to take something where there's a single violation of the max-heap property at some subtree of this heap that is given to you, and there's a single violation of that. And it's going to fix that. And we need to be able to do this recursively at different levels to go build a max-heap from an unordered array. Then once you have that, you can do all sorts of things like insert and extract max, and heap sort, and so on and so forth.

So let's take a look at max-heapify using an example. I'm not going to write pseudocode for max-heapify. I'll run through an example, and the pseudocode is in the notes. The big assumption, and you think of this as a precondition, for running max-heapify, is the trees rooted at left i and right i are max-heaps. So max-heapify is going to look like a comma i. a is simply the array, and i is the index. Max-heapify is willing to, you're allowed to crash and not do anything useful if this precondition is violated in max-heapify. But if the precondition is true, then what you have to do is, you have to return a max-heap correcting this violation. That's the contract.

So let's take a look at an example. I think what I want to do is start over here. I want you to see all of the steps here. So we'll take a simple example, and we'll run through max-heapify. And let's take a look at 16, four-- I'm just going to draw the indices for this first example, and then I won't bother. So there you go. Is this a max-heap? No. Because right here, I've got a problem. 4 is less than 14, therefore I have a violation. And so, if you look at the call max-heapify A comma 2, this is an index 2, and all you have to do is to look at this subtree. And what you need to be satisfied in order to run max-heapify, is that the subtrees of nodes index two, which is this four node, are max-heaps. And if you go look below, you see that this is a max-heap and that's a max-heap.

Most of the time, by the way, you will be sort of working bottom up, and that's why this is going to make sense. This will all work out, because leaves are by definition max-heaps. Because you don't have to check anything. When you put two leaves together, and you want to create a tree like that, or a heap like that, then you run max-heapify. And then when you have a couple different max-heaps, and you want to put them together to make it a bigger max-heap, you'd have run max-heapify. So that's the way it's going to work.

So you want to do a max-heapify A comma 2. One of the things that's going to be important, not in this example, but when we get to sorting, is that we want to know what the size of the heap is. And in this case, the heap size is 10.

So, what does max-heapify do? Well, all max-heapify does is exchanges elements. And so, if you looked at the code for max-heapify, and you walked through it, this is what it would do. You're going to look at 4 and 14, and it's going to say, OK, I'm going to look at both my children. And I'm going to go ahead and exchange with the bigger child. So I'm going to exchange A[2] with A[4]. And what that would do is, take this, make this 4, and make this 14. And that would be step one.

And then when you get to this point, recursively, you'd realize that the max-heap property at this level is violated. And so you would go ahead and call max-heapify A comma 4. And when that happens, that call happens, you're going to look at the two children corresponding to this little subtree there, and you're going to do the exchange. You're going to have 8 here and 4 here. So you would exchange A[4] with A[8]. And now you're done, so there's no more calls.

So, fairly straightforward. It's actually not any more complicated than this. There may be many steps. What might happen is that you'd have to go all the way down to the leaves. And in this case, you went a couple of steps, and then you got to stop. But obviously, you could have a large heap, and it could take a bunch of time. So, what is the complexity of max-heapify? Anybody? Yeah. Back there.

AUDIENCE: Ultimately, potentially, if the tree is totally upside down, you could potentially switch every node to make it order in.

PROFESSOR: Every node to make it order in. Everybody, anybody. Do you have a different answer?

AUDIENCE: Log n.

PROFESSOR: Why? Why is it log n.

AUDIENCE: Because I think the worst case scenario, all of your-- the worst case scenario you would have [INAUDIBLE] on the left-hand side, [INAUDIBLE] right-hand side. And it would be skewed. [INAUDIBLE].

PROFESSOR: So you're arguing that the solution to the recurrence gives you a logarithmic complexity. Alright. Not quite. There's an easier way of arguing this. this Yeah. Back there.

AUDIENCE: [INAUDIBLE].

PROFESSOR: That's right.

AUDIENCE: [INAUDIBLE].

PROFESSOR: That's right. So what is the complexity?

AUDIENCE: Log n.

PROFESSOR: Log n. Great. Excellent. Definitely worth a cushion. Missed you by that much.

AUDIENCE: Thank you.

PROFESSOR: It's pretty soft, by the way. Right. OK. So, if I hit somebody, they get a cushion. OK. That's exactly right. Thanks for that description.

So, first off, there's two important aspects to this argument. The first thing is, that we're visualizing this is a nearly complete binary tree. It is not an unbalanced tree. Alright? We'll talk about unbalanced trees and balanced trees in the next couple of lectures. But the visualization of a heap is a nearly complete binary tree. And, in fact, if you had 15 elements, it would be a perfect binary tree. So the good news is, that the height of this visualization tree is bounded by log n. That's the good news. And you want to exploit that good news by creating algorithms that go level by level. If you can do that, you're going to have logarithmic complexity algorithms. So that was one aspect of it.

The other aspect of it, is the key assumption that we're making, with respect to build-max-heap, that there was a single violation. It is true that the answer that was given that was order n, would be a problem. I could set it up so that's actually the right answer, if I did not have this assumption-- where do I have that here-- assume that the trees rooted at left i and right i are max-heaps. So maybe that's what you were thinking.

But this is a key assumption. This is going back and like making connections between classes. This is a precondition that makes the algorithm more efficient. Makes the implementation easier. And this precondition essentially says that you have to just go down and do a number of steps, that's the number of levels in the tree, which is logarithmic.

So that's the story here with the max-heapify. It's order log n, in terms of complexity. That's the number of steps that you have. And it's a basic building block for all of the other algorithms that we look at for the rest of this lecture, and in section tomorrow.

Let's talk about how you would take max-heapify and use it to do build-max-heap. So the first step now, let's say that we want to go and get a nice sorting algorithm. We don't like insertion sort, we don't like merge sort. We'd like to get a heap-based sorting algorithm. One of the things that we need to do, as I said, is to take an unordered array, and turn it into a max-heap, which is a non-trivial thing to do. And once we do that, we can do this extract-max deal to sort the array.

So the first step is, we want to convert an array A 1 through n into a max-heap. And the key word here is max-heap, because every array can be visualized as a heap. And I am going write the pseudocode for build-max-heap, because it's just two lines of code. And that's about the limit of a size of a program I can really understand, or explain, I should say. And this is what it looks like. Alright. that's it.

Build-max-heap says go from i equals n, by 2, down to 1. Max-heapify A of i. So someone explain to me why I can start with n over 2, and why I'm going down to 1. Yep. I saw you first.

AUDIENCE: [INAUDIBLE].

PROFESSOR: Leaves are good. Leaves are good. I'll let you go on in a second. Leaves are good, because if you look at elements A of n over 2, plus 1 through n, are all leaves. That's a good observation. And this is true for any array. It doesn't matter what n is. Doesn't have the power of 2, or 2 [INAUDIBLE] minus 1, or anything like that. And leaves a good, because they automatically satisfy the backseat property. Continue.

AUDIENCE: OK. [INAUDIBLE].

PROFESSOR: That's exactly right. Beautiful. I won't hit anybody here. So that's it. The reason this works, is because you're calling max-heapify multiple times, but every time you call it, you satisfy the precondition. And the leaves are automatically max-heaps. Then you start with n over 2. You are going to see two leaves as your children for the n over 2 node, right? I mean, just pick an example here. Our 2 is an A of 5, right? You're out here. In this case, depending on the value of n, you may have either two children, or just one child. And you have one child. But regardless of that, that's going to be a max-heap, because it's a leaf.

And so you'll have two leaves, and you need to put them together. And that's a fairly straightforward process of attaching the leaves together. You might have to do a swap, based on what the element is. One operation and you get a little small tree, that's a max-heap. And then you do a bunch of other things that all work on leaves, because n over 2 minus 1 is probably also going to have leaves as it's children, given the large value of n. There will be a bunch of things where you work on these level one nodes, if you will, that all have leaves as children. And then you work on the level two nodes, and so on and so forth. And as I said before, you're working your way up, and you're only working with max-heaps as your left child and your right child. That make sense?

If you do that, and this is a fairly straightforward question, if you do a straightforward analysis of this, what is the complexity of build-max-heap? Yep.

AUDIENCE: [INAUDIBLE].

PROFESSOR: Right. So that's order. Order n log n. Now, this is through a simple analysis. Now I'm going to give you a chance to tell me if you can do better than that. Or not. In terms of analysis. It's a subtle question. It's a subtle question, that I'm asking. I'm saying, this is the algorithm, alright? I don't want you to change the algorithm, but I want you to change your analysis. The analysis that you just did was, you said, I got [INAUDIBLE] n steps here, because it's n by 2 steps. Looks like each of the steps is taking log n time. So that's n log n. And I was careful. I put big O here. OK? Because that's an upper bond. So that's a valid answer. Can you do better? Can you do a better analysis-- and I'll let you go first-- can you do a better analysis that somehow gives me better complexity?

AUDIENCE: I think you bring it to [INAUDIBLE].

PROFESSOR: OK. How?

AUDIENCE: So each node get a maximum of two [INAUDIBLE]. So, for some n, there will be a constant number of comparisons to max-heapify that [INAUDIBLE].

PROFESSOR: Yeah. It's hard to explain. You're on the right track. Absolutely on the right track. So it turns out that, and I'll do this, it's going to take a few minutes here, because I write some things out. You have to sum up a bunch of arithmetic series, and so on. So it's a bit unfair to have to speak out the answer, but the correct answer, in fact, is that this is order n complexity. This algorithm that I put up here, if you do a careful analysis of it, you can get order n out of it. And we'll do this careful analysis.

And I'll tell you why it's order n, in terms of a hand wavy argument. A hand wavy argument is that you're doing basically, obviously no work for the leaves. But you're not even counting that, because you're starting with n over 2. But when you look at the n over 2 node, it's essentially one operation, or two operations, in whichever way you count, to build max-heap.

And so for that first level of nodes, it's exactly one operation. The first level that are above the leaves. For the next level, you may be doing two operations. And so there is an increase in operations as you get higher and higher up. But there are fewer and fewer nodes as you at higher and higher up, right? Because there's only one node that is the highest level node. The root node. That node has logarithmic number of operations, but it's only one node. The ones down on the bottom have a constant number of operations.

So I'll put all of this down, and hopefully you'll be convinced by the time we've done some math here, or some arithmetic here, but you can quantify what I just said fairly easily, as long as you're careful about the counting that we have to do. So this is really, truly counting. Analysis has a lot to do with counting. And we're just being more careful with the counting, as opposed to this straightforward argument that wasn't particularly careful with the counting.

So let's take a look at exactly this algorithm. And I want to make an observation. Which is what I just did, but I'd like to write it out. Where we say, max-heapify takes constant time for nodes that are one level above leaves. And, in general, order L time for nodes that are L levels above the leaves. That's observation number one.

Observation number two is that we have n over 4 nodes that, give or take one, depending on the value of n. I don't want to get hung up on floors and ceilings. And in any case, we're eventually going to get an asymptotic result, so we don't have to worry about that. But we have n over four nodes with level one, n over 8 with level two. And 1 node with log n, sort of the log n level, which is the root.

So this is decrease in terms of nodes as the work that you're doing increases. And that's the careful accounting that we have to do. And so all I have to do now to prove to you that this is actually an order and algorithm, is to write a little summation that sums up all of the work across these different levels.

And so the total amount of work in the 4 loop can be summed as n divided by 4, times 1, times c. So this sum, I have one level here, and I'm going to do some constant amount of work for that one level. So I'm just going to put c out there, because eventually I can take away the c, right? That's the beauty of asymptotics. So we don't need to argue about how much work is done at that one level, how many swaps, et cetera, et cetera. But the fact is that these n over four nodes are one level above the leaves. That's what's key.

And then I have n over 8 times 2c, plus n over 16 times 3c, plus 1 times log of n c. I've essentially written in an arithmetic expression exactly what I have observed on the board above. Stop me if you have questions.

Now I'm going to set-- just to try and make this a little easier to look at, and easy to reason about-- I'm going to set n over 4 to 2 raised to k, and I'm going to simplify. I'm just pulling out certain things, and this thing is going to translate to c times 2 raised to k, times 1, divided by 2 raised to 0, 2 divided by 2 raised to 1, 3 divided by 2 raised to 2, et cetera, k plus 1 divided by 2 raised to k. Now, if that was confusing, raise your hand, but it's essentially identical given the substitution and sort of just applying the distributive law.

And the reason I did this, is because I wanted you to see the arithmetic expression that's in here. Now we do know that 2 raised to k is n over four, of course. But if you look at this expression that's inside here, what is this expression? Anyone? Can you bound this expression? Someone? For the cushion. Remember your arithmetic series from wherever it was.

AUDIENCE: [INAUDIBLE].

PROFESSOR: Yeah. You know better than I. I guess you took those courses more recently, but what happens with that? Those of you who have calculators, I mean, you could plug that in, and answer that. No one? Go ahead.

AUDIENCE: [INAUDIBLE]. You know that it's going to merge to two.

PROFESSOR: That's exactly what I was looking for. Essentially, well, it's not quite two, because you have a 1 here, and you have a 1 here, but you're exactly right. I mean, two is good. It's asymptotic, I mean, come on. I'm not going to complain about two versus three, right? So the point is it's bounded by a constant. It's bounded by a constant. This is a convergent series and it's bounded by a constant. And we can argue about what the constant is. It's less than three. And it doesn't matter of k goes to infinity. And you want k to go to infinity, but it doesn't matter if k is small or k is large, this is bounded by a constant. And that's the key observation.

What do we have left? What do we have left? We have a constant there. We have a c, which is a constant, and we have a 2 raised to k, which is really n. So there you go. There you have your theta n complexity. Now I can say theta n, because I know it's theta n. But big O of n, theta n, that's what it is.

So that's what I'd say is subtle analysis. Clearly a little more complicated than anything we've done so far, and let me see if there are questions. How many people got this? I did too. Someone who didn't get it, ask a question. What didn't you get? What step would you like me to repeat here? Any particular step?

AUDIENCE: [INAUDIBLE].

PROFESSOR: This thing here? Right here? OK, so you're not convinced that this expression got translated to this expression. So let me try and convince you of that, alright? So let's take a look at each of the terms. n by 4 is 2 raised to k. I'm just looking at this term and this term. n by 4 is 2 raised to k. c is c. And I just wrote 1 as 1 divided by 2 raised to 0, which is 1.

And the reason I want you to do this, is because I want to show you an expression where in some sense, this is the term that is the summation for your expression. If we just replace this, you can write this out as i equals 0 through k, I plus 1 divided by 2 raised to i. That is the symbolic form of this expression, which came from here.

And then the argument was made that this is a convergent series and is bounded by a constant. That make sense? Good. So that's pretty neat, right? I mean, you have the same algorithm and, whala, it suddenly got more efficient. Doesn't always happen, but that tells you that you have to have some care in doing your analysis, because what really happened here, was you did a rudimentary analysis. You said, this was order log n, big O log n, and you said this was theta n, and you ended up with this. But in reality, it's actually a faster algorithm.

So that's the good news. Build-max-heap can be done in order n time. Now in the time that I have left, it turns out, we are essentially all the way to heaps sort. Because all we have to do is use, once we have build-max-heap, I'll just write out the code for heap sort, and you can take a look at examples in the notes. The pseudocode, I should say, for heap sort. And it looks like this.

The first step that you do is you build max-heap from the unordered array. Then you find the maximum element A[1]. All of this I've said multiple times. Now the key step is, you could do extract max, but one nice way of handling this, is to swap the elements A[n] with A[1]. Let me write this out and explain exactly what that means.

Now the maximum element is at the end of the array. When you do the swap. That's the one step that I will have to spend another minute on. Now we discard node n from the heap, simply by decrementing heap size. So the heap becomes n minus 1 in size from n in the first iteration.

Now the new root after the swap may violate max-heap, we'll call it the max-heap property, but the children are max-heaps. So that's the one node that can possibly violate it. So what that means, is we can run max-heapify to fix this. And that's it . Once you do that, you go back to that step.

So what's happened here exactly? Well this part we spent a bunch of time on. element is the maximum element, so you grab that. And you know that's a maximum element. One way of doing it is to use extract max, but rather than doing extract max, which I haven't explained to you, you could imagine that you go off and you swap the top element with the bottom element, and then you discard it.

So here's a trivial example, where let's say I had 4, 2, and 1, which is a max-heap. What would happen is you'd say, I'm going to take 4 and I'm going swap it with 1. And so you have, 1, 2, and 4. Now four used to be A[1], and that's the maximum element, and I'm just going to delete it from the heap, which means I'm going to end up with a heap that looks like-- a heap, not a max-heap-- that looks like this. And I write down 4 here. 4 is the first element in my sorted array.

Now I look at 1 and 2, and 1 and 2 there's obviously not a max-heap. But I can run max-- I know the child is a max-heap, so I can run max-heapify on this. And what this turns into is 2 and 1. And at this point, I know that the max is the root, because I've run max-heapify and I take 2 out, and after this, it becomes trivial. But that's the general algorithm.

So this whole thing takes order n log n time, because even though build-max-heap is order n and max element is constant time, swapping the elements is constant time. But running max-heapify is order log n time, and you have n steps. So you have an order n log n algorithm. But the first step was order n, which is what we spent a bunch of time on.

So I'll show you examples in the notes, and that will get covered again in section. I'll stick around for questions. See you next time.

Lecture slides (PDF) (Courtesy of Charles Leiserson, Piotr Indyk, Constantinos Daskalakis, and Srini Devadas. Used with permission.)

Free Downloads

Video

• iTunes U (MP4 - 113MB)
• Internet Archive (MP4 - 113MB)

Subtitle

• English - US (SRT)