Lecture 13: W Hierarchy

• Download this transcript - PDF (English - US)

The following content is provided under a Creative Commons license. Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free. To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu.

 

PROFESSOR: Today, we begin a new topic, which is parameterized complexity. This is one of the most recent areas of hardness that has been invented that we will cover so far. It started around 1999, and then since the 2000s, it's been getting really big, originally by Downey and Fellows.

And the general idea is we're going to take problems, which normally we think of as being parameterized by the problem size, n, and thinking about polynomial versus exponential growth of n. And we're going to add another parameter. So you could say this is a two parameter analysis in some sense. Usually, we call problem size n and then we call the parameter k.

In general, a parameter is just some way of measuring instances. So parameter, k, is a function from instances to non-negative integers, 0, 1, 2. And usually, this number k is just part of the instance, for example-- well, let me go over here to examples. Let's say problem that's called k-vertex cover-- and usually, I'll just call it vertex cover for simplicity.

You know the problem vertices to cover edges. And usually, the decision version of that is are there k vertices that cover all edges?

So k is an input to this problem, and it appears in the statement of the problem. But in particular, it's an input. And so this function, k, of a vertex cover instance just pulls out that one number and throws away the graph. That's the typical parameter. But in general, a parameter could be some hard to compute thing. Maybe k-- you think of the parameter-- you take vertex cover as your problem, but you define your parameter to be the smallest independent set-- or largest independent set. Smallest is pretty small, 0.

Or, pick your favorite. You could, for example, think about vertex cover with respect to-- this is how we'll write the parameter-- crossing number.

So the idea here is-- sorry I forgot to covet-- we always measure our instances with respect to the problem size, which is the size of the graph for vertex cover instance. But we're also going to parameterize by some other quantity here. Maybe it's the minimum crossing number for your graph, so 0 if it's planar and so on. We prove that that's NP-complete to compute. So this parameter may not be easy to compute in general. But a lot of the times it's just part of the problem. And in general, this is called the natural parameterization.

If you have an optimization problem, like vertex cover is-- minimum vertex cover-- and you convert the optimization problem into a decision problem, you're asking is opt less than or equal to k. That k is the natural parameter for an optimization problem. So most the time, we'll be thinking about that. But things like this also arise.

So just to be a little more precise a parameterized problem is a decision problem plus the parameter.

So when I said vertex cover with respect to crossing number, I meant my decision problem is vertex cover, and I'm going to choose to use this particular parameter function to parameterize those instances. And in general, we'll call them parameter k and problem size n. And our goal is to get a very good dependence on n at the cost of a bad dependence on k.

So let me start with a so-so dependents, XP, would be the set of all prioritized problems solvable in n to the f of k time for some function k-- for any function k, any function f. And FPT is the set of parameterized problem solvable in f of k times n to the order 1.

So this is considered a good running time. This is considered a bad running time. F is presumably exponential. Assuming your problem is NP-hard, you have to have exponential in something. And the goal is to get the exponential away from n because n is hard to control. We like to solve big problems, but maybe we can somehow characterize that the problems we care about have some small measure, k-- for some interesting measure k. If such an algorithm is possible and, in practice, your k's are small, then you're golden.

These give really good algorithms for solving lots of problems. Any problem in FPT, if k is somewhat reasonable, even when n is huge, we can solve the problem.

This running time is also polynomial for fixed k, but the polynomial changes depending on k. So here, we typically get linear time algorithms for any fixed k. Here, as k increases, the polynomial gets bigger and bigger. And in practice, you probably can't handle more than an n squared algorithm for a large n. So this is considered not useful even for small k. This is considered very useful for small k, depending on your notion of small, of course.

And parameterized complexity, the main name of the game is distinguishing between FPT. And XP is actually rather large, but distinguishing between when this is possible and when it is impossible.

Cool. So for example, vertex cover is FPT. Vertex cover with respect to crossing number, I think, is FPT. I have to think about that a little bit more.

AUDIENCE: Can you say what XP and FPT roughly stand for?

PROFESSOR: Oh, right. FPT is fixed parameter tractable. That's the good word. intuitive-- etymological that means when you fix the parameter k, you get a tractable problem although it's a strong sense of tractable. You can say it is n squared for any fixed k. Or, it's n to the fifth for any fixed k. This constant does not depend on k. But the lead constant does. So that's FPT.

XP, experience points. Good question.

One other good thing to know. You might wonder why f of k times polynomial n. Maybe I could hope for better like f of k plus polynomial n. That actually is the same class. So FPT also equals the set of all parameterized problems solvable in f of k plus n to order 1. That's a nice general theorem. I won't bother proving it here. I think the f ends up becoming square of the original f. It's a very easy case analysis based on whether which of these two things is bigger.

I will leave that as an exercise. It would be a good problem set problem. Cool, so that answers one plausible question.

Let me give you an example of an FPT algorithm, let's say, for vertex cover. So back over here to vertex cover. How about an XP algorithm for vertex cover. So that's really easy, not very interesting. You could just guess a vertex cover, try all possible vertex covers of size k. Remember, we want to know whether there's a vertex cover size k. k is given to us, so we could try all vertex covers-- or, I shouldn't say that-- all vertex sets of size k.

There's only n to the k of them. And for each of them, we'll spend linear time to check whether it's a vertex cover. So this proves that the problem is in XP. That's trivial. Most problems you'll find are in XP with reasonable parameterization. On some, it's not obvious. And now, let's prove that it's in FPT. This is a little more interesting. There are lots of FPT algorithms for vertex cover. It's an active area. Every year there's a better algorithm pretty much. Question?

AUDIENCE: So in the inputs of the problem, the k is an input?

PROFESSOR: For vertex cover, k is an input.

AUDIENCE: Oh, because you can also think of vertex cover where k is not an input, but you can analyze the run time.

PROFESSOR: Yes. Right. So you could also think about vertex cover with respect to vertex cover. So here, let's say-- this is a little bit funny, but here the idea is k, the parameter, is the minimum vertex cover. Let me write minimum vertex cover. And here, you're given some other value. Let's call it j. And you want to decide whether vertex cover is at most j, but you're not told what k is. You're not told what the optimal value is.

So this is a little bit subtle. It's essentially the same problem. You're just not told the critical value. But if you can solve k vertex cover when k is given, you can just run that algorithm for k equals 1, 2, 4, and you'll end up solving this problem. So even though-- so in this problem, you can say, well, if I discover there's no vertex cover of size k, I can just return no.

Here, it's a little more annoying, but just by iterating and stopping when you get the first vertex cover, your algorithm will end up having a good running time with respect to k even though you didn't know it. So it's a subtlety. In general, this is messy to work with, so we usually think about k being an input. But that reduction shows they're more or less the same. But the complexity theory for this is annoying.

In the same way that for NP, we restricted decision problems, here we're going to usually restrict to the natural parameter. So good question.

Let me prove vertex cover is in FPT, so a much better running time than n to the k. I think I will get something like 2 to the k times n. Look at an edge of the graph, any edge. Pick an edge, any edge. And we know for this to be a vertex cover, one of the two endpoints has to be in there. Which one? I don't know. Guess. So I'm going to guess either the left guy or the right vertex is in the vertex cover, and then repeat.

So when I put somebody in the vertex cover, I delete all of the incident edges. That takes linear time. Over here, I would delete whatever edges are incident to this guy and the vertex itself, of course. And so I get a smaller graph. I do this k times, so in general, my execution tree will have two branches, every time I pick an edge and say, do I put the left guy or the right guy in.

But I only need to worry about a tree of height k because after I make k decisions, I'm supposed to get a vertex cover of size k. After I make k decisions, if there's anything left to the graph that means that wasn't a vertex cover of size k and I should backtrack. So I just explore this tree, look at all the different leaves. If there's any vertex cover of size k, I will find it. And so I only spend 2 to the k to explore this tree.

2 to the k is the size of the tree times n because I maybe delete n things at every node. And that's one of the simplest FPT algorithms for vertex cover. There are much cooler, fancier ones. But this is now going to be about hardness, so I will stop with algorithms.

Also related is a stronger notion of PTAS. So I mentioned before, with PTAS, you could have a running time something like n to the 2 to the 1 over epsilon-- which is actually fairly common in a lot of PTASes, not so great. If epsilon is anything good, then this is going to be a really huge polynomial. Well, the corresponding notion in FPT land is a running time of f of epsilon-- or, I'll write 1 over epsilon. It doesn't matter-- times n to the order 1.

So this is really saying that the approximation scheme is FPT with respect to 1 over epsilon. This is to get a 1 plus epsilon approximation. So this is relating the two worlds. If you want a FPT algorithm for approximating a problem and getting within a factor of 1 over epsilon, if you get this kind of running time verses this kind of running time, I consider this good, this bad.

And in general, these are called Efficient PTASes, or EPTAS. And in general, if you have an EPTAS, you also have an FPT algorithm with respect to the natural parameter, which is to basically compute what the optimal value is, because you can let epsilon be something like 1 over 2k-- that ought to do it-- something less than 1 over k.

You're trying to decide whether you have a solution of less than or equal to k or not, whether optimal is strictly greater than k. And I said k is an integer, so this is an integer valued problem you're trying to optimize. And so the difference became k and k plus 1, multiplicatively, is about 1 plus 1 over k factor. It's exactly 1 plus 1 over k. So if we let epsilon be smaller than 1 over k, then to get within a multiplicative factor of 1 plus 1/2 1 over k, you actually have to be within an additive plus 1 of the right answer at k.

So this will decide whether opt is less than or equal to k based on whether that approximation algorithm will give you something of value at most k or strictly bigger. Strictly bigger would be k plus 1.

So this is particularly interesting because it means if a problem is not in FPT, then it does not have an Efficient PTAS. So we can use-- before, in previous lectures, PTAS was considered gold. Now, we can distinguish between EPTAS and PTAS. And there are problems that have PTASs that do not have FPT algorithms assuming FPT does not equal w1-- an assumption we will get to in a moment.

So if you can, in general, establish something is probably not in FPT, then we get it probably does not have an Efficient PTAS. So this is useful for lower bounds about approximation. Even if you don't care about fixed parameter tractability, this will give us cool things about approximability.

AUDIENCE: Can you give a few examples of EPTAS?

PROFESSOR: A few examples of EPTAS. For example, most of the planar results I've talked about, planar independence set, planar vertex cover, those things all have Efficient PTASs. But that gets us into the world of bounded treewidth algorithms. But in general, if you have like a-- a lot of bounded treewidth algorithms are FPT. There will be some exponential in treewidth times polynomial n

So a lot of times, you can use those to get EPTASs for planar and h-minor free problems. So that's one set of examples. There's no short example I can give you, but that's a big class. So for either reason, either you want to solve problems for small k or you want to figure out whether there's an EPTAS, how can we prove that problems are hard in this world using reductions like usual?

So in this context, luckily, there's only one type of reduction we need to learn unlike approximation where there were a lot. And I'll call it parameterized reduction although throughout this lecture and the next one I'll just say reduction usually because we're always talking about parameterized problems.

So in general, we have some problem A-- decision problem A and parameter k. And we want to convert into some decision problem B with parameter k prime. And of course, as usual, the set up is we're given an instance x. This is going to look almost identical to NP Karp-style reductions, but then we're going to have one extra condition.

So instance x of A gets mapped to by a function f to an instance x prime of B. X prime is f o x as usual. This needs to be a polynomial time function just like for NP reductions, which means, in particular, x prime has polynomials size reduced back to x. It should be answer preserving. So x is yes instance for A if and only x prime is a yes instance for B.

So, so far, exactly NP reductions. And then when we need one extra thing which is parameter preserving.

This is there's some function g, which I'll call the parameter blow up-- or, I guess you call it parameter growth for g-- such that the new parameter value for the converted instance is, at most, that function of the original parameter value of the original instance. Question?

AUDIENCE: Are there any limits on the amount of time that g can take to compute?

PROFESSOR: g should be a computable function. I think that's all we need. Probably polynomial time is also a fine, but usually this is going to be like linear or polynomial or exponential. It's rarely some insanely large thing, but computable would be nice. Cool.

So that is our notion of parameterized reduction. And the consequence, if this exists and B is fixed parameter tractable, then A is because we can take an instance of A converting it to B. If the original parameter was founded by some k, this new parameter will be bounded by g of k. New instance will be bounded of g of k. So we run the FPT algorithm for B, and that gives us the answer to the original instance of A.

So if we don't care about what this function is, we are basically composing functions. So there's some f dependence on k in this algorithm, and we're taking that function of g of k is our new function. And we get a new dependence on k and the running time over FPT. So what that means is if we believe it A does not have an FPT, then B does not have an FPT if we can do these reductions.

So same style, we're going to reduce from a problem we know is hard, A, into a problem that we don't know about, and that proves B is hard as well. Yeah?

AUDIENCE: What is the relationship between FPT and XP? Is there like a difference?

PROFESSOR: Yeah

AUDIENCE: If there's overlap.

PROFESSOR: Well, FPT is contained in XP. And they are different if you believe the exponential time hypothesis. If you believe SAT does not have two to the little of n algorithms-- if that's not possible for SAT, then XP and FPT are different, and much, much more things are different. But we'll talk about that more next class-- relating to exponential time hypothesis.

So at this moment, we don't really care what g is. But if you assume exponential time hypothesis, then g matters, and you can get very explicit lower bounds about how good an algorithm you could hope for. So we won't just prove you're probably not an-- we won't just prove you're not an FPT, but we will give an actual lower bound and how much running time you need, something like n to the little k.

But the running time we get will depend on this blow up function. So next class, we'll care about what g is. So I do try to minimize it, but so far we don't care. Any g is fine. XP will not turn out to be the class that we think about very often. There are things in between that are a lot easier to work with, which we will get to. In particular, W1 is the most common.

So let me do two examples. One of which is a correct example and the other is not. And you can help me figure out which is which. Independent set to vertex cover, these are reductions we've seen before. We have tons of reductions, but usually we weren't thinking about parameters. And independent set to clique, these are basically identical problems left and right. And I want the natural parameter. Which of these is a parameter preserving reduction?

I need some quiz show music.

AUDIENCE: The second.

PROFESSOR: The second, yeah. Because if we take a graph and a parameter k, what we convert it to is the complement graph with the same parameter. So that's obviously parameter preserving. Whereas independence set, these are complementary in a different sense that if you take everything that's not in the independence set is a vertex cover. So it's actually the same graph but with n minus k as the new parameter.

So this is not a parameterized reduction. This is. And in fact, vertex cover, we just showed is FPT. Independence set is not, if you believe w1 does not equal FPT or if you believe exponential time hypothesis. And that would contradict this statement. So this is definitely not a valid reduction. But those are some pretty trivial reductions. Let's do some more interesting things. And I want to start to introduce the notion of w1.

I won't define w1 yet. I'd like to wait a little bit because the definition is not super intuitive, but here is a fairly intuitive hard problem that you should be fairly convinced is not FPT. So I'll call this k-step non-deterministic Turing machine. The one downside is I've never mentioned Turing machines in this class. I thought I could get away without it, but I'm going to have mention them. How many people know what they are? Anyone does not? OK, a couple. Here's a Turing machine.

I will give you, very briefly, a very non-standard definition of a Turing machine. Let's think of-- so you have this infinite tape, infinite memory, and for our purposes, these are not just binary symbols, you can write up to n different symbols on each of the squares. So this is basically your memory, but it's in the style of old-fashioned tape drives in that you-- In order to get to position k away from you, you have to spin k time. So you can only step one unit at a time.

And this is basically computers that we are used to. So let's say it has a sequence of instructions, and it has an instruction pointer. And in general, I want to the machine to have order n stateS-- yeah, let's say order n lines of code and order n options per cell of the tape. This will look actually a little weird if you're used to Turing machines, but I want to use this definition.

So it's basically a regular algorithm, but you're very limited the number-- in the amount of internal state. These are basically registers, but you can have only a constant number of registers that vary from one to n. One of them will be what line of code are you on. And then you have those lines of code are instructions like jump here, compare these things, write a particular symbol to the current square of the tape, move the tape left, move the tape right. So regular types of instructions, let's say, except you have this weird tape thing.

Now, I mentioned these parameters. And usually we think of Turing machines of having constant size, but here I need an n, and I need a k. Question?

AUDIENCE: I was going to ask what n was.

PROFESSOR: OK. So n is the input. I mean basically the Turing machine is specified by these things. You have order n instructions, and we're guaranteed there's only order n possible states, let's say. And now what we-- and this is a non-deterministic machine. Now non-determinism, we have talked about this in the context of NP. A non-deterministic Turing machine has a funny instruction which says non-deterministically branch to one of n different locations in my memory. Or, let's say choose a symbol from my-- I have this alphabet that I'm using in my cell-- choose a symbol non-deterministically.

So I have order n choices made non-deterministically. In the usual sense of NP, that if there's any way for the Turing machine to output, yes-- one of the instructions is output, yes-- then, I will find it. These are guesses, and they're always lucky guesses, so I always end up finding the return, yes, if there is such a path. Otherwise, all paths return, no, and then the machine returns, no. Yeah?

AUDIENCE: What does it mean for the number of states to change as the input changes?

PROFESSOR: I mean the states are also given to you as part of the Turing machine. There's not one turning machine-- well, there is actually one Turing machine to rule them all, but that's not the point here. I give you a machine that has-- think of this as firmware built into the machine, and the number of states in the machine includes which instruction you're currently executing. So I mean this is just saying you're given an arbitrary machine of size-- of complexity n. And usual Turing machine land, there are n states because I give you a state diagram with size n.

So you're given everything about the Turing machine. So that is the input to this problem. And the question is, can I find a return, yes, solution that only is k steps long. So I'm given this huge machine, and yet I want a very short execution of the machine. I want the running time to only be k. k, again, is parameter. n is big. k is small.

So I want to know is there a yes path from the initial state of length k. So this is basically-- Yeah, question?

AUDIENCE: Are you given the input to the Turing machine?

PROFESSOR: Yeah, let's say it has no input. Input is all embedded in the code. So the tape is initially blank. Good. There's a reason I need to do Turing machines here instead of usual algorithms, but if you define-- usually NP is defined in terms of these things. So by analogy to NP, we expect there are no lucky algorithms. And so we expect that when you have non-deterministic branches, the best thing you can do is to try all the branches.

And so if I have an execution time of k, and at each step, I can potentially make a non-deterministic choice among n different options, then you would expect the best algorithm is n to the k. Try all the branches, that's just like our vertex cover algorithm, but in the bad case where I have branching factor n instead of branching factor 2. So presumably, there's no way to replace a guess among n options with a guess among two options.

That's an assumption. And this problem is w1 complete. I will to define w1 a little later, but for now, just take this as given. You could define w1 complete to mean problems that are reducible via parameterized reductions to this problem. And then we'll get lots of examples of problems that are as hard as this. So a modulo the annoyance of having to define Turing machines, I think this is a pretty natural assumption.

It's stronger than p does not equal NP, so we would imply that. So let's do some simple reductions-- some reductions, maybe not simple.

So I mentioned this problem independent set. That's also w1 complete. And to prove, first of all, that it's w1 hard, I'm going to reduce this problem to independent set. So I'm given a Turing machine and a number k. I want to convert it into a graph, and a number k such that if there's an independent set of size k in that graph, at most k in that graph if and only if there's an accept path of length at most k in the Turing machine.

So I'm going to skip a lot of details because I've been a little vague about how Turing machines work, but the idea is nice.

So my graph is actually going to consist mostly of k squared cliques. The cliques are actually quite large because this graph has size n, total size, but there's going to be k squared clusters, which are cliques. Plus cliques, not very many different cliques, only k squared of them. And my target independent set is size k squared, which means if there's going to be an independent set, I must choose exactly one vertex from each clique. So one vertex per clique.

Independence that has to have no edges among them. So if I chose two from a clique, there would be an edge among them, that's disallowed.

And so the cliques I'm going to parameterize by two parameters i and j between one and k. And the idea is that ij represents memory cell i at time j plus the state of the machine.

So what I mean is there are order n states in the machine, in general, that completely characterize what the machine is about to do and what it's thinking about, all of its internal state. And so I'm just going to take those n states and plop them in. And each node in this clique represents one of those states. But also I want to keep track of what symbol is written on that square of the tape.

And so, in general, my running time is bounded by k-- also the number of squares I write to the tape is bounded by k. Space is at most time-- so I only have to worry about k different cells for times 1 up to k. So there are only order n states for this, only order n states for this. So the size of each clique is order n squared polynomial. That's cool. It's a messy clique. I won't try to draw what it really looks like.

And then the general approach is I'm going to add some edges between pairs of vertices to say, well look, if this cell was this thing at this time, and the state of the machine was move right, then you better not change what was on the cell at that time because nothing changed. You weren't here. So we can just forbid that by saying, well, here is one state. These two states should be mutually exclusive.

So if I choose this vertex in an independent set, I can't choose this vertex. So you just draw all these connections of forbidden things. If something is true time i, times j, then something else should not be true at time j plus 1. You draw in all those connections, and it works. I'll leave it at that because the details are messy, but it gives you some idea.

At this point, I can plug a book on this topic that is coming out next year, a little bit far in the future but it is-- I didn't write it down here of course. Sorry. It is by these guys again, Fomin, Kowalik, Lockshtanov, Marx, Pilipczuck brothers, and Saurabh. And it does go through the details, the gory details. If you're interested in that, I could share with you that section.

But unfortunately, the book is not released yet. Cool. So that was a reduction to independent set, which shows the independence set is at least as hard as this problem, which we think is hard, w1 hard. So that proves w1 hardness. But there's another question of is independent set in w1. As you might guess from the name w1, there's more than just w1. There's w2 and w3 and w4 and even more.

And there's no one notion of hardness like we had with NP here, so we get a bit of complexity in that hierarchy. But in fact, independent set is w1 hard-- sorry-- w1 complete. So we can do a reduction from independence set to Turing machine. Let's say k-step non-deterministic Turing machine.

And the idea is pretty simple.

We have k-steps, so let's guess all the vertices. You can see here this is where we need the ability to guess among n different options in each guess step. So we're given-- we want to know whether there's an independent set of size k, so guess all those vertices of size k and then check it. So this was our XP algorithm, but now phrased as a non-deterministic Turing machine algorithm. And it's a little bit subtle what it means to check.

Of course, what we want to do is make sure there's no edge between any pair of the chosen vertices. So what we're going to do is when we guess these k vertices, we're going to write them one at a time on the tape squares. So we'll use k tape squares to store the k vertices that we've chosen. Then, we want to check each pair of them. So this is like a doubly nested loop for each vertex among the k, for each vertex among the k.

So what that means is you're going to have to-- sorry, that's a little bit annoying-- loop through the vertices on one tape. And then the fun part is the graph is encoded inside the machine. You could think of there being a data section within the code that says where all the edges are. The graph is size n, and so you can put the entire graph into the machine. And then given two vertices, you could check that there is indeed no edge between them basically by using the code as a look-up table.

Say hey, is there a vertex from i to j. If yes, then we're in trouble, abort this option, try the other guesses. If you look up in the table and it says there's no edge, then you keep going. So you test all the pairs, make sure there's no edge by-- the code, as it is, is a constant size algorithm which looks like this plus an order n size thing, which is the look-up table for what edges are in the graph. So that part's a little bit weird. Yeah?

AUDIENCE: So in case of the k in k vertices aren't the same k?

PROFESSOR: That's right. I'm sorry. This is k prime. Yes. So probably k prime here is exactly order k squared. Thank you. It makes me a little happier. Because I'm doing this doubly nested loop, I'm going to have to loop over this thing several times. I would look at the first vertex, memorize it by reloading it into a register, and then loop over all the other guys for each of them check that they're bad. So I'm going to have to go over the tape k squared steps.

And also n prime is whatever. We get a machine that's size-- basically the size of the graph V plus E. So the point of doing that was to show the independence set is also w1 complete. So if you were not happy with Turing machines, now you can completely forget about them. They are mainly used here as motivation for why you should expect this problem to be hard.

But it's equally difficult from a parameterization perspective. If you ignore how big the blow up is, it's just as hard as an independence set. So independence set is a problem we think there's no good algorithm. And by this reduction, and also the reverse reduction, clique is also w1 complete.

So a bunch more reductions.

In general, most w1 hardness results start from clique. So that's a good problem to know about, or independent sets, I think. And there are some simpler versions of clique that are also hard. So first one is clique in regular graphs. So it's helpful to assume all the vertices have exactly the same degree. Is it hard in three regular graphs? No. Because three regular is only a clique of size three. So that's pretty easy.

But in s regular graphs, the degree is going to be huge, going to be some function of n. Clique is hard, and in general, let's say capital delta is the maximum degree in your given graph, and we want to convert it into a delta regular graph. So we're going to increase all the lower degrees up to delta. What we're going to do is take delta copies of the graph. So for every old vertex, we get delta copies of it.

And then we do this reduction. So the blue is the original-- is the graph that's been duplicated delta times. Here, delta is 5. But suppose, in the graph, this vertex only had degree 3, some number less than 5. Then, what we're going to do is create delta minus d new vertices and then add a biclique here, bipartite clique, connecting all of those delta minus d things to all of these delta things.

They're delta copies of the original vertex. This is v over here, v1 through v delta. And so what that means is these vertices will have entered degree delta because there were delta things to connect to over there. And now these things will have degree delta because they used to have degree d, and now they have an additional degree delta minus d on the left. So cool, everything's now delta regular.

And I claim that this reduction preserve cliques because if you look at these added vertices, they do not belong to any triangles. This thing here is an induced bipartite graph, and so there are no triangles, no things of size-- no cycles of size 3 because all cycles are even here, which means if you're in a clique-- if you have three guys in a clique, then they better have triangles everywhere.

So if you put one of these vertices in, you'll have a very small clique, namely size 2. And there are always cliques of size 2 in a graph. There's at least one edge. So if there was at least one edge before, afterwards, we do not increase any of the clique sizes. So clique size is preserved. Everything's cool.

So in this situation k prime equals k when you blow up anything, except the graph size, of course. That got a little bigger. And now everything's delta regular.

Le me give you an example of why this is useful. Of course, every time I say something is true of clique, it's also true of independence set. You can just flip things. Here, we have to be a little careful, but when you complement a graph, if you started with a regular graph, your new graph will also be regular. Every vertex would have degree n minus 1 minus delta or n minus delta, something like that.

So here's another problem, eerily similar to vertex cover, called partial vertex cover. This is also w1 complete. We want to know, can I choose k vertices that cover l edges?

So usually vertex cover, you need to cover all the edges, and that let us do some crazy things because whenever we looked at an edge, we knew one of the two things was in the vertex cover. Now, we don't know that anymore. Now, it's a matter of which vertices get the most bang for your buck. And this is easy to reduce from delta regular independence set because-- k independence set if we want to be explicit.

We want to know is there an independent set of size k. I just give that to partial vertex cover with k prime equal to delta times k. Sorry, that's l prime is delta times k. So the idea is here I want to choose k vertices that are independent. And if I can do that, I will be able to choose k vertices that cover exactly delta times k edges because if they're independent, none of the edges will be shared among my independence set. And this is if and only if.

If I try to choose k vertices that cover exactly delta k edges, then they can't be adjacent. The vertices can be adjacent. So these problems become the same under this mapping, so I didn't even blow up. My parameter here is k. I forgot to mention. I mean you could guess from the letter, but parameterized by k, this problem is w1 complete. Parameterized by l, this problem is FPT. So be a little careful.

But here's the reduction for k. There is no reduction for-- there's only good reductions for l. So there we are clearly using that the graph was regular. Otherwise, it would be at most k-- at most delta for everybody, and then it's hard to get the actual independence. Yeah.

AUDIENCE: Sorry, I'm still a bit confused about the relationship between w1 and XP and FPT. I don't know what w1 is.

PROFESSOR: Yeah, well we'll get there. For now, you can think w1 is all problems they can be parameterized, reduced to k-step non-deterministic Turing machine. That's a fine definition. Some people use that. And in general, FPT is contained in w1. It's contained in other things, which we will get to. w2 and so on is all contained in XP. And these are strict if you believe exponential time hypothesis.

So if you believe there are no sub exponential algorithms for SAT, then this problem has no FPT algorithm with respect to k and all the w1 complete things. I mean from a complexity theory standpoint, it will be fun to look at these larger classes. From is there an FPT algorithm standpoint, all you care about is it's not here. And any hardness, w1 or worse, will imply there's no FPT algorithm if you assume ETH.

AUDIENCE: Do you know if any of these inclusions are strict?

PROFESSOR: Like I said, they're all strict if you assume exponential time hypothesis. If you prove XP is different from FPT, then you prove P does not equal NP, I think, pretty sure. So we're not going to non-categorically say these things are strict, because these are all stronger versions of P does not equal NP. But if you believe exponential time hypothesis, then they're all strict. So that's one standard assumption that gives us everything. Cool, let me give you another version of clique that's hard.

Mufti-colored clique if you remember way back to three partition, we had a variation on three partition called numerical three-dimensional matching where you had to choose your triples from three different sets. So this is the analog of that, or the analog of three-dimensional matching where you had three different sets of vertices. For clique here, set the vertices partition into k clusters.

And the question is, is there k clique with one vertex per cluster? And in fact, so you think of these things as being colored, color 1, and these are color k and so on. And in fact, we can assume this is a proper coloring because if you're not allowed to choose two vertices from one class, then there's no reason to have edges between vertices of the same color. So this is in fact a k-coloring of the graph.

So we are given a k-color graph. And we want to know, does it have a k clique? And if it has a d clique and it's k-colored, then in particular, you will have exactly one vertex per color class.

We can prove this is hard by a reduction from clique.

Namely, if we have a vertex v, we're going to make k copies of it. You'll see this is a common trick.

And we will color them 1 to up to k. So good, now we have vertices in each color class. And then if we have an edge (v,w) convert it into edges (vi, wj) for all i not equal to j. So pretty much the obvious reduction. Once you said make k copies, we'll also duplicate the edges in all versions except, because it's supposed to be a k-color, we're not allowed to connect (vi,wi). But otherwise, we'll just throw in all of those edges. And the point is this, again, doesn't really blow up your cliques because if you have some clique in this structure, you can just forget about the i indices.

Because you know it's a clique, you will never choose two verses from the same color class, two vertices with the same index, and so you can take any clique here and collapse it to a clique here. Conversely, if you have a clique here, you can just assign those vertices arbitrary numbers as long as they're all distinct any permutation of one through k, and you'll get a clique down here. So it's the same problem. Question?

AUDIENCE: That v is different from the v above it, right?

PROFESSOR: Yeah, this is for all v and for all vw. If there's an edge in the graph, then we do that. And if there's a vertex in the graph, we do that. Cool. So k prime here equals k, no expansion.

This may seem trivial, but it's actually a fairly recent innovation to think about multicolor clique and, in general, it simplifies proofs. I have heard of proofs simplifying from tens of pages to one page. We'll probably get to some more-- to such sophisticated example soon. But I can give you one simple example.

Before I get to a simple example, I want to show you a fun example. Not simple, but let me tell you what the problem is. I won't cover that proof.

The problem is shortest common super sequence. This is a problem that comes up in computational biology. Your given k strings. Let's say alphabet sigma. And yes, you're given a number, which I'm going to write l. You want to find a string of length l that's a super sequence of all input strings.

So maybe you're given the DNA sequence of human and DNA sequence of mouse, and you want to know what is the shortest DNA sequence that contains all of the letters of those strings in the correct order. So this is often called an alignment. If you have ACGG-- this is probably not valid, but whatever-- And we have some other guy like CAGGAT.

So I tried to draw those aligned. So then the common super string here is ACAGGACT. This is a super string, meaning I can drop letters from down here and get this. Or, I can drop letters from down here and get this. That was for k equals 2. In general, this problem does have an n to the k dynamic program. But the question is whether you could get FPT in k. Could you get a small dependence on the-- better dependence on the number of strings?

And the answer is no because this is w1 complete. And this is a sketch of the proof. it's a reduction for a multicolored clique. I think this the paper that introduced multicolored clique in fact. It's the earliest one I could find, and this is 2003. It became more popular since 2009.

So you're basically encoding the edges by whether there's a 0 here in between a huge set of ones. And then this is intuitively rep-- and then for every vertex you're looking at, all of the other vertices and encoding whether they have an edge to them. And so if you have a clique, that will be a pattern that you see repeated among multiple vertices. And so you'll end up being able to shrink your longest-- your shortest common super sequence.

But the details are messy. It's a few pages to prove it. I will skip that and tell you why I wanted to tell you about this problem other than its computational biology and useful. Because it's used to prove that flood-it is hard. Time for some fun. So here is flood-it.

You have a grid of colored squares. And in the top left, we have colors. And this is the special square. What I can do is control the color of that square. So for example, I could set it to red and then anything that was in that connected group becomes red. And now I've got a bigger connected group. So now I can make it blue, and then pink, and then red, and then blue, and then red. Let's see how well I can do. And the trouble is I have a limited number of moves.

So this is a model for Ebola virus spreading or zombie infection or pick your favorite. I mean you imagine these are different species-- green. Thank you. A lot easier to play when I have an oracle. Yellow, that's better. I don't know. I'm actually doing pretty well here. I think now I just have to do them all. I made it! 25 moves. Wow! Felicitations!

That's the first time I've won. It's a good demo. Only played a few times, but-- this apparently became quite popular in 2006, a company called LabPixies, since bought by Google. But there's tons of free versions out there you can play. It's w1 complete with respect to number of colors and number of leaves. So I don't know if this is a generalization or specialization, but it's a variation of the problem that's been studied.

It's the only one I could can that talked about parametrized complexity of flood-it. And so this is flood-it on trees. So same set up, but instead of being a square grid graph, I'm on a tree graph. And the root of the tree is the one I can control, and my graph is going to look like-- my tree is going to look like this.

These are going to be my strings. And if I want to be able to solve this in some number of moves, l moves, then I need to-- the sequence of colors I do is a longest common subsequence, super sequence, of these strings because they all have to get to the bottom.

They'll sit there and wait, and so you can essentially drop letters. And that's cool. Happy? This is not literally true because if I have, let's say, a zero and a zero here, both the same color-- So I'm representing letters by colors, then when I play-- if I eventually play here and play zero, I actually get advanced two spaces instead of one. So for that, you need to first map every letter a.

So let's say in string i, we map every letter to a followed by a special character for that string. And so this blows things up. In particular, it blows up l. l grows by a factor of the total length of all strings because of all these things that we add. These are not really compressible, but it means that we alternate between regular characters and special characters. And therefore, we never have two characters in a row that are the same.

So first you take this problem, then you reduce it to the version of the problem that has no repeated characters. And then you can reduce that to flood-it because that's what we care about.

AUDIENCE: Sorry, what's the parameter on the shortest common super--

PROFESSOR: Right. Parameter is-- they're actually two parameters, so this is a fun thing, k and sigma. So two parameters, you can think of that as being parameterized by the sum or just by the vector k, sigma. Basically, we get to assume both of them are relatively small. We could be exponential in both k and sigma. If you put l in there, it would not be hard. But if you put k and sigma and it's still w1 complete.

And so over here it's the number of leaves in the tree is one parameter and the number of colors is the other parameter. So we're fixed parameter tractable with respect to that joint parameter. Yeah?

AUDIENCE: Why is it obvious that you can reduce flood-it on trees to not flood-it?

PROFESSOR: It's not obvious that the grid problem is related to trees. There are hardness results for like 2 by n flood-it. But they're just NP hardness. I didn't see a w1 hardness for 2 by n flood-it. So as far as I know, 2 by n colored is hard-- is open from a parameterized complexity standpoint. Cool. I want to do one more reduction, and then I'll finally define w1.

Dominating Set, this problem is actually w2 complete, so this is even harder. But before we worry about w2, let's prove that it's w1 hard. So we're going to reduce from multicolored clique to dominating set. I have a nice figure. This is a preview of this cool parameterized algorithms book. So I'm going to represent each-- got it right. Yeah, funny.

I'm going to represent each vertex in the multicolored independence set problem by a vertex in the dominating set problem, so vertices mapped to vertices. This is the joy of multicolored cliques. We did all this k duplication stuff once so that we don't have to duplicate our graphic anymore. We just take this graph. We poured it over. Now, what that tells us is that the vi's form color classes.

And what I'm going to do is connect each color class. So this is the set of all things of color one, set of all things of color two. First of all, I'm going to add 2 dummy vertices to each color class just to hang out there. And then I'm going to connect everything in the color class by a clique. Before we only had edges between color classes, but now I want to color--

I'm going to also change those edges. But the vertices mapped to vertices, I add two vertices of each color, and then this circle represents that there's a clique in here, clique in here, clique in here. Now, my goal is to find a dominating set. And so the intention is that you need to choose one vertex from each color class in the dominant set. And that's actually what these dummy vertices are doing.

They are only going to be connect to things in here. They have to be covered by somebody, so that means you have to choose someone in this clique. You have to choose someone in this clique. When you do, it covers everybody in there. And there's going to be no point of choosing these dummy vertices because they're only connected to other things in the clique. You might as well choose these things.

Now, how do I represent an edge in my independence set graph. If I have an edge between say u and v, that's in this color class and this color class. We know they're in different color classes by multicolored clique property-- multicolored independent set property. Sorry, same thing. So I want to not choose both u and v. So I'm going to represent that by adding a vertex here, which must be dominated, and connect it to everyone except u over here and everyone except v over here, the red patches.

So there are edges from this vertex to every red vertex here and here. So what that means is if I don't choose u, I will cover this added vertex. If I don't choose v, I will cover this added vertex. So I will cover it as long as I don't choose both u and v. So it's like I should choose at most one of these two guys, then this will be covered for free. And it has to be covered for free because we just-- if I set k prime to be the number of color classes, which is k, then I don't have any flexibility. I can't choose any of these vertices to dominate.

And so that simulates independence using dominating set, pretty simple and clean. So that gives you a flavor of nice reductions you can do with multicolored clique or independence set.

AUDIENCE: What's two extra vertices?

PROFESSOR: Why two?

AUDIENCE: The two you added, it looks like are not adjacent so--

AUDIENCE: Oh, so we derived them.

PROFESSOR: Right We want them to be there in order to force choosing somebody down here, but we don't actually want to choose one of them. And so we emit that edge so if you choose one, you don't cover the other. Thanks. I think it simplifies the argument. It's probably not necessary because I think you could argue by replacement argument that you won't choose them, but anyway.

Time for some definitions. I wanted to do some fun problems before we got to the somewhat weird definitions. The time has come.

By the way, dominate set is w2 complete, so is set cover. And most w2 hardness reductions start from one of these two. We already know how to reduce dominating set to set cover. You make a set for every neighborhood set in the graph. So this reduction is easy, and that preserves a parameter so just from the same reduction we did two classes ago, I think.

Here is a problem which will look familiar, but under a slightly different name. This is what I might call circuit ones. One's meaning you want to set some number of the inputs to be one, just like our optimization version. In this universe, it is called weighted circuit set. I'm not a fan of weighted because there aren't actual weights here. The goal is to get minimum hamming weight, meaning the minimum number of ones in your input.

So I give you a circuit-- I'll draw a very simple circuit, and it has one output. It has some number of inputs. My goal is to set k ones and get a one out. Did I do it? I did it! ! wasn't even looking. So that's an example of an input and an output to weighted circuit set. The parameter here, k, is the number of ones, one inputs. So the question is can you satisfy a circuit using only k ones. That's weighted circuit set.

This problem defines the class called WP. This is the original definition of WP is all problems that reduce to weighted circuit set in a parameterized reduction sense. It's all parameterized problems that reduce to weighted circuit set.

I lost my hierarchy. But this is even bigger than all the things. So we have FPT is contained in w1. It's contained in w2, et cetera. And then we have WP, and then we have XP. So it's not bigger than everything. It's not bigger than XP. I won't prove this here, but it is true.

Now, I want to specialize this problem. I'm going to simplify it using a notion called weft. So first the depth of the circuit, this is the longest path. That's a normal notion. Then, the weft of a circuit is the maximum number of big gates on an input to output path.

Big gates means let's say more than two inputs. In general, it's more than some constant number of inputs, but two is fine because if you use a bunch of two input gates, you can build a ten input gate. But there's a distinction between constant input gates and super constant input gates. So if you count how many super constant input gates you need in a constant depth circuit, then we get the w classes, w for weft.

w(t) is the set of parameterized problems that reduce to constant depth weft-t weighted circuit set. t is not the parameter. K is still the parameter. k is the number of ones. t is the thing. Let's do some examples.

AUDIENCE: Depth has to be larger than weft numbers?

PROFESSOR: Yes, depth is always larger than weft. So we allow-- so t is a fixed constant. Depth could be 100 times t or something.

AUDIENCE: Then, w in WP is a different w than w(t)?

PROFESSOR: They both have to do with weft, but that's the P is a polynomial. It corresponds to polynomial weft, which is basically unbounded weft. w does not stand for weighted. In both cases, it stands for weft. What is weft? Weft is the opposite of warp. When you're weaving, you've got the warp threads and then you've got this weft thread that goes back and forth and ties the whole circuit together.

So that's sideways, so if you imagine that constant input gates do not need a weft to hold them in, but super constant ones do, then it's how many layers of wefting do you have to do to cover all those things? That's the larges the longest path in terms of counting the-- that's the term, OK? But it's actually fairly useful to think about. So for example, independence set, inputs are up here. Output is down here. I need to choose, for each vertex, whether it's in the independent set. And if I negate them, then I need various constraints.

This is the graph that's being represented. I want to say either I don't choose this or I don't choose this. And then all of those clauses must be true. That's the and of those things. This is big. These are not big. They have two inputs. This has one input. So these are all free. And then we just do one in terms of the depth here where there's one level of big gates. So independent set is in w1. That's a proof that it's in w1. So that proves all the things we've been talking about, except dominating set, is in w1.

Now dominating set, you need two levels. So we have the a constraint. So this is saying that a should be covered. a is adjacent to a, b, and c. I mean if I choose a, b, or c, then a will be dominated by the definition of dominating set because a is adjacent just to b and c. But also if I choose a, then it's dominated. So this is an or of those three things. And a should be dominated, b should be dominated, c should be dominated. So it's an and of all those things.

But if we don't have bounded degree, which we can't because this problem is easy for bounded degree, then we have two levels of big gates. So this problem is in w2. Now, funnily enough, w1 and w2 are the most common things you see. I've never seen a paper about w3. Some statement, I think, about problems that we tend to care about, ands of ors are common. This, of course, CNF SAT is in w2. I should say weighted CNF SAT is in w2. It's actually w2 complete. Weighted three set is w1 complete because things of size 3 are OK.

We can split those into two things of size 2. So in general, order one SAT, you have five SAT. That's w1 complete. And CNF SAT is w2 complete. And I think most of the problems we care about can be expressed as-- this should be weighted. I'm going tow rite w, but I mean weighted. But this is a capital W, different. That's weft. So fine that's-- and most of the problems we care about can be represented as CNF SAT.

I think that's why we rarely get outside of w2. But there are other things there. You could do an or of ands of ors or an and of ors of ands. Now, we know such formulas can be converted into CNF form and only get a polynomial blow up. What's going on?

Well, when you do that inefficient form, you add extra variables. When you add extra variables, you are no longer preserving the weight. It would not be a parameterized reduction. So if you're just caring about satisfiability, that's fine. But if you're considering weighted satisfiability, you want to minimize the number of ones, it totally changes when you convert to CNF. So it matters.

One more fun result, if you look at 2-tape non-deterministic Turing machines-- the reason I had to define 1-tape non-deterministic Turing machines is because they're different from 2-tape non-deterministic Turing machines. These are w2 complete. The k-step version of 2-tape. 2-tapes is you have 2-tapes that can advance independently. Or, you can think of there are two fingers you can independently move on one tape. Those are equivalent, and they give you w2. Three tapes, gives w2.

Any constant number of tapes, you still get w2. And this is another, I think, natural reason why w2 comes up a lot. I have seen problems that are hard for what's called w-star. This w-star is wt for all fixed t. You can think of that as w order a. So it's the same thing. So I've seen problems that don't depend on what t is. They're hard for all of them. But I've never seen a w3 complete problem. Maybe we can think of one.

I've also seen WP come up, and there's also a problem called W-SAT which is if you have weighted formula sat instead of weighted circuits sat, you get w-sat instead WP. I think it's a little weaker because in circuit SAT you can reuse things and in formula set, you can't. And it turns out to give slightly different classes. So things are messier here. But if you just care about whether a problem is fixed parameter tractable, these are all bad.

But the point of knowing about them is you want to know if your problem-- first you should check where it fits in the w hierarchy by just thinking about what's a trivial way to write it down in the CNF style thing. Figure out whether it's w1 or w2. Then, you'll know which problems you should start from. If it's w1, probably want to start from a version of clique. If it's w2, you probably want to start from a version of dominating set. That's why we talked about them here. And usually it's w1 or w2.

Cool. Questions? We'll do more next time. This is a fun area. And next time we'll talk about, in particular, how the exponential time hypothesis relates to all this.