Decision Making

• Download this transcript - PDF (English - US)

The following content is provided under a Creative Commons license. Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free. To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu.

PROFESSOR: Everyone, welcome back to Poker Theory and Analytics. We're very lucky today to have a guest speaker, Matt Hawrilenko. Matt's background is really interesting. He was a Princeton grad who went to work for a Cisco International group for a couple of years, and then he left to become a professional poker player full time. He was considered one of the best limit hold'em players in the world in addition to winning a World Series of Poker bracelet. More recently he retired from poker and is focusing on clinical psychology where he studies in Boston at both Clark University and Harvard. So with that, I'll pass it along to Matt Hawrilenko.

[APPLAUSE]

MATT HAWRILENKO: OK. Hey. So I'm super excited to be here today. And as I understand, I've been told that I'm your second game theory guy. And I guess Bill, Bill Chen, one of my best friends in poker who I've done a lot of work with came in and talked to you guys last week. I'm going to take a slightly different angle than Bill did last week.

So first I kind of just want to get a sense of who's in the room. So my presumption is that there might be sort of like widely varying levels of experience with game theory. So if you just don't mind kind of helping me out, so who here has seen a game theory game before, like prisoner's dilemma, something like that? OK. So most of us. Who here has actually solved a couple of game theory games, like with pen and paper, matrix algebra, whatever? OK. About half. Has anyone actually taken a game theory class? All right. Great.

So I'm kind of hoping this talk is sort of going to be equally useful for everyone, but we'll find out. So Bill talked to you guys about Cepheus last week. I'm going to talk about something a little different than Bill usually talks about. So I'm going to try to talk about how to play good poker. And so if we think of this problem of playing good poker is this big game space. There are two ways that people generally approach it in that they'll take a read-based approach, which is pretty much what everybody does, right? You try to figure out what the other guy has, what he might have, and go from there. You can take a game theoretic approach. So I'm going to try today discriminate a little bit between these two approaches, and I'm going to talk about why, whatever level of poker you're at, game theory is something they can complement your game or just be your game. For me it is my game.

And broadly speaking, the reason that I think game theory is particularly useful for poker goes something like this. So one of the things that I love to do off the table, I do jujitsu. Which if you guys don't know what that is, it's kind of like wrestling where you're trying to choke the guy or hyperextend their arm or something like that. And when you start out doing jujitsu, you start out as a white belt, and you're competing against, rolling with other white belts. And you can learn this whole repertoire of moves. And tons of stuff works against white belts, right? Like, whatever you do, it's probably going to work against white belts.

As you get better or as you start going against better competition-- blue belts, purple, brown, black belts-- a lot of the moves that worked against white belts don't work anymore. Not only do they not work, they can tend to get you in quite a lot of trouble. So I think that the best way to approach this game is from day one, you're not training to be white belts. You're training to be black belts, right? You want to learn the moves that work all the way along against the best competition.

And I think it's sort of the same way with poker, right? So if you think about it, if you play in a home game or whatever with a bunch of buddies, you might have one strategy that works fairly well, right? Maybe you play really tight, you bet a lot with your best hands, and you don't bluff that much. And this works really well because most people don't really have a sense of what's what, of the strength of hands. They overvalue their hands.

So you take that strategy to the Bellagio and you play in even mid limit games with reasonable players, and all of a sudden you're starting to get eaten alive. So I'm going to talk about how game theory can sort of help you avoid this and help you be strong and grow stronger all the way along.

So now if we think about game theory, really there are two audiences that we could be speaking to, right? So Bill came in last week and he talked about Cepheus, this algorithm that solved my best game. And, well, that is sad for me. I'm not going to spend a lot of time dwelling on that here. Rather I'm going to talk about what humans can do, how humans can apply game theory at the table to their game.

And this is for a couple of reasons, but maybe the bigger one is I've had a fair bit of math, but I can't solve the kinds of problems that Bill can solve. But my work has sort of been on taking game theory, taking these concepts, and really using them in a practical sense. So that's what I want to talk about today.

So if we're thinking about this there are, again, a couple angles we could take. We could talk about it from a theoretic perspective. We could solve a lot of games. We could solve dynamical systems, and see how that works. Or we could talk about it in practice. What does game theory tell me? How does it tell me I should think about this situation? Of course, I think the theory practice dichotomy is a false economy.

So what I want to do is I want to spend a little bit a time solving some really, really simple toy games, two toy games. And I want to sort of use that to bridge the theory and practice gap. So I want to take a couple of toy games and then apply them to a real hand of high stakes poker I played. So I'm going to use that hand to motivate how we might apply some of these ideas.

So then finally at the end of the day, we can come up with a list, right? We can think of a rule-based strategy. That's something that learning from Cepheus might give you. It might tell you what to do in every specific situation. But that's really hard. Poker's really big. You can't really remember all of that. So the thing that I really hope you guys take away from my talk today is some training principles, some ways to think about the game which, as you finish this class I guess in a couple of days and move on throughout your poker careers, whatever those might be, some tools that can sort of help you continue working on your game.

So let's just get going. So sometimes in poker we get put in really tough spots, and it can start to feel like we're trying to guess our way out of them. Varying degrees of guess our way out of them. Sometimes we might be sort of making educated guesses, sometimes they might be less educated. But what do we do? What do you do in that really tough spot where you just don't know what's going on?

So this happened to me a couple of years ago in-- this happens to me all the time. But this happened to me a couple of years ago in a World Series tournament that I played. We were deep. We were in the money. And the one thing I want you guys to keep in mind here is I'm playing this hand against a player who is just much better than me. So that sucks. On the other hand, I have aces. So what I'm going to do is I'm going to take you really briefly through the hand and let you know how I was thinking about it, and we'll kind of look at it from a couple perspectives.

So we're both really, really deep. The blinds are 12 and 24,000 with a $3,000 ante, and we're six handed. So I raise two off the button with aces. I raise a little bit, he calls. Flop comes, king, jack, eight. He checks, I bet, he calls. It turns a five. He checks, I bet about 2/3 of the pot. He calls. And then the river's a king. And then he reaches for his chips, and I feel good. I'm like, oh, money. And then he bets just about $1.1 million to a $700,000 pot. And now I'm like, I don't know. I have no idea. So he's betting I'm happy. I see the amount. I'm confused.

So here's how the hand looks. So I'm sitting there and I'm just trying to get a sense of his range, and I'm thinking, like, bet of 1.5 times the pot. What does that mean? What kind of hands is he betting so much with? Why isn't he putting me all in? Why is it a little below all in? Is he trying to entice me? Or maybe he's trying to save $300,000 if I fold. All these thoughts are going through my head.

So before we talk about what I did, I'm actually curious. Are you guys happy with the information I gave you? Is there more information that you want? What do you want to know?

AUDIENCE: How big is the gap between getting out now and getting out next?

MATT HAWRILENKO: Ooh, good question. So let's just say it's pretty linear at this point.

AUDIENCE: OK.

MATT HAWRILENKO: Anything else you want to know? Oh, you want to know lots. What do you want to know?

AUDIENCE: Has he shown down big bluffs?

MATT HAWRILENKO: Has he shown down big bluffs? Good question, we've played with him a little bit, and I don't actually recall. But he's certainly capable of showing down big bluffs. I fancied myself a pretty good player at the time, and he is someone that I would consider one of the top players in the game. Anything else? Anyone else? Any questions? Right.

And so it sounds like what you're trying to do is you're trying to get a sense of, what is his range? Does he have a lot of bluffs in this spot? Does he have any bluffs in this spot? And I think that's how pretty much most of us tend to approach the game. It's most natural way to approach the game. Is he bluffing, isn't he? Does he have too many bluffs here? Out of curiosity, does anyone want to stare him down for tells? Nobody wants to stare him down. You might want to stare him down. This guy wants to stare him down.

[LAUGHTER]

This guy definitely wants to stare him down. This guy wants to stare him down for sure. By the way, if you get this look, good work. It means you've done your job. It's uncomfortable. But I want to tell you exactly what he's thinking right now. What he's thinking goes something kind of like this.

[MUSIC PLAYING]

But it can feel like this, right? Tough players can put us in spots where we're just thinking about monkeys clashing symbols. And when we don't have a repertoire, I think, that we start reading really heavily into small signals, right? We're trying to figure something out. So generally speaking, my view, there are some tells that are OK. There are some tells that kind of work against weak players and less and less against better players. But these are small signals in a whole lot of noise. Very small signal in a very noisy environment. Certainly not something to build a career on, right? In the big games, you rarely see tells. You just don't see them enough for that to be profitable.

But actually some people do build careers on them. So FBI interrogators build careers on these kinds of tells. So how do they do? Well, we actually have some data in that spot. There have been a whole series of studies where basically the paradigm is you bring in some FBI interrogators, and then you bring in some random people from the street. And you have them watch someone interview a person. And then at the end of the day you have to figure out, is this person telling the truth, or is this person lying? How do you think they do?

AUDIENCE: Same.

MATT HAWRILENKO: Same? Yeah. Same in every category. So everyone is basically chance, or maybe like 53%. So they do exactly the same as the random people off the street, these body language experts, except for one difference. The one difference is they are way more confident that they are right, OK? So you ask them their level of confidence, and most people are like, eh, I don't know. Like 50%. The interrogators are, like, 90% sure that they're right. So they're not alone, right?

So I don't know if you guys have heard of self-assessment bias. It's one of my favorite biases. So one more study. So it was sort of motivated by this old study of GE engineers. It's like 30 years ago. And what they do is they ask, OK, of all the engineers at GE, where do you rank? What is your percentile rank of everyone? And they ask basically everyone. So you would think these being engineers, engineers being math guys, that they have a good sense of percentiles. You'd think that they might get this right.

The average engineer ranks themself right about at the 80th percentile. Of all the people they asked-- they asked, like, maybe a hundred of them-- of all the people they asked, two ranked themselves as below average. That is my favorite data point from this study. Two ranked themselves as below average. Should be about half. And they're not alone too, right? So poker players also famous for their self-assessment bias.

And one of the things that I think sort of feeds into this, at the beginning of Rounders-- I haven't seen this movie for, like, five years. I don't know if you guys have seen Rounders. But at the beginning of Rounders, Matt Damon's character quotes this poker book. It's one of my all time favorite quotes from a poker movie which, I guess, isn't saying much. But gives this quote, "Few players can remember the big pots they've won, but every player can remember with remarkable accuracy the outstanding tough beats of their career."

And I think it's these sort of memory biases that feed into our self-assessment bias, right? So you walk into the Bellagio, you see a table of all pros. There has to be self-assessment bias here, right? There has to be. So don't be that person.

Let me circle back to my point. I'm going to tell you about my favorite poker hand of all time. Of every hand I've ever played, this is my favorite for a number of reasons. So I'm playing a tournament in the World Series. This is a big tournament. And we're on the exact money bubble. And what that means is that the next person to bust out gets nothing. Everyone else is guaranteed something, couple dozen bucks. I don't know.

So I'm sitting there, and there's this guy at my table who, as we approach the money bubble, he's so excited to be there. It's his first World Series tournament, he's about to make the money. He's calling his girlfriend and his buddies every hour telling them, yeah, I think we're almost in the money. He's so excited. And then it gets to the point where, like, 40 minutes before the bubble, he just leaves. He doesn't want to play a hand. He just leaves the table so he doesn't have to bust out.

So he comes back. And it's now the actual bubble. He's to my direct left. I'm in the small blind, he's in the big blind, and it folds around to me. And at this point, I feel like there are just dollar signs in my eyes, right? I'm so excited. So I look down and I see four deuce off suit. And we both have medium stacks, but I have this hell of a read on this guy, so I shove all in.

He says, I knew you were going to do that. I call blind. He calls without looking at his cards because he knew I was going to shove in. It's like the strongest [INAUDIBLE] I've ever had. He flips over his hand. He has nine deuce off the suit. He has me dominated with a terrible hand.

Let's think about this. How terrible does my read have to be? How far off does it have to be? What has to happen here for him to call with nine deuce off suit. Not only does he have to not care about bursting out, busting out on the bubble. He has to not care so much that he won't even look at his cards. If he looks at his hand, he has to fold. So he proceeds to win the hand. And they count us down, and he has me literally covered by one or two chips.

And I walk away thinking I have to go home. We rent a math house every year out in Vegas. Me and Bill Chen, Jerrod Ankenman, Mike who's sitting right over there, and a bunch of other of us. I have to go home and I have to tell these guys what just-- Kenny, who's right here-- I have to tell these guys what just happened, and I still don't hear the end of it.

So what does this mean? So I realize the beginning of this talk sounds a little bit like a commercial for like, just be humble. Don't be an idiot. But I think it's a little more than that. Here's the takeaway. Any time you try to divine your opponent's strategy, she can do the same thing back to you. And this happens in really obvious ways, like when my super tight guy all of a sudden doesn't care.

But it also happens in subtle ways, right? So a lot of times we think about pot odds. So you might be sitting there thinking, OK, I'm getting two to one. Do I have the best hand here at least one in three times? Very natural way to think, right? But now you're kind of trying to do magic. That's one of my favorite quotes of all time, by the way, from Harry Potter. And the trouble is, the other side can do magic too. I'm trying to figure out what their distribution is and respond to it. They're trying to figure out what I think their distribution is and shape it around that. And all of a sudden, we're playing this leveling game. And we're both kind of trying to do magic, right?

So in the hand that I showed you guys in the beginning, I could be sitting there and trying to figure out exactly what my opponent's range is when she bets one and a half times the pot. I can try to figure out why he's not putting me all in. I can try to figure out a million other little things that are going to help me get a little more sense of this range, but I'm trying to do magic. And I think that's kind of our go to move when we don't really have a repertoire, right? And this is going to happen more and more-- well, it happens with really bad players, bu that's OK-- but it's going to happen more and more with good players.

They'll take you to a place where they have this paved road. They've been there before. They take people to these weird places. And you haven't. And what do you do? So what I would say is forget their hand. Forget their range. Don't think about it.

So one of my favorite people in the world, Jerrod Ankenman, he's the one who wrote Bill's book and let Bill put his name on it, by the way. He has this quote, "If I truly played optimally, I could write down my entire strategy on a piece of paper, what I would do in every single situation, and I could give it to you and you couldn't beat me." That's what we're trying to do here. So how do we get there?

Well, heads-up limit hold'em turns out to be a pretty big game. It has about a quintillion game states, and that's the smallest poker game that we really play for money. So we can do a couple things, right? We can try to do some, like, sweet programming like the team out in Alberta did. And that's great. You know, for example, humans play chess, right? And computers now crush them, but human learning is really strongly aided by computers. But you can't memorize every position. You can't memorize every line. You have to know what's going on underneath the hood. And these algorithms, these programs like Cepheus, they will give you the strategy. They will not tell you why. And as humans, we need to start to understand why if we have any hope of carrying these strategies with us and actually playing them in real life.

So they're a black box, right? So we can solve the games. But solving the games does not get us out of the woods. So if we want to wrap our puny human minds around it, we have to be a little more clever. So we're going to look at a couple of toy games here, really simple ones that can start to help us wrap our minds around how to behave in these situations.

So the first one is a clairvoyance game. And a clairvoyance game is basically when either one or both players have complete game state information. So what would you do if you lived in a world where you always knew your opponent's hand and he knew that you knew? That's the idea of a clairvoyance game. So we're going to look at a game called Coin Flip Clairvoyance. And the game goes like this. You can and should play it for money by the way.

So each player antes a dollar. There are two players. And then you flip a coin. If it's heads, you win. If it's tails, your opponent wins. However, there's a round of betting. Only you see the coin after the flip. Then you can bet or check. So you choose to bet a dollar or check, and your opponent can only check. If you check, they have to flip over hand or call. If you bet, they either have to call a dollar or fold, OK? So you know if you win or not. Your opponent doesn't know if they'll win a showdown. That's the idea of this game. Make sense? Yeah OK, easy game.

So how do we play? So scenario one. Flip the coin. It's heads. What do we do?

AUDIENCE: Bet.

MATT HAWRILENKO: Bet. OK. Scenario two. It's tails. What do we do? So there are two main questions. How often should your opponent call, and how often should you bluff? So how do we solve this? What we want to do is we want to call enough to make your opponent indifferent to bluffing or giving up. That's what your opponent should be doing. So to do that, we're going to set the expectation of bluffing equal to the expectation of giving up. So the expectation of bluffing is just this, right? It's the pot times the amount they fold, which is one minus the proportion of the times they called, right? That's how much you win when you bluff.

So now how much do you lose when you bluff and get caught, right? It's the amount you bluff, which is going to be one unit, times the amount you get called. So we can just sort of do a little simple algebra, and it'll reduce to-- you should be calling p over p plus 1 of the time, where p is the pot, right? So if the pot is two units and you're bluffing one unit, we should be calling with a proportion of two over three. 2/3 of the time with our kings. That is how we make the opponent indifferent to bluffing or giving up. OK?

So a couple of things to note about this before we push on. So what happens here as the pot gets bigger? Do you call more or less?

AUDIENCE: More

MATT HAWRILENKO: More. Yeah/ So as the pot gets bigger, this asymptotes to 1 which makes sense, right? Most of the value's already in the pot. So there's more money in there. You have to protect against being bluffed at more. Totally intuitive with poker, right? So how often do you bluff? Well, you want to set the expected value of calling equal to the expected value of folding.

So the expected value of calling here is just-- we're going to use the ratio of bluffs to value bets rather than a percentage. It just works out nicer. So the ratio of bluffs to value bets, so how frequently are you bluffing, times the pot plus one because that is what they win when they call. So bluffs value that times the pot plus the unit that you bluff. And then what do they lose when they call in their wrong? They lose the amount you bet, so that's going to be one unit, the value you bet. So you're going to bluff p plus 1. So you're going to bluff 1 over p plus 1 of the time.

So now what happens here? So as the pot gets bigger, what are you doing? Bluffing more or less?

AUDIENCE: Less.

MATT HAWRILENKO: Less. Yeah. Is that counterintuitive?

AUDIENCE: No.

MATT HAWRILENKO: Is it? I don't know. It was counterintuitive to me because I'm like, oh, there's more money in the pot. But what it means is there's more money in the pot, so I don't really need to bluff very frequently to make sure I get value, because the values in there, and my opponent is calling more of the time. So the bigger the pot is, the more my opponent calls, right? And what I'm doing essentially is I'm bluffing so I can get value from the time that I'm winning. Yeah?

OK. So we can actually generalize this too, right? So we can generalize it to no limit games pretty simply. So we've sort of flipped things around here. So here the pot is 1 and s is the proportion of the pot that you bet, so you're going to be calling 1 over 1 plus s, right? So if the pot is two units, s is one unit, so s would be 0.5, right? 1 plus 0.5 is one half. And you'd be bluffing s over 1 plus s, right? So you're going to be calling 1 minus the bluff ratio. Sometimes we call the bluff ratio alpha. 1 minus alpha.

But we can actually calculate a value for this game and for all toy games. So Bowling and their team with Cepheus they calculated the value of having the button in limit hold'em. We always kind of knew it was good, but they calculated precisely just how good it was. And if you're looking for a job at a place like [? Sig ?] or somewhere else in finance, you actually probably should just calculate the value of this, because you're going to be getting interview questions like this.

So again, what we're taking here is the larger amount your opponent gets, the less frequently you have to call. Right? The more frequently your opponent bluffs, the more frequently she has to value that, right? So as a bluffing region gets larger, the value betting region has to get larger with it. So in a coin flip game it doesn't make sense to think of regions, but in poker it's going to.

So a question I get here a lot is, what if it's not a repeated game? What if you're playing it just this one time? Or what if you're at a table with a player that you'll never play with again? How do you play that? And the answer is that it's a repeated game. So I have a feeling this will be intuitive for you guys, so I'll do it quickly. But I want you to imagine a scenario where you're going to play the Coin Flip Clairvoyance game a thousand times against different players. So against a thousand different players, right? And suppose you take the position, since my opponents can't learn from my past, I'm going to bluff 100% of the time.

So what's actually happening here? You can sort of think of each opponent as a random sampling from the distribution of possible strategies that are out there for the Coin Flip Clairvoyance game. So some of them will fold too much, and you will own them. And then some of them will call too much, and they will own you if you're bluffing all of the time, right? So even if you haven't seen this person before, even if it's your first hand of poker against them, it is a repeated game, OK? So get that, it's not a repeated game notion out of your heads. It's bad for business.

So taking like a half a step back, what have we learned about this so far? OK. So the Coin Flip Clairvoyance game, it's not about just value betting or just bluffing. It's about the combination of the two. We're also trying to maximize the value of our entire set of hands, right? Because what happens? So suppose our strategy is, we're going to bet every time we have heads, we're never going to bet when we have tails? What's our opponent going to do?

AUDIENCE: Fold to every bet.

MATT HAWRILENKO: They're going to fold to every bet. Yeah, exactly. So we sort of can calculate this ratio where now they don't do so well if they fold to every bet. And so what's cool about this is the math in this game is very simple, right? It's not hard. But it buys you a lot. It buys you a lot of intuition about poker. Some really useful concepts.

So I want to move on. This is probably my favorite of all the toy games, and there are actually a million versions of it. We're going to do the simplest one because I think it sort of gets everything that you need to know, more or less. So this is an ace, king, and queen game. It's an incomplete information game. So each player antes $1 and is dealt one card. So if I get delta ace, my opponent only has the king or the queen. They can't also have an ace. So then you can check of bet and your opponent, just like the Coin Flip Clairvoyance game, can only check or call or fold. They can't bet if you check.

So this is going to be our first mapping, our only mapping, of a toy game that resembles real poker, right? Now we have a real range. Again, I think that you probably can and should play this game for money. I think there's a real difference between what we're about to do, which is solve it and sort of get it intuitively, and actually get it experientially. So go forth and gamble.

But what do we do? So case one. So you get the ace. Are you going to check or bet?

AUDIENCE: Bet.

MATT HAWRILENKO: You're going to bet. Yeah. Now your opponent bets and you have an ace. What are you going to do?

AUDIENCE: Call.

MATT HAWRILENKO: You're going to call. All right, good. And now your opponent bets, and you have a queen. What are you going to do?

AUDIENCE: Fold.

MATT HAWRILENKO: Yeah. Hey, yeah. OK. So it seems trivial, right? These first three cases seem really trivial. But an important thing to note is that they are dominant strategies or dominated strategies. So a dominated strategy in game theory, for example, calling with a queen here would be dominated. A dominated strategy is one where the decision has equal or lesser-- strictly equal or lesser value to another decision. If I call with a queen, I cannot win. I just lose money. That has strictly lesser value than folding, OK? So dominated strategies, important concept.

So how about this one. You have a king. Check or bet?

AUDIENCE: Check.

AUDIENCE: Check

MATT HAWRILENKO: Check.

AUDIENCE: Split.

MATT HAWRILENKO: Split? OK. All right, well, I actually want to see where everyone's in. So we're going to have three options. Who wants to check? All right, who wants to bet all the time? Who wants to bet sometimes? OK. See, I tricked you. This is also a dominated strategy. So what happens if you have a king and you bet? What is your opponent going to do with an ace?

AUDIENCE: He's either going to call with an ace or fold with a queen.

MATT HAWRILENKO: Exactly. Your opponent-- because the ace and the queen, dominant strategies, right? Your opponents always calling with an ace, always folding a queen. So betting with a king here would be a dominated strategy. Strictly dominated by checking. How about here? Now we have a queen. What do we want to do? Check or bet?

AUDIENCE: Bet.

MATT HAWRILENKO: Check, bet, exactly. We want to bet some of the time. Right? And we'll go through [INAUDIBLE]. And now we have a king and our opponent bets. What do we want to do?

AUDIENCE: Mix.

MATT HAWRILENKO: Mix. Good. You got the idea. Does anyone have any guess as to how the mix might break down?

AUDIENCE: Depends on his proportion of bluffing with the queen.

MATT HAWRILENKO: OK. But suppose we're trying to solve it. So it depends on his proportions bluffing with the queen. Exactly. And let's get tighter. How do we solve it, right? So if we have a king, our opponent has an ace half the time and has a queen half the time. They're going to bet all the time with the ace and sometimes with the queen.

Turns out in this game that it's going to be about the same or exactly the same. She should be calling 1 over 1 plus s, s being the fraction of the pot. So you should be calling with 2/3 of hands that beat a bluff. So the hands that beat a bluff are aces and kings, not queens, right? So if we're thinking about it this way, before-- eh, no. I'll show this first.

So aces are going to represent 50% of the hands that beat a bluff, right? Because you're going to have aces 50% of the time and kings 50% of the time. So calling with aces seems better than calling with kings. So we're going to call with all of our aces. So now we're up to half, but we need to get to 2/3, right? We want to be calling 2/3 of the time, kind of per our formula. So we're calling with all of our aces, and then a third of our kings times having a king half the time, that's another sixth, right? So all of our aces and a third of our kings.

So how is thinking about it this way different from thinking about it using pot odds? So for pot odds we're trying to figure out, what does this person have in this situation? So I'm sitting here with a king with pot odds, and I'm thinking, am I ahead at least a third of the time here? I don't know exactly. But I know that I can try to make my opponent indifferent to bluffing or calling. So I'm thinking about what I'm doing with my whole range of hands. So yeah, OK.

So these are the two observations we had from the Coin Flip Clairvoyance game. So adding one. One thing that we're noting here with the ace, king, and queen game, what are we doing? We're sort of implicitly mapping three different types of hands. Value hands, bluff catchers, and bluffs. And the big thing here is your strategy for what you do with one hand determines your strategy for other hands, all right? I'm definitely calling with the aces, so I need to call with some kings, right? I'm definitely betting all my aces, so I need to bluff with the lowest-- like, the worst part of my distribution, right? That's the part that's going to gain the most.

So a more subtle thing that I think is super important and is going to play into sort of the last half of this talk is if I am playing a hand differently from you, I should do different things with other hands than you should. Say for whatever reason I'm only betting half of my aces but I'm still betting a third of my kings. Whoops. Now I'm out of whack, right? Now I'm out of balance, and I'm going to lose more in this game by being out of balance. So your strategy for one hand determines your strategy for other hands. That's the whole key here, OK?

So to sort of summarize what we've done so far is the Temple of Apollo. This is where I'd like you to go to see the oracle. Like, you want to go and get a prediction, this is where you would go. And walking into the Temple of Apollo back in the day in ancient Greece, I'm wondering if they knew a thing or two about game theory. So there are three inscriptions above the temple. The first one is know thyself, right? Know your own hand. Know your own distribution. The second is nothing in excess. Play with balance. So know thyself, nothing in excess. And the last one is make a pledge and mischief is nigh. Yeah, it's really a real stretch to make that one work, so we'll just leave it at mischief. Mischief can sometimes be good.

So let me be very clear on how important I think this concept of knowing your own hand, knowing where you are in your own distribution is. I think you should not think about anything else in poker until you have bought and paid for a house by knowing where you are in your own distribution, in shaping it to be balanced. Don't think about anything else. Everything else is just window dressing compared to this concept.

So here we are again. So we're going to go through this hand, the one that I told you about at the beginning of the talk, and we're going to try to read our own hand. So again, we're playing against this player who's better than us, some stuff happens, what do we do on the river? How do we think about that from a game theoretic perspective?

So there are kind of three ways again that I alluded to in the beginning that people might think about it, right? So the first one is my hand versus your hand, OK? Well, I have aces. What do I think he has? King queen? King 10? Maybe he has queen 10? Maybe he's bluffing? Maybe he has queen nine? What is his most likely hand? And probably as most of you have already realized-- so Kevin told me that you'd all played at least 100 tournaments so far, I'm guessing some a lot more-- it's really hard to put someone on a particular hand, not particularly useful.

So the next thing you might try to do is my hand versus your distribution. How are my aces doing against all the hands that you might have given the actions you've taken? I can't really put you on one hand, but I can look at your actions and see what sort of distribution they might suggest.

And then the last one is my distribution versus your distribution. And this is I can look at your actions and I can look at my actions, and I can try to shape my actions such that they maximally exploit your actions. So it's not so much about what I'm doing with my aces. It's about what I'm doing with all the hands that I'd have, where I just happen to have aces here. And this is the style that's most complimentary to game theory.

So again, we're on this river and with pot odds with my aces I might be thinking, am I good at least a third of the time here? But if I'm doing that I'm trying to do magic, because I'm trying to figure out exactly what he has. And I don't think we need to do that here. Right? What I want to be thinking is, well, how much of the time do I need to call to make my opponent indifferent to bluffing? Any guesses? Yeah?

AUDIENCE: Like 40% of the time?

MATT HAWRILENKO: Yeah. So something like 1 over 1 plus s. So how do we get there? So this is the slide-- guys, if you pay attention to one slide this whole talk, this is the slide to pay attention to. This is the slide where we map the ace, king, queen game to actual poker. So the whole idea of poker from a game theory perspective is we're going to try to make bluffing zero EV for our opponents. So we're going to call with the proportion of our hands. We're going to make bluffing zero EV. So in real poker, games that allow raising, that means we could potentially be raising. So we want to be continuing 1 over 1 plus s at the time, at least calling, pondering arrays.

So what do we see? So if this is the ace, king, queen game, we're going to map it to real poker this way. This is like our 99th percentile hand, the very best hand we can have in this spot, right? So on that board there were two kings on the board, so a 99th percentile hand would be like quad kings, right? Would be four of a kind. This is our worst hand, the very worst hand we could have in this spot, I don't know. A four deuce.

And so what we're going to do-- so again, these are the hands that are the very best at showdown, very high EV. Very worst at showdown, very low EV. So we're going to be calling or raising 1 over 1 plus s of the time. So the question is going to be, what is our distribution here? What is 1 over 1 plus s? Our bluffed value ratio is going to be s over 1 plus s of our worst hands, right? These are the hands that gain the most value by bluffing, right? The ones that are going to do the worst at showdown. Well, not for us in this spot. I'm going to talk about that later. Let's leave that for a second.

But those are the hands that my opponent should be bluffing, right? So if we think about our value betting range, I might have a different value betting range than you. And that has implications for how we play differently than each other. So the wider my value betting range is, the more hands I need to be bluffing. As this region expands, this region expands. So if my opponent is value betting more hands, they should also be bluffing more.

So when you see some of the very best no-limit players play these guys who are just complete animals-- you're like, how are they bluffing that there? How are they calling that there? How are they making this value bet so thin? This how they do it, right? If they're value betting a lot, they're also bluffing a lot. So you have to call them more. So the more they bluff, the more they have to value bet. This is one of the places where a lot of beginners just get way, way out of whack, because bluffing is-- it starts scary and then it gets sexy, and then it gets something in between. So the whole idea, don't let this get out of whack. And again, the larger amount that my opponent bets, the less frequently I have to call. The less they bet, the more frequently I have to call.

So we're about to our own hand. Let's keep these couple of things in mind. So again, I think reading your own hand is the most important skill in poker, and it's because what you do with part of your distribution shapes what you do with the rest of it. And so what we're about to do is we're going to go through and we're going to sort of make some frequency updates on each street.

So we're going to do two updates. We're going to sort of update what our hands might be given the cards that have come out. So there's a card removal effect, right? If an ace comes out, I'm a lot less likely to have a pair of aces because there are fewer combinations. And then we're also going to account for the actions that we take. So let's just do it. It'll be clear.

So, OK. So I've opened two off the button. So here's some kind of reasonable range for opening two off the button. So we've gotten rid of all the hands that aren't here. This is what I might have right now, OK? So you don't really need to pay attention to the specifics here, but flop comes, king, jack, eight. So now I'm going to update for card removal. So all the hands in orange are the frequencies that have changed, right? So eights, I could have had six of them before the flop came, but now that there's an eight, I only have three of them, et cetera. So our total combination down the bottom here has gone down, right?

So now what happens? Well, they check, I bet. So what hands am I betting here when my opponent checks? Well, I'm getting rid of some of them. I don't know if they're the right ones or not, but I'm getting rid of some pairs, some bottom pairs, some gut shots. I might not be betting those here, right? So the hands in white are all the hands I'm still betting. And again, our frequency is coming down. Our distribution is narrowing. So the turn comes. It's a five. We get rid of some hands with fives in them. I bet 2/3 of the pot. So we get rid of all the hands in this distribution where I'm not betting 2/3 of the pot.

And really quickly, when you're thinking about reading your own hand-- and I'm going to say that the most important time to be doing it is probably off the table-- when you're thinking about it, you should be really thinking about every street, OK? So OK. So these are the hands that I'm value betting. Turns out I'm value betting about 94 combinations here. How's my proportion of bluffs? So I'm value betting 94 combinations. I have a total of 120. So that leaves what? 26 hands.

So how often should I be bluffing? Or-- hm. 96. I'm value betting 96 leaving 24 bluff hands. So how often should I be bluffing? OK, so 96 times-- what's that? Is it like 1 over 1 plus s, s over 1 plus s, s over 1 plus s? So times 0.4-- that's the amount I'm betting, I'm betting 40% of the pot-- times 1 plus 0.4. So 96 times 0.4. Yeah. 96 times 0.4 over 1.4. All right.

So I should be bluffing about 27 hands. What do I have? 16, 24? Huh. Did pretty well. I'm happy with that. Pretty good shape. I don't know. I don't know about all the other actions. We can argue about what I'm checking behind, but we don't want to over think it. So generally speaking, I want to be checking in on each street. Like, ooh. Am I balanced here in the way that I should be balanced?

So the river comes a king, and now we have fewer combinations. So what do I do with my aces? So again, s, the bet size, he bets $1,080,000 into a $720,000 chip pot. So s is 1.5. I should be calling 1.5 over 2.5, 40%. That's 40% of hands that beat a bluff. So what does that look like? So if we're thinking about our calling region, what I've done here is I've just sort of taken all my hands and I've ranked them, and so kings represent my top 1%, king jack. Now we're at 9%. This is like the cumulative sort of frequency distribution with my hands ranked. And I'm saying here in this distribution, queen jack is the worst hand that still beats a bluff. We can argue about that, but rough guess.

So first nothing in excess, calling 1 over 1 plus s of a time. Where are we? Ooh. This is surprising to me. This felt like a tough decision, right? So we're actually at the 62nd percentile here. So one question is, so from this, what should I be doing? Should I be calling or folding? Folding. Yeah. So it looks like I'm calling with my king queen. I might be folding king 10 maybe. We can talk about that a little bit. But aces seem like a pretty clear fold here. Suppose my opponent thinks that I'll fold good hands like aces, and so he's like, mm. I'm going to bluff 90% of the time here. Is he exploiting me? Seeing head shakes. He's not exploiting me, right? So why not? Yeah?

AUDIENCE: Because you win on his bluff. Such a big question at the time, it makes up for the pot that you're losing.

MATT HAWRILENKO: Yeah. My distribution is really strong here, right? Of the hands that beat a bluff, like half of them have trips. That's crazy to me. So no. So he's not exploiting me, but we will talk about exploitation in a minute.

So we solved it, right? So we fold aces. We even fold king 10. So one question is, do we actually want to have a distribution in the spot where we have to fold trips? And if that feels kind of bad, well, two possibilities. One it just feels bad, or two, it might mean we sort of screwed up on the way here. It's not the worst. It's not a four flush board. There are no straights on it, right? Some folding trips here, that feels kind of bad.

So what might that mean? A couple of things, right? So one thing we could do is I probably want a distribution where I don't have to fold it. And again, it depends a little bit on bet size, but this is a pretty reasonable rule of thumb when you're thinking about shaping your play off the table. So one thing that we can do, we can add some hands in from earlier. Ooh, maybe I should have played the turn a little bit differently. Maybe I should have played preflop, maybe I should have played some more hands, right? So we can start to expand our range. Right? Now we're calling with most all of our kings.

We can actually construct a distribution where we have to call it aces, right? If we're playing way more hands and if we're value betting way more hands throughout, all of a sudden our distribution is wider, right? And now I should be in the same spot with the same hand and do something differently because my strategy to get here was different. And again, that's the same idea. You see these really good players making these crazily thin value bets, and this is why. They're bluffing a lot so they value bet a lot.

The main idea that I want you to take away, if we sort of think of a principle, if I find myself on the river and I have more medium strength hands, I have to call more with medium strength hands, otherwise I can get exploited. Suppose we were deeper and our opponent bets into us, and we have a bunch more chips. Which hands should we be bluff raising from this distribution?

So most the time, if I'm just straight bluffing, I want to be bluffing with the very bottom of my distribution because that's the part the gains the most when my opponent folds. If I'm bluff raising, it's a little different. If I'm bluff raising, I want to think about, OK, so what is the set of hands I would fold? What are the very best hands in that set that I would fold? And I may as well choose those, right? Because it seems like those hands have more value, so if I'm going to fold them anyway, it's a dominated strategy to bluff raise with hands that are weaker than that, right? So if I'm bluff raising, it should be really with the very best hands that I would otherwise fold, not with the very bottom of my distribution.

So what do we see here? So first off, we want to check for balance on all streets. That's the big thing we take away. We can argue about little bits of distributions, but really, we're not solving to the second decimal point here, right? Second, we can look at this board all day. We can look at this king jack eight five king board all day. Unfortunately, that's not going to be the board that comes every time, and if we just look at this board all day, we're going to start to overfit our strategy a little bit to this board.

So what I suggest you focus on is fixing the glaring errors, and there will be glaring errors, places where your distribution is way, way imbalanced. I still find glaring errors when I play poker, and I've been trying to do this for a little while now.

Another thought which is not quite so obvious from what I've said so far is, you don't want to needlessly bifurcate your distribution. How do you bifurcate your distribution? Well, suppose preflop, I raise some amount with some set of hands and in a different amount with a different set of hands. Ooh, right? All of a sudden I started off here with 310 combinations, and I got down to, I don't know, 60 or something. But now all of a sudden, I'm starting off with half that. I'm starting off with 155, in this game tree gets smaller really, really quickly.

So I'm not saying don't do it. But I'm saying if you do it, A, have a really good reason for doing it. B, be really careful. Be really careful of betting different amounts with different hand types. I basically don't. I will bet different amounts based on the texture of the board. So if boards are more [? drawy ?] I might tend to make my bet sizes larger earlier. But I won't bet different amounts with different hand types because I think it the possible gain is so small and the possible loss is so big.

And the last thing that I found pretty cool about this was when I started really, really spending time trying to read my own hand, I'd start to find these consistent situations where I would get really imbalanced. And then when you want to start to think about moving on to exploitive play after you have paid off a house with your poker winnings, you want to maybe identify those spots in your own opponents, right? If you're getting imbalanced there, other people probably are too, so be ready for that.

So what do I want to say here? So I guess we've ignored a few assumptions of this model, right? The biggest assumption in a toy game like the ace king queen game or the Coin Flip Clairvoyance game is that distributions are symmetric. If I'm applying the ace king queen game to my river situation here, I'm implicitly assuming that my opponents and I essentially have an equivalent distribution of aces, kings, and queens. That's not always true in poker.

So generally I'd say that this sort this mapping is actually fairly robust, but at the same time, you have to be aware of situations where your distributions aren't symmetric, right? Like if you raise under the gun at a ten handed table and the big blind calls, your distribution is way, way stronger. It's not symmetric. So you're going to have a different set of actions. However, as you progress throughout the hand, distributions sort of tend to become more and more symmetric. So generally speaking, it's a model with some assumptions which are meh. But in my experience, it holds pretty well.

So another thing I haven't talked about today is I haven't told you which hands to value bet, right? So I've told you the calling proportion, bluffing proportion. I haven't said anything about, how do you choose what to bet? And there are game theory games that can kind of give you insight into that. You can get insight into that from Bill and Jared's book.

But I think that what's most important here is reading your own hand is the thing that I think can integrate into everybody's game. Whatever hands you currently value bet, it can work for you. Right? And it can sort of work equally well for the right player as it can for the very loose player. So the reason I've decided to talk about the reading your own hand approach is this is a very flexible approach that can work for a lot of people in a lot of situations.

And, you know, as you get better and you start to play more and more hands-- or fewer and fewer, depending where you fall on the spectrum-- but generally as you start to play more and more hands, what you're doing here, thinking from sort of like this game theoretic perspective is you're moving towards hands that are closer to threshold hands, closer to zero easy. So adding in the nth hand has a much smaller effect than adding in that, like, fifth hand, right? Or if you think about all the good hands in poker, right? So you're always going to play your jacks, you're always going to play your 10s, right? Adding in that nine six suited has a much smaller impact on your EV than adding in the 10s, the nines, the eights. Yeah, that's all I want to say there.

So in the last few minutes I'm going to talk really briefly about exploitive play. So exploitive play. So if we're looking at the top, our best hand's at showdown, our worst hand's at showdown. And this is about where our value betting threshold is. The way to exploit from a game theoretic perspective isn't to say, ah, I know he's bluffing! I call a blind! Although apparently that can work pretty well sometimes.

The way you do it is you expand to the margins. So if normally I'm value betting here and my opponent-- or normally I'm calling here and my opponent bluffs way too much, I might expand to the marginal calls. That's one thing I might do. If they don't value bet enough, I might contract the marginal calls, right? But again, the ones that are very close to zero EV, I'm never making the big fold here.

If they fold too much, I might expand to the marginal bluffs a little bit and contract the marginal bets a little bit. If they call too much, I'm going to expand the marginal bets and contract the marginal bluffs. So what that means is you get away from, I think he's bluffing 80% of the time in this situation, to-- in this exact situation, and I'm going to catch him-- to, hm. This person seems a little bluffy, so I'm going to shape my distribution just a little bit around that. OK.

So the other idea about exploitive play is as your read grows stronger, as you have more confidence, you can expand to the margins, right? So a little read, you move the margins a little. Big read, you move the margins more. For example, well, OK. So with our example hand, we have a margin right about here, but we can start to move it wider as our read grow stronger that our opponent might be bluffing. But again, we don't want to move it wider with just the hand that I happen to be holding right now. We need to be thinking, how strong is it, and how wide am I comfortable moving it? Yeah.

So if we're thinking about exploiting with my sort of, like, shamefully exploitive hand, what did I do? Well, it wasn't quite a 1 over 1 plus s situation. We're in this situation where I'm going to be shoving in the top hands and folding the worst. So if my margin was here, if my margin was, I don't know, eight six suited or something, maybe I'd go, like, eight five suited. Maybe I'd go king deuce off suit, something like that. Instead what I did was I went all the way to the very bottom of my distribution and got suitably punished. Yeah. So however confident you think you are in your read, you are probably overestimating it. So, yeah. That's where I was.

One other thought here before I wrap up, and that's this idea of advanced exploitive play, and this is fun. So we think about expanding the margins. We think, oh, in this spot he's bluffing. I should call more. But now if you start to make slightly more subtle reads, this is an opponent who bluffs a little too much on the river, I could punish him on the river by calling, or this is an opponent who folds too much on the river. I could punish them, or I could punish her by betting the river. But they're going to get that feedback pretty soon if you're just starting to hammer every river.

Another thing you can do is you can make the pot a little bit bigger, right? So don't forget about this. This is the pot. If you start making this bigger on earlier streets, maybe against this particular player, I raise a little bit more preflop. I bet a little bit more on the flop. Right? The pot's bigger on the turn, so now I'm betting on the turn. And now with the river I take all my normal actions except I know that I'm winning a few too many pots because they're folding too much, and now the few too many parts that I'm winning are proportionally larger. So this is this idea of we can exploit downstream. We can anticipate where they're weak. We cannot change our play in that spot and tip them off, but we can change it earlier.

So to wrap up, so you want to know yourself. You want know your own hand, right? That's the first key. The second key is to keep it balanced, and you want to exploit the margins. And so as we think about Cepheus, as we think about these algorithms coming and exploiting players, so even in 20 years, computing power-- oh, what's that? Like 1,000 times greater or something? Is that right? Wrong?

AUDIENCE: Sounds all right.

MATT HAWRILENKO: Depending. Depending if you believe Moore's law, I guess. OK. So even then, even when more poker games are tractable, you're going to need toy games to draw insights on what's going on. As these strategies come out of the black boxes, if you hope to grasp them, if you hope to hold onto them, you need these kind of insights to be able to have some scaffolding to start to put them on. So I think I'm done, but if I were to summarize this talk one way, don't be this guy. Be this guy. OK, we're done. Any questions? Ooh, wait.

AUDIENCE: So if you're doing, like, Nash equilibrium, well, that guarantees you a positive EV. But [INAUDIBLE] that's always enough. Like, if you're in tournament having a positive EV, your chips will grow a little but [? maybe blinds ?] are growing faster. Is that an [INAUDIBLE] Nash equilibrium strategy, or do you have to deviate if you want a higher variance [INAUDIBLE]?

MATT HAWRILENKO: Yeah, so good question. So the question-- and tell me-- let me make sure we answer it-- the question is, OK, well, you can try to play an equilibrium strategy and that's fine. But if you're exploiting, you're winning more. And so if you're playing in tournaments or these situations, you really need to exploit to win more money. Is that kind of-- yeah?

So yeah. That idea has been around for a long time, and I think that that idea is driven by people not really knowing just how strong game theory strategies might be, and just how strong sort of like, OK. So what is a Nash equilibria here? When you're playing a Nash equilibria, the idea is that your opponents are going to impale themselves on their own mistakes, and you're trying to make as few mistakes as possible.

And how big is that, right? It's kind of an empirical question. In my experience, it can be a lot bigger than most people think. Because when you start exploiting, you start making mistakes. And you start sort of getting impaled a little bit on your own mistakes too. And my guess is for most of us, for 99% of us, we're going to make a lot more mistakes than we think we do. So we think we're exploiting. At the same time we're getting exploited.

So actually, thinking about that is an empirical question, right? Like, how do human players do right now against Cepheus? My guess is not very well. In fact, my guess is that they're losing more than the top players, the top human players are winning from other human players. So that was certainly kind of my experience a few years ago. So it's one of these things where you'd see it, right? Like I think-- I don't know. I feel like I saw some stat where Cepheus was beating really good professionals for four bets per hundred hands, which is a lot. Which is a lot. Like, most top players are winning less than half of that.

So Nash equilibria, yeah. Pretty darn good. Maybe one day when human players are better, you need to start exploiting more. I don't think we're anywhere close. Other questions? Anything else? All right, I guess we're done. Thanks guys. I'll stick around for a little bit if anyone wants to chat.

[APPLAUSE]