Podcast: Play in new window | Download
Johannes Levemann is the head of Research and Development for the Snowie Group one of the creators of PokerSnowie, a neural net that has over the course of several years supposedly learned to play a nearly unexploitable no-limit hold ’em game. We talk with Johannes about what exactly a neural net is and how it learns a game as complex as poker, how he expects PokerSnowie to change the internet poker landscape, and the role that he thinks artificial intelligence will play in shaping poker’s future.
You can use PokerSnowie to evaluate and improve your own game with the PokerCoach software. You can keep up with the latest PokerSnowie news on their blog or on Twitter or Facebook.
Time Stamps
0:30 Hello and Welcome
13:15 Strategy: GTO or GTFO
49:19 Interview: Johannes Levermann
Your interview is rather timely, I saw this the other day….a short video of Jungleman playing Snowie HU recently.
http://www.pokerlistings.com/videos/v/man-vs-machine-jungleman-heads-up-with-pokersnowie-ai-c8846788
Just realized this video is in the link you posted on their blog, feel free to delete the post if you wish.
Just to push back on the river three-bet ranges versus button opening competency/incompetency surprise.
The difference is the trees. There are so many trees when you are in the big blind versus a certain button opening range and player type. When you are on the river with a potential three-betting hand, most likely your candidates are {nuts or air} and opponent has taken a long list of actions that has each subsequently narrowed the probability distribution of their range. An experienced player’s brain can reduce many river decisions to a solvable discrete function, whereas while in the BB facing a BTN may be a discrete ‘solvable’ function, people don’t know it because the multitude of possibilities makes it appear like a continuous function and hence nebulous and unsolvable for human-brain.
Basically adding more information makes decisions easier. Particularly when you arrive at a river in a nonoptimal way (though not necessary for the above point) because that nonoptimal way should include a list of conditional and relatively accurate assumptions equipping you to make a good river decision and pretty precise ranges even if they are situations you see few repetitions in.
Basically its complexity versus repetitions. Not so clear that infinite repetitions with complexity that is overwhelming will generate good play from a human, while complexity that is not overwhelming could be solved by a smart poker player given limited, if any, repetition.
But there are a few minutes left and maybe Nate is about to say this on the air.
Very interesting. You could say it is actually good that Snowie forces us to come to terms with reality. With or without neural networks, strong NLHE bots were on their way, or already here; but until now you couldn’t “see” them so it was easier to deny. Because the makers of Snowie use it so responsibly, unsegregated online poker will not die immediately; as for the the longer-term outlook, that was bleak anyway.
Another possible example, in addition to what was said in the podcast, of Snowie confouding current intuitions: Some of the Snowie-trained players from Montenegro seem to open half-pot from UTG and full-pot from button. If true, this would be completely contrary to what good human players do. Andrew has described the rationale behind the latter in the excellent article “No such thing as a free hunch”:
“If you are going to announce that you have a strong hand, which you pretty much are when you raise in first position, then you might as well charge a higher price to opponents who want to take advantage of their position to use that information against you.”
But could there also be a reason for doing exactly the opposite? Thinking of it, after listening to Johannes on blind defence, there could be: you don’t have to bet so big from first position to discourage the blinds from defending, since your range is strong. Anyway, anyone who uses Snowie could easily check whether it really advises that.
And finally we are getting a ratings system! Back in 2008, in 2+2 magazine, in my only poker article ever, I wrote about the same system that Johannes describes, but the article didn’t have any impact. Reading it again, I think it is still partly interesting rather than all outdated:
https://sites.google.com/site/zoltzsite/cardluckfactors
(How things have changed since 2008. Roger Federer is no longer winning many Wimbledon tournaments, and Phil Hellmuth now seems to get respect. And it says “hand simulators playing optimal unexploitable game-theory strategy would obviously be ideal for the job” without considering them realistic, but now one is actually available!)
I’ve always thought that it would be nice for a televised poker game to show not only equity percentages but also the value of the card distribution, both for individual hands and then at the end for the whole session, so you can see who did well objectively and who didn’t.
I don’t have anything really substantive to add to the above comments, you guys are well beyond my level of thinking (currently, hopefully improving). I just wanted to say what a interesting concept pokersnowie is. Many thanks for getting this interview guys. Awesome podcast as always….
I was a bit surprised that the backgammon case seemed like it wasn’t that well known to Nate especially, since it occupies that niche between programming and games playing which he seems so interested in.
I was also intrigued about the world of professional backgammon playing. We always hear about how Gus was a pro BG player, and I’ve always wondered how, if the game was big enough to support pros, I’ve never heard of any others. I tended to assume that it was either totally nonsense or he just crushed some rich fish for millions in casual games. How many pros are there out there? Is there a backgammon tour, and if so why has there not been more crossover with poker? Also, how do pros compare with the old guys playing in the streets in the middle east?
I’d be a bit surprised if you gave up that much restricting yourself to a few standard bet sizes, and my guess would be that the worst spots are when the SPR ends up *just* too big for your biggest bet sizing to conveniently make the last effective bet.
You might enjoy this New Yorker article about “Falafel” Khatchadourian, a pro backgammon player. Also has some mention of poker in there, too. So I guess there are others, although they will not usually be household names. http://www.newyorker.com/reporting/2013/05/13/130513fa_fact_khatchadourian
Thanks for the link!
The world of Go AI might also be an interesting case.
Go is much harder to programme than chess because of two reasons: 1) much larger branching factor, and 2) a lack of a convenient mid-game metric to assess who is wining (i.e. something like in chess the classic material count). 1) is obviously solved over time by computing crunching power, but in the last few years 2) has been very impressively ‘solved’ by what are known as monte carlo methods.
In essence, the AI assesses the balance of a given position by playing it out *at random* a few thousand times, and then sees which side wins most often. This proves to be an effective way of assessing the game state, and AIUI, has been the key to the best AI going from being quite a lot worse than me to quite a bit better in the space of a few years.
Pretty interesting Snowie commentary of how I played a hand the other day.
100nl, I open MP to 3x with A4hh (Snowie approves ~+0.14bb), CO calls with 105x stack, BB 50bb stack calls.
Qs 5h 3s flop
check, Hero checks (Snowie approves, EV 3 for checking, EV 2 for betting), CO bets 5.5bb, BB calls.
Hero check-raises to 14bb (Snowie approves!)
CO calls, BB folds
Turn 9s
Hero bets 28bb into 43bb (Snowie does not approve!) checking has nonzero EV, betting – 3bb EV.
CO shoves 🙁
I found it interesting that Snowie approved of both my check and my check-raise. Sometimes you will see Snowie approve of something after a string of suboptimal decisions. Well you’re here now, fuck up, but you did this right to try and bail yourself out, EV-wise. But here it doesn’t have a problem with me check-raising flop as the initial raiser against this bet size, given this action, on this flop.
I mean, I knew this was GTO flop play all along :p
Here is an example of me making not approved decisions to lead to approved ones later. I open A8o in sb, BB 3bets to 7bb. Snowie recommends to fold 138bb deep. Initially completing and raising from my perspective in sb are very close fwiw. I four-bet to 19bb (naturally) and villain calls. Snowie thinks this is the worst option (usually I take the 2nd worst option if I make a Snowie mistake) with folding 0EV, calling -1bb EV, four-betting to 19x -2bb EV.
Flop T52r I bet 15bb, snowie does not approve. Both bet and check are +EV, checking a whopping +3.6bb to betting’s 0.8bb.
turn Q bringing backdoor diamonds, I have Ad, I bet 26bb. Snowie approves highly.
River Jh, I shove 100x into 120x, Snowie approves. This is an entire 1.6 big blinds better than folding! Though I am led to believe Snowie’s ideal river bet size here is 78. Either way, Snowie thinks we should bet this combo practically all the time on this river.
OK Gareth.Before you analyze Snowy logic you have to realize one thing
Snowy is not not math-based program that calculates probabilities,EVs,ranges etc.
Snowy is black box where nobody knows really how the answers are produced.
I assume that even Johannes Levermann does not know the exact logic behind Snowy moves.
Johannes Levermann knows one thing for sure.He know how to build the complex robust neural network.
The fascinating feature of a neural network bot is that it is created completely differently from a classical program bot.
For a traditional poker bot, you would think about how to implement the EV+ floating or bluffing and then create all of the necessary programming logic to do it(opponent modeling,board analysis,bet sizing ,etc).
For a neural network, you just provide input examples(transformed handhistories) and the expected outputs.
It is up to the neural network to learn how to provide these expected outputs.
Usually, you really have no idea how it actually learns to provide its output.
This is especially true for complex neural networks like Snowy that may have hundreds of neurons.
This does not change the fact that the human analysis of Snowy logic can be very valuable for the theory of poker.
Great interview.Great questions.
I believe that PokerSnowie or even glued cheap surrogates like the Deep Blue of OFC are game changer for learning poker.
It is difficult to predict when the source code of PokerSnowie and the expertize will leak to “public” space.
The stock market story could give some glimpse of future for online poker future.
Computers revolutionized the stock market.It started so called “The Rise of the Machine Traders”.
Poker players will be just floored.http://www.youtube.com/watch?v=tCcxr-fyF4Q
Thanks for the praise (and for the comment)!
You both did much better job than gambling columnist from NYT.
http://www.nytimes.com/2013/09/08/magazine/poker-computer.html?pagewanted=1&_r=0
My apology for my eng. spelling and grammar.
Interesting if occasionally painful read, thanks for sharing.
Best episode yet IMO.
That’s high praise from a loyal listener! Thanks.
I, for one, welcome our new neural network overlords. Actually, after getting a knot in my stomach after Andrew’s explanation for why so many see this as a harbinger of the end of online poker, I actually wound up feeling fairly optimistic by the end of the episode. While it cannot approach Snowie in terms of raw computing power, the human brain really is pretty amazing. I know this is not a particularly controversial statement for people with an AI background, but I find it fascinating nevertheless. Inevitably, that margin of human uniqueness will continue to be encroached on in the coming decades. Where that process will ultimately end is both exciting and terrifying to consider. Will a computer eventually create original, spontaneous art? Or will we find ourselves in a “Best of Both Worlds” scenario in a Kurzweilian singularity? I can’t figure it all out today, sir, so I’m just going to hang out with your daughter.
You mentioned on the podcast that Pokersnowie never slowplays AA preflop, but that’s not entirely true.
When you three-bet a Small Blind opener from the Big Blind and the SB four-bets, Pokersnowie advises calling eighty-five percent of the time with AA, and thirteen percent of the time with KK.
Why? I speculate that because the SB stacks off with so many one-pair or drawish hands post-flop in order to apply pressure, AA has more value against a flop-stacking off range than as a Value-oriented Five-bet. Surely this must be due to the smaller S:PR. Is this the same reasoning behind flatting with monsters late in tournaments, or have I made a mistake in my analysis?
Keep these awesome podcasts coming.
This info is from the Preflopper iPhone app, by the way.
“Pokersnowie never slowplays AA preflop.”
There is no “never rule” for neural network.
The Snow engine was built by coding rules, but Snowy intelligence(algorithms) are build by training (by examples) his neural net.
Snowy “brain” resides in the cloud not on your iPhone HD. It could mean his learning is ongoing process.
It is natural for neural network to “mutate” his play at random time after make some learning progress.
“Why?”
Even Snowie does not tell you.
The Snowie neural network knows nothing about reasons behind its decision.
It can make only recommendations not deep analysis.
I believe Snowie does not support tournaments only cash games
Nate,
you seemed to spend some time trying to figure out whether “Snowie plus partner” was stronger than just “Snowie”, and then whether “Snowie” alone was stronger than “Anti-Snowie plus Partner”.
I sense you felt this was important but it was unclear why. Could you explain further or perhaps (if i got this wrong) just let me know I need to listen closer in the future?
which
The first case seems trivial.(“Snowie plus partner” vs “Snowie”)
Snowie uses very limited arsenal.According to Snowy Group the strategy is just execution of pure GTO by neural network.
Snowy does not use opponent modeling,adjustments or any other model extensions to implement its strategy.
“Snowie plus partner” have much wider spectrum of strategies.
Bot plus partner is typical setup for winning bots playing real money space.
The rest of cases are quiet complex.I wonder what Nate can say about that.
I have another question to Nate and Andrew.
This claim that Snowie uses GTO is interesting and at least shaky.
How they came to this conclusion? how they know this strategy is GTO???.
If this is the case their marketing department should offer 1000000$ prize for anybody who can beat(exploit) Snowy.
if my memory serves, Andy, the first few versions of SnowieBG ‘were’ beatable by experts. There were a few situations where the experts could maneuver SnowieBG into positions which the experts were confident the software played less than perfect. The SnowieBG 3 & 4 however had corrected (?) these, and the live players had lost often enough to the software that the experts became believers.
Oddly enough, for bearoffs (no contact, all checkers within the homeboard), which are absolutely a database solved event, the SnowieBG was never able to get perfect, and Hugh Sconyers marketed a 9 CD database that solved this issue.
I expect SnowiePoker to mimic this arc. The first versions would probably be met with much skepticism, with some situations seemingly obviously wrong. Future versions would not only put “current thinking” on its head, but eliminate all but most obscure mistakes.
Just my opinion though, of course. I would be very interested to see where the current line of expert strategy would be ‘proved wrong’ by the software. Perhaps light 3 betting OOP would be tossed away as too risky to your stack? Or maybe position would be seen as less important, perhaps because we now use position to exploit early position player’s tendencies (maybe they fold too much to aggression) where in the future, maybe SnowiePoker teaches players to be stickier postflop?
I guess I am wondering how my impression of SnowieBG having a certain ‘fearlessness’ in going for gammons will translate to poker. Maybe it amps up the aggression leading to many more stack-offs? Or maybe it is more willing to get to showdown vs aggression both in and out of position?
Gonna be an interesting five years for sure…. SnowiePoker Trainer will be a big predictor I believe.
which
Another interesting question is: How The SnowieBG 3 & 4 leaks were corrected?.
a)These leaks could be corrected by Leverman’s team- or
b)Snowy was already very robust neural network and fixed the leaks in result of neuron learning GTO by examples produced by experts.
Well Andy, it is hard to see in the above whether you mean BG or poker.
if you ARE referring to BG, I believe the Neural Nets just continued to play vs each other, I do not think they ever used experts. IF they had, they could have just implemented the bearoff database, which never happened.
In the language you use ‘GTO’, which BG never really needs to address. BG being an open info game, the software just always chose the most EV play. (Exploitive in BG might mean intentionally choosing moves to get into a “prime vs prime” game, but since the software does not ‘push’ games towards one style or another, it just chooses best moves for dice given, it never really exploits weaknesses of players in BG.
which