Stochastic Optimal Control

Here I have some notes introducing some key concepts in Stochastic Optimal Control. Stochastic Optimal Control is a branch of Control Theory that seeks to define strategies for controlling systems with some random input or randomness in the output. It also defines strategies that use randomness as a method for discovery, an example being Monte Carlo Methods.

I briefly introduce some applications in robotic control and financial engineering, but the applications are much wider and varied than those two examples. These notes will be updated as I learn more.

Joker Joker Deuce Deuce

Joker Joker Deuce Deuce seems like the de facto style of play for the younger generation of spades players. It’s played late at night in the college dorm rooms by folks with 8am classes. It’s played in off-campus apartments accompanied by Four lokos, Amsterdam, and other low cost spirits that may or may not come in a plastic bottle. And of course it’s played at BBQs, cookouts, and other family gatherings.

In this version of spades, the deuce of hearts and deuce of clubs are removed from the deck in favor of the two Jokers (big Joker and little Joker). The addition of the two Jokers and the use of the deuce of diamonds as a spade changes the number of cards from each suit in play from 13 to the numbers below:

• Spades : 16
• Hearts: 12
• Clubs: 12
• Diamonds: 12

Intuitively, more spades in the deck (4/13 compared to 1/4) means you have a higher chance of ending up with one or more. On average, in JJDD each player will have four spades and three of each of the other suits.

Okay, before you tell me about how you always end up with six hearts (that sucks btw) we all know that spades doesn’t go by averages. It goes by the heart of cards. Like other versions of spades, there are going to be hands when you have the juice, other hands where you’re struggling to get two, and once in a while you’ll still end up having a hand with no spades.

So what really IS the difference besides a few extra spades? Besides changing the initial probabilities, using one less cards per suit increases the likelihood of an early cut (compared to the Ace down version of spades). The first time that Heart, Diamonds, or Club is played, the highest card played (usually the Ace) will walk. Please pray for that King though – one of them is probably not going to make it.

Saving the hypergeometric probabilities talk for later, let’s go over some rough probabilities. Each book has 4 suits of a card played. After the first book, there are 8 cards of that suit left in play split among 4 players. How many cards of that suit do you have left in your hand?

If every player has at least 1 more card of that suit (84% chance), then the King is safe. So probability-wise, you should still count your King as a book unless you have four or more cards of that suit left in your hand.

BTW, I’m saying the King because it’s usually the second card in the suit played but that is not necessarily the case.

However, if one player received 1 card of that suit (about a 13% chance) then that King will be cut (unless that player who can cut is your partner – don’t you love when that happens?). With 13 cards per suit, there’s only a 8% chance that a player has 1 card of a suit. If you’re thinking 8% compared to 13% isn’t THAT big of a difference then…you can’t be on my team.


Spades is all about probability

Imagine you’re playing spades. You’re down to three books (or “tricks” as the old folks say). You’re in a tough position because your team need to win two of the remaining books or y’all will be set. On top of that, you’ve been talking shit all game.

Right now your partner has the highest card on the table but you’re not sure if she’s gonna win the book. Do you cut or play over your partner to give your team the better chance of winning the book? Or do you play under with the hope that your partner’s card will hold? If you play spades regularly you’ve no doubt run into this situation. And this is not a trivial decision – the book, the hand, your reputation, and hell, even your relationship with your partner is at stake (and I promise I’m not being dramatic).

So what did you do?

Now there’s no hard and fast rule about the best move in this circumstance. Some folks say “play to win,” but if you play over your partner’s strongest card then you may cost your team a book (and the game). Alternatively, some folks believe you should NEVER play over your partner but this strategy can be problematic as well.

Now back to the question. What did you do?

I’m guessing what you ultimately decided depended on number of factors. What cards have been played?* What cards are left in your hand? What’s the probability that you can win the remaining books with those cards? What cards do you think your partner has left? What cards do your opponents have left? How many sandbags do we have?

If you were confident that you could win the last two books  you might have been more willing to play under. If y’all really needed that book and you thought you could win it, you might have been more willing to play over your partner.

So in short, what you did (whether you knew it or not) involved calculating the probabilities of winning the hand based on the competing strategies.

Like poker, spades is a game that involves the implicit calculation of probabilities. When you think “will that card walk? what’s the chance that this card is going to be cut? who’s cutting? or who has the Joker or Ace of Spades?,” you’re calculating probabilities. Probabilities describe the chance of an event occurring (usually given as a ratio or percentage).

Astute spades players intuitively recognize the probability of a specific card walking can be much different if the card is the first one played versus if it’s the third card played. The chance of a card being cut generally increases as more cards of that suit are played. Identifying who is cutting is more difficult and requires you to watch who is playing what. And if you don’t have the highest card there’s a 1/3rd chance your partner has it and a 2/3rd chance that one of your opponents has it.

Okay so spades involves probabilities. So what?

As you are playing spades you are (or should be) constantly calculating and re-calculating probabilities. I’m not talking hardcore conditional probabilities but rough calculations that you can help you choose the best option. In the forthcoming posts I’ll discuss how you can use knowledge of probabilities to help make you a better spades player.

* If you haven’t kept track please re-evaluate your life as a spades player.

Spades is all about probability