A decision heuristic for Monte Carlo tree search doppelkopf agents

Abstract

This work builds up on previous research by Sievers and Helmert, who developed an Monte Carlo Tree Search based doppelkopf agent. This four player card game features a larger state space than skat due to the unknown cards of the contestants. Additionally, players face the unique problem of not knowing their teammates at the start of the game. Figuring out the player parties is a key feature of this card game and demands differing play styles depending on the current knowledge of the game state. In this work we enhance the Monte Carlo Tree Search agent created by Sievers and Helmert with a decision heuristic. Our goal is to improve the quality of playouts, by suggesting high quality moves and predicting enemy moves based on a neural network classifier. This classifier is trained on an extensive history of expert player moves recorded during official doppelkopf tournaments. Different network architectures are discussed and evaluated based on their prediction accuracy. The best performing network was tested in a direct comparison with the previous Monte Carlo Tree Search agent by Sievers and Helmert. We show that high quality predictions increase the quality of playouts. Overall, our simulations show that adding the decision heuristic increased the strength of play under comparable computational effort.