Counterexamples for Model Checking of Markov Decision Processes

Abstract

The debugging of stochastic system models relies on the availability of diagnostic information. Classic probabilistic model checkers, which are based on iterated numerical probability matrix operations, do not provide such diagnostic information. In precursory work, we have devised counterexample generation methods for continuous- and discrete-time Markov Chains based on heuristics guided explicit state space search. In this paper we address the problem of generating diagnostic information, or counterexamples, for Markov Decision Processes (MDPs), which are a convenient formalism for modelling concurrent stochastic systems. We define the notion of counterexamples for MDPs in relation to an upwards-bounded PCTL formula. Next we present our approach to counterexample generation. We first use an adoption of Eppstein's algorithm for k-shortest paths in order to collect the most probable MDP execution traces contributing to a violation of the PCTL formula. We then use the data structure of AND/OR trees in order to adequately extract from the collected execution sequences the most informative counterexample and to compute its probability. In our experimental evaluation we show that our approach scales to models of realistic size, and that the collected diagnostic information is helpful in system debugging.