A Pragmatic Approach to Biases in Visual Data Analysis

Abstract

Visual biases and more generally cognitive biases are a part of human life. Often to the frustration of the rational decision makers we aspire to be. Research into these biases has sparked a recent burst in interest, and more and more people are aware of possible pitfalls. In this paper, we argue that the consequences of biases during data analysis have to be considered rather than the occurrences themselves. In applying this, we distinguish between (visual) analysis for exploration and validation. Especially the latter turns out to be hard in some cases, indicated by a qualitative measure we call validation cost. Examples are provided of situations with a high validation cost and the role of visualization is discussed in these cases. For cases with a low validation cost, we argue that biases leading to false positives are far better than trying to avoid biases and ending up with false negatives.