Diversity Driven Parallel Data Mining

Abstract

With increasing availability and power of parallel computational resources, attention is drawn to the question of how best to apply those resources. Instead of simply finding the same answers more quickly, this thesis describes how parallel computational resources are used to explore disparate regions of a solution space by using diversity to steer the solution paths away from each other, thereby discouraging strictly greedy behavior. The formulation of models in a concept/solution space and its relationship to a search space are described as well as common search algorithms with heuristics for time or space computationally prohibitive searches. Measures of diversity are introduced, and the application of a beam search to the solution space for the Krimp algorithm for frequent itemset mining is described. Experimental results show that it is indeed possible to get better results on real-world datasets with these methods.