Multi-Label Classification by ART-based Neural Networks and Hierarchy Extraction

Abstract

This paper presents a data mining system for multi-label classification and hierarchy extraction from the predictions provided by a multi-label classifier. Classes in multilabel classification tasks are often hierarchically organized and the hierarchy is assumed to be known. A much less investigated approach and a more challenging task, however, is to suppose that the underlying class taxonomy is unknown and that a data mining system can infer it automatically. In our setting, the proposed system is trained with multi-label data and is subsequently able to produce multi-label predictions along with hierarchical relationships between classes. The hierarchy extraction algorithm is based on building association rules from label co-occurrences. Within the framework we examine the performance of two recently introduced multi-label extensions of Adaptive Resonance Theory (ART)-based neural networks: Multi-Label Fuzzy ARTMAP (ML-FAM) and Multi-Label Fuzzy Adaptive Resonance Associative Map (ML-ARAM) in comparison with two state-of-the-art classifiers Multi-Label k-Nearest Neighbors (ML-kNN) and BoosTexter, taking into account the quality of hierarchy extraction. We also develop a novel distance measure for the quantitative evaluation of the derived class hierarchies and compare it with two other distance measures. To demonstrate the effectiveness of the proposed approach, experiments on several benchmark datasets have been performed.