On-line Clustering for Real-Time Topic Detection in Social Media Streaming Data

Abstract

The continuous growth of social networks and the active use of social media services result in massive amounts of user-generated data. Worldwide, more and more people report and distribute up-to-date information about al- most any topic. At the same time, there is an increasing interest in information that can be gathered from this data. The popularity of new services and technologies that produce and consume data streams imposes new challenges on the analysis, namely, in terms of handling high volumes of noisy data in real-time. Since social media analysis is concerned with investigating current topics and actual events around the world, there is a pronounced need to detect topics in the data and to directly display their occurrence to analysts or other users. In this paper, we present an on-line clustering approach, which builds on traditional data mining methods to address the new requirements of data stream mining: (a) fast incremental processing of incoming stream objects, (b) compactness of data representation, and (c) efficient identification of changes in evolving clustering models.