Design and Evaluation of Event Detection Techniques for Social Media Data Streams

Abstract

Unprecedented success and active usage of social media services result in massive amounts of user-generated data. A leading player in producing a large volume of data as a continuous stream of short messages, so-called tweets, is the social network platform Twitter. The brevity of tweets, with a maximum of 140 characters, makes them an ideal mobile communication medium. Therefore, Twitters popularity as a source of up-to-date news and information source for current events is constantly increasing. In response to this trend, numerous research works on event detection techniques applied to the Twitter data stream have been proposed. However, most of these works suffer from two major shortcomings. First, they tend to focus exclusively on the information extraction aspect and often ignore the streaming nature of the input. Second, although all of the proposed works provide some evidence as to the quality of the detected events, none relate this task-based performance to their run-time performance in terms of processing speed or data throughput. In particular, neither a quantitative nor a comparative evaluation of these aspects has been performed to date. This thesis mainly describes our research work to fill these gaps and to tackle the posed challenges. In the first part of the thesis, we present a technique for real-time event detection and tracking, which focuses on the streaming nature of the data. Additionally, we describe a technique for event detection in pre-defined geographic areas. In the second part of the thesis, we study the run-time and task-based performance of several state-of-the-art event detection as well as baseline techniques using real-world Twitter streaming data. In order to reproducibly compare run-time performance, our approach is based on a general-purpose data stream management system, whereas task-based performance is automatically assessed based on a set of novel measures. This set of measures is especially designed to support the quantitative and qualitative comparison of event detection techniques. The last part of the thesis describes the design and evaluation of two visualizations to support visual event detection. First, we present “Stor-e-Motion”, a shape-based visualization to track the ongoing evolution of importance, emotion, and story of topics in user-defined topic channels applied to the Twitter data stream. Second, we present “SiCi Explorer”, a visualization that supports analysts in monitoring events/topics and emotions both in time and in space. The visualization uses a clock-face metaphor to encode temporal and spatial relationships, a color map to reflect emotion, and tag clouds to show the events and topics. Finally, we demonstrate the usefulness and usability of the visualization in a user study that we conducted.