Discovering OLAP dimensions in semi-structured data

Abstract

With the standard OLAP technology, cubes are constructed from the input data based on the available data fields and known relationships between them. Structuring the data into a set of numeric measures distributed along a set of uniformly structured dimensions may be unrealistic for applications dealing with semi-structured data. We propose to extend the capabilities of OLAP via content-driven discovery of measures and dimensional characteristics in the original dataset. New structural elements are discovered by means of data mining and other techniques and are therefore prone to changes as the underlying dataset evolves. In this work we focus on the challenge of generating, maintaining, and querying such discovered elements of the cube.We demonstrate the benefits of our approach by providing OLAP to the public stream of user-generated content of the popular microblogging service Twitter. We were able to enrich the original set by discovering dynamic characteristics such as user activity, popularity, messaging behavior, as well as classifying messages by topic, impact, origin, method of generation, etc. Application of knowledge discovery techniques coupled with human expertise enable structural enrichment of the original data beyond the scope of the existing methods for generating multidimensional models from relational or semi-structured data.