Subspace Clustering of Very Sparse High-Dimensional Data

Peng, Hankui and Pavlidis, Nicos Georgios and Eckley, Idris Arthur and Tsalamanis, Ioannis (2019) Subspace Clustering of Very Sparse High-Dimensional Data. In: 2018 IEEE International Conference on Big Data (Big Data). IEEE, USA, pp. 3780-3783. ISBN 9781538650356

Full text not available from this repository.


In this paper we consider the problem of clustering collections of very short texts using subspace clustering. This problem arises in many applications such as product categorisation, fraud detection, and sentiment analysis. The main challenge lies in the fact that the vectorial representation of short texts is both high-dimensional, due to the large number of unique terms in the corpus, and extremely sparse, as each text contains a very small number of words with no repetition. We propose a new, simple subspace clustering algorithm that relies on linear algebra to cluster such datasets. Experimental results on identifying product categories from product names obtained from the US Amazon website indicate that the algorithm can be competitive against state-of-the-art clustering algorithms.

Item Type:
Contribution in Book/Report/Proceedings
Additional Information:
©2018 IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the IEEE.
ID Code:
Deposited By:
Deposited On:
30 Nov 2018 15:18
Last Modified:
16 Sep 2023 03:15