Sentiment analysis with knowledge resource and NLP tools

Piao, Scott and Tsuruoka, Yoshimasa and Ananiadou, Sophia (2009) Sentiment analysis with knowledge resource and NLP tools. The International Journal of Interdisciplinary Social Sciences, 4 (5). pp. 17-28.

Abstract

Automatic sentiment analysis is an important and challenging topic in Human Language Technology (HLT) and text mining, with several applications for social sciences. Over recent years, much effort has been devoted to this subject. Many published works on this subject employ various machine learning techniques. In our work, we investigate the feasibility of automatically identifying text sentiment orientation by combining knowledge resources and NLP techniques, as a complementary method to those based on machine learning. Our approach involves resources and tools including a subjectivity lexicon (Wilson et al., 2005), a set of NLP tools and weighting algorithms. We developed a sentiment analysis tool with a web demonstrator (http://text0.mib.man.ac.uk:8080/opminpackage/opinion_analysis). This tool has been used jointly with other text mining tools to support social scientists in frame analysis from newspaper articles. In this paper, we describe our system and report on our evaluation of the functionality of identifying sentiment orientation at the sentence and document levels. We used the Multi-Perspective Question Answering (MPQA) Corpus (Wiebe et al., 2005) and a collection of 2,000 manually classified film reviews (Pang and Lee, 2004) as the test data. As evaluation measure, we used f-score, a performance measure that combines precision and recall and ranges between 0 (worst performance) and 1 (best performance). In the evaluation, our sentiment analysis tool obtained encouraging results, producing f-scores ranging from 0.6012 to 0.8333 for sentence-level sentiment analysis and an average f-score of 0.7196 for document-level sentiment analysis.