Seminar by Özgür Türetken (Ted Rogers School of Management, Ryerson University, Canada) titled "Empirical Evaluation of Automated Sentiment Analysis as a Decision Aid"

ABSTRACT: This research is a part of a larger program on using text analytics techniques to increase the utility of textual content, especially in the social media and the Internet, for improved decision making. In the consumer decision making context, research has consistently shown that online word-of-mouth (WOM) plays an important role in shaping consumer attitudes and behaviors. Yet, despite their documented utility, explicit user scores, such as star ratings have limitations in certain contexts. Automatic sentiment analysis (SA), an analytics technique that assesses the "tone" of text, has been proposed as a way to deal with these shortcomings. These series of studies investigate the utility of SA tools as decision aids. In the first study, we compare two main techniques for analyzing sentiments in the context of consumer product reviews. Given that a noted gap in prior research has been the almost sole focus on short textual information that concerns specific contexts, we examine the role of contextual factors, namely review length and product category, on the accuracy of various sentiment analysis tools. Our results indicate that the lexicon-based approach to sentiment analysis outperforms machine-learning in almost all product contexts regardless of the length of the review. Machine learning had a better accuracy only in the case of long reviews about experience products and services. In the second study, we compare the results of the SA scores to consumer review ratings generated by human judges. In a third study, we propose to examine the effects of SA scores on decision making. Specifically, we develop a model that investigates the effects of review characteristics (length, extremity), product context (search vs. experience), and SA score representation on the utility of SA scores as measured by decision confidence, effort, and perceived usefulness.