Consistency of Online Consumers’ Perceptions of Posted Comments: An Analysis of Tripadvisor Reviews

Authors

  • Foo Sheng Khoo Department of Computing and Information Systems Sunway University, Malaysia
  • Phoey Lee Teh Department of Computing and Information Systems Sunway University, Malaysia
  • Pei Boon Ooi Department of Computing and Information Systems Sunway University, Malaysia

DOI:

https://doi.org/10.32890/jict2017.16.2.9

Keywords:

Cohen Kappa, comments, degree of agreement, rating, online reviews

Abstract

Ratings and comments play a dominant role in online reviews. The question, thus, arises as to whether or not there is any consistency in consumer perception of the reviews, and how future choices might be influenced. We analysed 2000 comments of 20 different hotels posted on TripAdvisor to determine if the comments posted by previous guests of a hotel influence the decisions of potential guests. Two hundred human raters were asked to consider 20 reviews and to rate a hotel based on the reviews. The Cohen Kappa coefficient was used to evaluate the degree of agreement on the hotel quality as determined by the human raters and the star rating given by the original reviewer. The results showed a high consistency between the human raters’ evaluation and the reviewers’ star rating. This research reveals the importance of website feedback such as TripAdvisor in influencing consumer choice.

 

References

Abe, J. A. A. (2011). Positive emotions, emotional intelligence, and successful experiential learning. Personality and Individual Differences, 51(7), 817-822. http://doi.org/10.1016/j.paid.2011.07.004

Alec (2013). The 25 worst YouTube comments ever. Retrieved 9 December 2015 from http://runt-of-the-web.com/worst-youtube-comments

Bermingham, A. (2014). Would you dare? One business’ risky approach to negative online reviews - Utter Digital. Retrieved 12 December 2015 from http://utterdigital.com/digital-marketing/would-you-dareone-business-risky-approach-to-negative-online-reviews/

Brown, B. (2012). Beyond recommendations: Local review websites and their impact. ACM Trans. Comput.-Hum. Interac, 19, 4, Article 27, DOI = 10.1145/2395131.2395134 http://doi.acm.org/10.1145/ 2395131.2395134

Chan, H. K., Lacka, E., Yee, R. W. Y., & Lim, M. K. (2015). The role of social media data in operations and production management. International Journal of Production Research, 1–11. http://doi.org/10.1080/002075 43.2015.1053998

Chen, B., Guo, J., Tseng, B., & Yang, J. (2011). User reputation in a comment rating environment. Proceedings of the 17th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, New York, 159-167.

Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20(1), 37-46.

Dean, D. (2004). Consumer reaction to negative publicity: Effects of corporate reputation, response, and responsibility for a crisis event. Journal of Business Communication, 41(2), 192-211.

Dietz, J. (2015). 2015 Game publisher rankings - Metacritic. Retrieved 7 December 2015 from http://www.metacritic.com/feature/gamepublisher-rankings-for-2014-releases [Accessed September 10, 2015]. Journal of ICT, 16, No. 2 (Dec) 2017, pp: 374–

Dincer, M. Z., & Alrawadieh, Z. (2017). Negative word of mouse in the hotel industry: A content analysis of online reviews on luxury hotels in Jordan. Journal of Hospitality Marketing & Management, 10.1080/19368623.2017.1320258

Evans, J. D. (1996). Straightforward statistics for the behavioral sciences. Pacific Grove, Calif: Brooks/Cole Pub Co. Filippi, P., Ocklenburg, S., Bowling, D. L., Heege, L., Güntürkün, O., Newen,

A., & de Boer, B. (2016). More than words (and faces): Evidence for a Stroop effect of prosody in emotion word processing. Cognition and Emotion, 9931, 1-13. https://doi.org/10.1080/02699931.2016.1177489

Fleiss, J. L., Levin, B., & Paik, M. C. (2003). Statistical methods for rates and proportions (3rd ed.). London: Wiley.

Fong, L. H. N., Lei, S. S. I., & Law, R. (2016). Asymmetry of hotel ratings on TripAdvisor: Evidence from single-versus dual-valence reviews. Journal of Hospitality Marketing & Management, 00(00), 1–16. https://doi.org/10.1080/19368623.2016.1178619

Gill, A. J., French, R. M., Gergle, D., & Oberlander, J. (2008). The language of emotion in short blog texts. Proceedings of The 2008 ACM Conference on Computer Supported Cooperative Work. USA: San Diego, doi>10.1145/1460563.1460612

Hassan, M. G., Hussain, F., & Saibu Rahman, M. B. (2013). Exploring usefulness of CRM and IT in Malaysian hotel industry: A qualitative approach. Journal of Information and Communication Technology, 12, 21-37. Hinojosa, J. A, Rincón-Pérez, I., Romero-Ferreiro, M. V., Martínez-García, N.,

Villalba-García, C., Montoro, P. R., & Pozo, M. A. (2016). The Madrid Affective Database for Spanish (MADS): Ratings of dominance, familiarity, subjective age of acquisition and sensory experience. PLoS ONE 11(5), e0155866. doi:10.1371/journal.pone.0155866

Huang, A. H., Yen, D. C., & Zhang, X. (2008). Exploring the potential effects of emoticons, Information & Management, 45(7), 466-473. Journal of ICT, 16, No. 2 (Dec) 2017, pp: 374–

Kauffman, R. J., Laf, H., Lin, H-C., & Chang, Y-S. (2009). Do textual comments and existing orders affect consumer participation in online groupbuying? Proceedings of the 42nd Hawaii International Conference on System Sciences. Hawaii, 1-10, 10.1109/HICSS.2009.165

Klan, F. & Ries, B. K. (2014). Recommending judgment targets for rating provision. 2014 IEEE/WIC/ACM International Joint Conferences on Web Intelligence (WI) and Intelligent Agent Technologies (IAT), 327–334. Available at: http://ieeexplore.ieee.org/lpdocs/epic03/wrapper. htm?arnumber=6927642.

Krestel, R., & Dokoohaki, N. (2011). Diversifying product review rankings: Getting the full picture. In Proceedings - 2011 IEEE/WIC/ACM International Conference on Web Intelligence, WI 2011, 1, 138–145. http://doi.org/10.1109/WI-IAT.2011.33

Lee, J., Park, D-H., & Han, I. (2008). The effect of negative online consumer reviews on product attitude: An information processing view. Electronic Commerce Research and Applications, 7(3), 341-352.

Leskovec, J. (2011). Social media analytics: Tracking, modelling and predicting the flow of information through networks. World Wide Web Internet and Web Information Systems, 277–278. Retrieved 9 May 2015, from http://dl.acm.org/citation.cfm?id=1963309.

Li, Y., & Yu, D. (2015). Development of emotive word comprehension in Chinese children from 2 to 13 years old: Relationships with valence and empathy. PLoS ONE, 10(12), 1–19. https://doi.org/10.1371/journal. pone.0143712 Ling, K., Beenen, G., Ludford, P., Wang, X., Chang, K., Li, X., Cosley, D., Frankowski, D., Terveen, L., Rashid, A. M., Resnick, P., & Kraut, R. (2005). Using social psychology to motivate contributions to online communities. Journal of Computer-Mediated Communication, 10, 00. doi:10.1111/j.1083-6101.2005.tb00273.x

Matos-rodríguez, Y. E. (2014). Online traveller ratings: Impact on determining the top 25 hotels. Retrieved 10 August 2016, from http://scholarsarchive. jwu.edu/mba_student/38/ Journal of ICT, 16, No. 2 (Dec) 2017, pp: 374–

Moghaddam, S., & Martin, E (2011). ILDA: Interdependent LDA model for learning latent aspects and their ratings from online product reviews. Proceedings of the Annual ACM SIGIR International conference on

Research and Development in Information Retrieval (SIGIR-2011).

O’Mahony, M. P., & Smyth, B. (2010). A classification-based review recommended. Knowledge-based Systems, 23, 323-329.

PhoCusWright. (2014). Hotel guests read 6-12 reviews before booking, says new TripAdvisor survey. Hotelmarketing’com. Retrieved 9 December 2015, from http://hotelmarketing.com/index.php/content/article/hotel_ guests_read_6_12_reviews_before_booking_says_tripadvisor_survey#

Raut, V. B., & Londhe, D. D. (2015). In: Guerrero J. E. (ed.), Proceedings of International Conference on Computational Intelligence and Communication Networks (CICN), India: Bhopal, 556-559.

Rayson, P. (2008). From keywords to key semantic domains. International Journal of Corpus Linguistics, 13(4), 519-549.

Samsudin, N., Puteh, M., Hamdan, A. R., & Ahmad Nazri, M. Z. (2013). Normalization of noisy texts in Malaysian online reviews. Journal of Information and Communication Technology, 12, 147-159.

Sander, T., Teh, P. L., & Sloka, B. (2015). Use of social network site’s profile for the employment-seeking process. Advances in Intelligent Systems and Computing, 353, 1023–1032. http://doi.org/10.1007/978-3-31916486-1

Sereno, S. C., Scott, G. G., Yao, B., Thaden, E. J., & O’Donnell, P. J. (2015). Emotive word processing: Does mood make a difference? Frontiers in Psychology, 6, 1-13. https://doi.org/10.3389/fpsyg.2015.01191

Siersdorfer, S., Chelaru, S., Penro, J. S., Altingovde, I. S., & Nejdl, W. (2014). Analyzing and mining comments and comment ratings on the social web. ACM Transactions on the Web, 8(3), 1-39.

TripAdvisor. (2015). All about your TripAdvisor bubble rating | TripAdvisor Insights. TripAdvisor. Retrieved 3 January 2016 from https://www. tripadvisor.com/TripAdvisorInsights/n2640/all-about-your-tripadvisorbubble-rating Journal of ICT, 16, No. 2 (Dec) 2017, pp: 374–

Teh, P. L., Rayson, P., Pak, I., Piao, S., & Yeng, S. M. (2016). Reversing the polarity with emoticons. In E. Métais, F. Meziane, M. Saraee, V. Sugumaran, & S. Vadera (Eds.), Natural Language Processing and Information Systems: 21st International Conference on Applications of Natural Language to Information Systems, NLDB 2016, Salford,

UK, June 22-24, 2016, Proceedings (pp. 453–458). Cham: Springer International Publishing. https://doi.org/10.1007/978-3-319-41754-7_48

Vásquez, C. (2011). Complaints online: The case of TripAdvisor. Journal of Pragmatics, 43, 1707–1717.

Vermeulen, I, E., & Seegers, D. (2008). Tried and tested: The impact of online hotel reviews on consumer consideration. Tourism Management, 1-5, doi:10.1016/j.tourman.2008.04.008

Wong, Y. J. (2015). The psychology of encouragement: Theory, research, and applications. The Counselling Psychologist, 43(2), 178–216. http://doi.org/10.1177/0011000014545091

Wyndham. (2015). Wyndham Midtown 45 TripAdvisor Reviews & Ratings | Hotel Reviews in New York, NY 10017-3301. Retrieved 19 December 2015 from http://www.wyndham.com/hotels/new-york/new-york/ wyndham-midtown-45/hotel-reviews-ratings

Zhang, Z., Guo, C., & Goes, P. (2013). Product comparison networks for competitive analysis of online word-of-mouth. ACM Transactions on Management Information Systems, 3(4). Retrieved 9 January 2016 from http://dl.acm.org/citation.cfm?id=2407744.

Zhang, Z. Q., Zhang, Z., & Yang, Y. (2016). The power of expert identity: How website-recognized expert reviews influence travellers’ online rating behaviour. Tourism Management, 55, 15-24.

Downloads

Published

06-11-2017

Issue

Vol. 16 No. 2 (2017): Journal of Information and Communication Technology (JICT) Vol.16, No.2, December 2017

Section

Articles

How to Cite

Khoo, F. S., Teh, P. L., & Ooi, P. B. (2017). Consistency of Online Consumers’ Perceptions of Posted Comments: An Analysis of Tripadvisor Reviews. Journal of Information and Communication Technology, 16(2), 374-393. https://doi.org/10.32890/jict2017.16.2.9

Research impact

Harvested 2026-09-06
5 citations, from OpenAlex — the highest of the sources checked

Counts differ between services because each indexes a different body of literature. None of them is the whole picture.

Identifiers DOI 10.32890/jict2017.16.2.9 OpenAlex W4248416746