Multi-Criteria Decision-Making Based Approaches in Website Quality and Usability Evaluation: A Systematic Review
DOI:
https://doi.org/10.32890/jict2020.19.3.11Keywords:
Decision making, website, usability, website evaluation, qualityAbstract
Websites are important in every organisation and tremendous effort is made to design websites that not only look and feel good, but are usable and of high quality. Nevertheless, one critical task is how to evaluate these websites to ensure that users are satisfied with its quality and usability. Although a variety of methods and approaches have been proposed, there is currently an increase in research efforts to model website quality and usability evaluation from the point of view of decision-makers which existing methods do not handle. Thus, this has led to the application of multi-criteria decision-making (MCDM) approaches in the evaluation of websites to handle complexity in decision-making. This paper, therefore, provides a review of the various MCDM methods that have been used in the usability and quality evaluation of websites. The search strategy which was adopted identified a total of 63 published articles in peer-reviewed journals and international conferences between 2005 and 2017. From the research questions formulated for the study, the papers were classified into various MCDM approaches, website genre, number and list of criteria used over the years and the localization of websites based on country. Some of the findings showed that the Analytical Hierarchy Process approach integrated with fuzzy logic has been the most common method over the years. In addition, e-commerce websites make up the most common website genre. Besides, currently most active websites are from Turkey and five is the average number of criteria for the evaluation of website quality and usability.
References
Adepoju, S. A., Oyefolahan, I. O., Abdullahi, M. B., & Mohammed, A. A. (2018a). A survey of research trends on University Websites’ Usability Evaluation. In 2nd International Conference on Information and Communication Technology and Its Applications (pp. 127–132).
Adepoju, S. A., Oyefolahan, I. O., Abdullahi, M. B., & Mohammed, A. A. (2018b). Integrated usability evaluation framework for university websites. I-Manager’s Journal on Information Technology, 8(1), 19–27.
Agirgun, B. (2012). Ranking B2C web sites with AHP and TOPSIS under fuzzy environment. Nevşehir Üniversitesi Fen Bilimleri Enstitü Dergisi, 1(2), 65–78. Retrieved from http://dergipark.ulakbim.gov.tr/nevbiltek/ article/view/1035000106
Akincilar, A., & Dagdeviren, M. (2014). A hybrid multi-criteria decision making model to evaluate hotel websites. International Journal of Hospitality Management, 36, 263–271. https://doi.org/10.1016/j. ijhm.2013.10.002
Alptekin, N., Hall, E. E., & Sevim, N. (2015). Evaluation of websites quality using fuzzy TOPSIS method. International Journal of Academic Research in Business and Social Sciences, 5(8), 221–242. https://doi.org/10.6007/IJARBSS/v5-i8/1778
Asemi, A., Sapiyan, M., Asemi, A., & Haji, R. B. (2013). Fuzzy multi criteria decision making applications : A review study. Retrieved from http://eprints.um.edu.my/12976/
Aydin, S., & Kahraman, C. (2011). A modified fuzzy analytic hierarchy process based multicriteria decision making methodology for assessing e-commerce website quality : A case study in Turkey. In World Congress on Engineering 2011 (Vol. II, pp. 5–9). London, UK. Retrieved from http://iaeng.org/publication/WCE2011/WCE2011_pp1174-1178.pdf
Aydin, S., & Kahraman, C. (2012). Evaluation of e-commerce website quality using fuzzy multi-criteria decision making approach. AENG International Journal of Computer Science, 39(1). Retrieved from https://pdfs.semanticscholar.org/d17a/92e66a83ff08cd5fbd4de87d954 4c5fc75c7.pdf
Aytuna, N., Karsak, B., & Albayrak, Y. E. (2012). A proposed model for the Turkish political parties web sites’ efficiency : An integrated method using analytic hierarchy process. International Journal of Information Technology and Business Management, 8(1), 55–72.
Bali, O., Burmaoğlu, S., & Kazançoğlu, Y. (2010). Performance evaluation of transportation websites by using fuzzy AHP. In International logistics and supply chain congress. Istanbul, Turkey. Journal of ICT, 19, No. 3 (July) 2020, pp: 399-
Behzadian, M., Kazemzadeh, R. B., Albadvi, A., & Aghdasi, M. (2010). PROMETHEE : A comprehensive literature review on methodologies and applications. European Journal of Operational Research, 200(1), 198–215. https://doi.org/10.1016/j.ejor.2009.01.021
Behzadian, M., Khanmohammadi, S., Morteza, Y., & Ignatius, J. (2012). A state-of the-art survey of TOPSIS applications. Expert Systems with Applications, 39(17), 13051–13069. https://doi.org/10.1016/j. eswa.2012.05.056
Bijan, Y., & Salehi, M. (2013). Comparison of user satisfaction of ecommerce websites by the analytic network process. International Journal of the Analytical Hierarchy Process, 5(2), 223–251. Retrieved from http://ijahp.org/index.php/IJAHP/article/view/180
Bilsel, R. U., Büyüközkan, G., & Ruan, D. (2006). A fuzzy preference-ranking model for a quality evaluation of hospital web sites. International Journal Of Intelligent Systems, 21(21), 1181–1197. https://doi.org/10.1002/int.20177
Blagec, K., Romagnoli, K. M., Boyce, R. D., & Samwald, M. (2016). Examining perceptions of the usefulness and usability of a mobile-based system for pharmacogenomics clinical decision support: A mixed methods study. PeerJ, 4, 1–24. https://doi.org/10.7717/peerj.1671
Burmaoglu, S., & Kazancoglu, Y. (2012). E-government website evaluation with hybrid MCDM method in fuzzy environment. International Journal Applied Decision Sciences, 5(2), 163–181. https://doi.org/10.1504/ IJADS.2012.046504
Büyüközkan, G., Arsenyan, J., & Ertek, G. (2010). Evaluation of e-learning web sites using fuzzy axiomatic design based approach. International Journal of Computational Intelligence Systems, 3(1), 28–42. https://doi.org/10.1080/18756891.2010.9727675
Byun, D., & Finnie, G. (2011). An AHP method for evaluating usability of electronic government portals. Electronic Government, An International Journal, 8(4), 343–362. https://doi.org/https://doi.org/10.1504/ EG.2011.042811
Calisir, F., Bayraktaroglu, A. E., Gumussoy, C. A., Topcu, Y. I., & Mutlu, T. (2009). The relative importance of usability and functionality factors for online auction and shopping web sites. Online Information Review, 34(3), 420–439. https://doi.org/10.1108/14684521011037025
Castro-Lopez, A., Puente, J., & Vazquez-Casielles, R. (2017). Fuzzy inference suitability to determine the utilitarian quality of B2C websites. Applied Soft Computing Journal, 57, 132–143. https://doi.org/10.1016/j. asoc.2017.03.039
Ceballos, B., Teresa, M., & David, L. (2016). A comparative analysis of multi-criteria decision-making methods. Progress in Artificial Intelligence. https://doi.org/10.1007/s13748-016-0093-1 Journal of ICT, 19, No. 3 (July) 2020, pp: 399-Çelik, P., & Cansu, G. K. (2017). Fuzzy AHP-fuzzy PROMETHEE approach in evaluation of e-service quality: Case of airline web sites. The Journal of International Social Research, 10(52), 1003–1013. https://doi.org/10.17719/jisr.2017.1954
Chou, W., & Cheng, Y. (2012). A hybrid fuzzy MCDM approach for evaluating website quality of professional accounting firms. Expert Systems With Applications, 39(3), 2783–2793. https://doi.org/10.1016/j. eswa.2011.08.138
Das, T., & Patil, S. R. (2014). A review of current trends in usability evaluation methods. International Journal of Engineering Research & Technology (IJERT), 3(9), 837–840.
Delice, E. K., & Gungor, Z. (2009). The usability analysis with heuristic evaluation and analytic hierarchy process. International Journal of Industrial Ergonomics, 39, 934–939. https://doi.org/10.1016/j. ergon.2009.08.005
Dey, S., Jana, B., Gourisaria, M. K., Mohanty, S. N., & Chatterjee, R. (2015). Evaluation of Indian B2C e-shopping websites under multi criteria decision-making using fuzzy hybrid technique. International Journal of Applied Engineering Research, 10(9), 24551–24580.
Diaby, V., Campbell, K., & Goeree, R. (2013). Multi-criteria decision analysis ( MCDA ) in health care : A bibliometric analysis. Operations Research for Health Care, 2(1–2), 20–24. https://doi.org/10.1016/j. orhc.2013.03.001
Diniz, E., Porto, R. M., & Adachi, T. (2005). Internet Banking in Brazil : Evaluation of functionality, reliability and usability. Electronic Journal of Information Systems Evaluation Volume, 8(1), 41–50.
Djordj, N. D., Rančić, D. D., & Vulić, I. (2013). The relative importance of the usability factors for web portals. In Telecommunication in Modern Satellite, Cable and Broadcasting Services (TELSIKS) (pp. 571–574). Serbia: IEEE. https://doi.org/10.1109/TELSKS.2013.6704443
Dodgson, J. S., Spackman, M., Pearman, A., & Phillips, L. D. (2009). Multi-criteria analysis : A manual. London: Department for Communities and Local Government.
Dominic, P. D. D., Jati, H., & Hanim, S. (2013). University website quality comparison by using non-parametric statistical test: A case study from Malaysia. International Journal of Operational Research, 16(3), 349–374. https://doi.org/https://doi.org/10.1504/IJOR.2013.052335
Dominic, P. D. D., & Jati, H. (2010). Evaluation method of Malaysian university website : Quality website using hybrid method. In Information Technology International Symposium (ITSim) (pp. 1–6). Kuala Lumpur, Malaysia: IEEE. https://doi.org/10.1109/ITSIM.2010.5561363
Dominic, P. D. D., & Jati, H. (2011). A comparison of Asian airlines websites quality: Using a non-parametric test. International Journal of Business Innovation and Research, 5(5), 599–623. Journal of ICT, 19, No. 3 (July) 2020, pp: 399-
Dominic, P. D. D., Jati, H., & Kannabiran, G. (2010). Performance evaluation on quality of Asian e-government websites – an AHP approach. International Journal of Business Information Systems, 6(2), 219–239.
Dominic, P. D. D., Jati, H., & Sellappan, P. (2011). A comparison of Asian e-government websites quality : Using a non-parametric test. International Journal of Business Information Systems, 7(2), 220–246. https://doi.org/https://doi.org/10.1504/IJBIS.2011.038513
Ecer, F. (2014). A hybrid banking websites quality evaluation model using AHP and COPRAS-G : A Turkey case. Technological and Economic Development of Economy, 20(4), 758–782. https://doi.org/10.3846/20 294913.2014.915596
Federici, S., & Borsci, S. (2017). Usability evaluation : Models, methods, and applications. In International Encyclopedia of Rehabilitation 2010 (pp. 1–17). Retrieved from http://cirrie.buffalo.edu/encyclopedia/en/ article/277/
Fernandez, A., Insfran, E., & Abrahão, S. (2011). Usability evaluation methods for the web : A systematic mapping study. Information and Software Technology, 53(8), 789–817. https://doi.org/10.1016/j. infsof.2011.02.007
Garg, R., & Jain, D. (2017). Fuzzy multi-attribute decision making evaluation of e-learning websites using FAHP, COPRAS, VIKOR, WDBA. Decision Science Letters 6, 351–364. https://doi.org/10.5267/j.dsl.2017.2.003
Govindan, K., & Jepsen, M. B. (2015). ELECTRE: A comprehensive literature review on methodologies and applications. European Journal of Operational Research, 250(1), 1–29. https://doi.org/10.1016/j. ejor.2015.07.019
Grigera, J., Garrido, A., Zaraté, P., Camilleri, G., Fernández, A., Dria, A., … Cedex, T. (2018). A mixed usability evaluation on a multi criteria group decision support system in agriculture. In Interactiion. ACM. Retrieved from https://doi.org/10.1145/3233824.3233852
Guimei, Z., & Taowei, Y. (2012). A quantitative research into the usability evaluation of China’s e-government websites. In 2nd International Congress on Computer Applications and Computational Science (pp. 437–442). Berlin Heidelberg: Springer. Retrieved from https://link. springer.com/chapter/10.1007/978-3-642-28314-7_58
Gupta, S. (2015). A comparative study of usability evaluation methods. International Journal of Computer Trends and Technology, 22(3), 103–106. Retrieved from http://www.ijcttjournal.org/2015/Volume22/ number-3/IJCTT-V22P121.pdf
Ho, W., Xu, X., & Dey, P. K. (2010). Multi-criteria decision making approaches for supplier evaluation and selection : A literature review. European Journal of Operational Research, 202(1), 16–24. https://doi.org/10.1016/j.ejor.2009.05.009 Journal of ICT, 19, No. 3 (July) 2020, pp: 399-
Hu, Y. (2009). Fuzzy multiple-criteria decision making in the determination of critical criteria for assessing service quality of travel websites. Expert Systems With Applications, 36(3), 6439–6445. https://doi.org/10.1016/j. eswa.2008.07.046
Jain, D., Garg, R., & Bansal, A. (2015). A parameterized selection and evaluation of e-learning websites using topsis method. International Journal of Research & Development in Technology and Management Science–Kailash, 22(3), 12–26. Retrieved from http://journal.rtmonline. in/vol22iss3/0530111.pdf
Jaroslaw, W., Paweł, Z., & Jarosław, J. (2016). PEQUAL - E-commerce websites quality evaluation methodology. In Computer Science and Information Systems (FedCSIS), 2016 Federated (Vol. 8, pp. 1317–1327). Gdansk, Poland. https://doi.org/10.15439/2016F469
Kabak, M., & Burmaoğlu, S. (2013). A holistic evaluation of the e-procurement website by using a hybrid MCDM methodology. Electronic Government, An International Journal, 10(2), 125–150. https://doi.org/https://doi.org/10.1504/EG.2013.052598
Kabir, G., & Hasin, M. A. A. (2012). Comparative analysis of topsis and fuzzy topsis for the evaluation of travel website service quality. International Journal for Quality Research, 6(3), 169–185.
Kabir, G., & Sutana, R. (2013). Evaluation of travel website service quality using fuzzy topsis. Suranaree Journal of Science and Technology, 20(1), 21–33. Retrieved from https://www.researchgate.net/publicatio n/267240857%0Aevaluation
Karabasevic, D., & Stanujkic, D. (2019). An approach to evaluating the quality of websites based on the weighted sum preferred levels of performances. Acta Polytechnica Hungarica, 16(5), 195–215.
Kaya, T. (2010). Multi-attribute evaluation of website quality in e-business using an integrated fuzzy AHP TOPSIS methodology. International Journal of Computational Intelligence Systems, 3(3), 301–314. https://doi.org/10.1080/18756891.2010.9727701
Kaya, T., & Kahraman, C. (2011). A fuzzy approach to e-banking website quality assessment based on an integrated AHP-ELECTRE method. Technological and Economic Development of Economy, 17(2), 37–41. https://doi.org/10.3846/20294913.2011.583727
Kaya, T., & Kahraman, C. (2013). A fuzzy approach to e-banking website quality assessment based on an integrated AHP-ELECTRE method. Technological and Economic Development of Economy, 17(2), 313–334.
Khan, H., & Dominic, P. D. D. (2013). Measuring quality of Asian airline websites using analytical hierarchy process : A future customer satisfaction approach. In Information Systems International Conference Journal of ICT, 19, No. 3 (July) 2020, pp: 399- (ISICO) (pp. 67–72). Departemen Sistem Informasi, Institut Teknologi Sepuluh Nopember, Indonesia.
Kitchenham, B., & Charters, S. (2007). Guidelines for performing systematic literature reviews in software engineering.
Kong, F., & Liu, H. (2005). Applying fuzzy analytic hierarchy process to evaluate success factors of e-commerce. International Journal of Information and Systems Sciences, 1(3–4), 406–412. Retrieved from https://pdfs.semanticscholar.org/6d54/058dd182ca15403d4e99b105f8 52edbb5433.pdf
Kostoglou, V., Papathanasiou, J., & Moschidis, O. (2014). Α comparative analysis of Greek universities’ presence on the World Wide Web using an analytical MCDM methodology. International Journal of Multicriteria Decision Making, 4(3), 279–298. https://doi.org/https://doi.org/10.1504/IJMCDM.2014.064794
Kubler, S., Robert, J., Derigent, W., Voisin, A., & Le, Y. (2016). A state-of the-art survey and testbed of fuzzy AHP (FAHP) applications. Expert Systems With Applications, 65, 398–422. https://doi.org/10.1016/j. eswa.2016.08.064
Lamichhane, R., & Meesad, P. (2011). A usability evaluation for government websites of Nepal using fuzzy AHP. In M. Phayung (Ed.), I7th International Conference on Computing and Information Technology IC2IT2011 (pp. 99–104). Bangkok, Thailand.
Law, R. (2007). A fuzzy multiple criteria decision-making model for evaluating travel websites. Asia Pacific Journal of Tourism Research, 12(2), 147–159. https://doi.org/10.1080/10941660701243372
Lee, C., Chiang, C., & Chen, C. (2012). An evaluation model of e-service quality by applying hierarchical fuzzy topsis method. International Journal of Electronic Business Management, 10(1), 38–49. Retrieved from http://dblp.org/db/journals/ijebm/ijebm10.html
Lian, T., Yu, C., Wang, Z., & Hou, Z. (2017). The evaluation study on tourism websites : From the perspective of triangular intuitionistic fuzzy multiple attribute group decision making. Journal of Applied Statistics, 44(16), 2877–2889. https://doi.org/10.1080/02664763.2016.1266466
Liang, R., Wang, J., & Zhang, H. (2017). Evaluation of e-commerce websites: An integrated approach under a single-valued trapezoidal neutrosophic environment. Knowledge-Based Systems, 135, 44–59. https://doi.org/10.1016/j.knosys.2017.08.002
Lin, H. (2010). An application of fuzzy AHP for evaluating course website quality. Computers & Education, 54(4), 877–888. https://doi.org/10.1016/j.compedu.2009.09.017
Liu, H., & Zhang, S. (2011). Approach of fuzzy analytic hierarchy process to evaluate e-commerce websites, 248–252. Retrieved from http://elib. bsu.by/handle/123456789/10032 Journal of ICT, 19, No. 3 (July) 2020, pp: 399-
Mardani, A., Jusoh, A., Nor, K., Khalifah, Z., & Valipour, A. (2015). Multiple criteria decision-making techniques and their applications – a review of the literature from 2000 to 2014. Economic Research-Ekonomska Istraživanja, 28(1), 516–571. https://doi.org/10.1080/1331677X.2015. 1075139
Mardani, A., Jusoh, A., Zavadskas, E. K., & Nor, K. (2015). Application of multiple-criteria decision-making techniques and approaches to evaluating of service quality : A systematic review of the literature. Journal of Business Economics and Management, 16(5), 1034–1068. https://doi.org/10.3846/16111699.2015.1095233
Mardani, A., Jusoha, A., & Zavadskas, E. K. (2015). Fuzzy multiple criteria decision-making techniques and applications – Two decades review from 1994 to 2014. Expert Systems with Applications, 42(8), 4126–4148. https://doi.org/10.1016/j.eswa.2015.01.003
Mardani, A., Zavadskas, E. K., & Govindan, K. (2016). VIKOR Technique : A systematic review of the state of the art literature on methodologies and applications. Sustainability, 8(37). https://doi.org/10.3390/su8010037
Markaki, O. I., Charilas, D. E., & Askounis, D. (2010). Application of fuzzy analytic hierarchy process to evaluate the quality of e-government web sites. In Developments in E-systems Engineering (pp. 219–224). IEEE. https://doi.org/10.1109/DeSE.2010.42
Masudin, I., & Saputro, T. E. (2016). Evaluation of B2C website based on the usability factors by using fuzzy AHP & hierarchical fuzzy TOPSIS. IOP Conf. Series: Materials Science and Engineering, 114. https://doi.org/10.1088/1757-899X/114/1/012091
Mirbargkar, S. M., & Zadmehr, O. (2015). Investigating the quality of electronic services using multiple attribute decision-making techniques case study (Aria, Ghaem, and Golsar Hospitals). Science Journal (CSJ), 36(6), 1233–1240. Retrieved from http://dergi.cumhuriyet.edu.tr/cumuscij/ article/view/5000139552
Mohsen, P., Madjid, T., Mimi, H. H., & Majid, B. (2011). A taxonomy and review of the multiple criteria decision-making literature in chemical engineering. International Journal of Multicriteria Decision Making, 1(4), 407–467. Retrieved from https://doi.org/10.1504/ IJMCDM.2011.043556
Nagpal, R., Mehrotra, D., & Bhatia, P. (2016). Usability evaluation of website using combined weighted method: Fuzzy AHP and entropy approach. International Journal of System Assurance Engineering and Management, 7(4), 408–417. https://doi.org/10.1007/s13198-016-0462-y
Nagpal, R., Mehrotra, D., & Bhatia, P. (2017). The state of art in website usability evaluation methods. In S. Saeed, Y. A. Bamarouf, T. Ramayah, Journal of ICT, 19, No. 3 (July) 2020, pp: 399- & S. Z. Iqbal (Eds.), Design Solutions for User-Centric Information Systems (Vol. 1, pp. 275–296). USA: IGI Global. Retrieved from http://www.igi-global.com
Nagpal, R., Mehrotra, D., Bhatia, P., & Sharma, A. (2015a). FAHP approach to rank educational websites on usability. International Journal of Computing and Digital Systems, 4(4), 251–260. https://doi.org/10.12785/ ijcds/040404
Nagpal, R., Mehrotra, D., Bhatia, P., & Sharma, A. (2015b). Rank university websites using fuzzy AHP and Fuzzy TOPSIS approach on usability. International Journal of Information Engineering and Electronic Business, 10(1), 29–36. https://doi.org/10.5815/ijieeb.2015.01.04
Nawaz, A., & Clemmensen, T. (2013). Website usability in Asia ‘from within’: An overview of a decade of literature. International Journal of Human-Computer Interaction, 29(4), 256–273. https://doi.org/https://doi.org/1 0.1080/10447318.2013.765764
Nirmala, G., & Uthra, G. (2017). Quality of online shopping websites in India : A study using intuitionistic fuzzy AHP. Journal of Advance Research in Dynamical & Control Systems, 9(4), 117–124.
Pathania, A., & Rasool, G. (2017). Investigating e tailer’s perceived website quality using analytical hierarchy process technique. Procedia Computer Science, 122, 1016–1023. https://doi.org/10.1016/j.procs.2017.11.468
Paz, F., & Pow-Sang, J. A. (2014). Current trends in usability evaluation methods: A systematic review. In 7th International Conference on Advanced Software Engineering and Its Applications (ASEA) (pp. 11–15). IEEE. https://doi.org/10.1109/ASEA.2014.10
Pearson, J. M., & Pearson, A. M. (2008). An exploratory study into determining the relative importance of key criteria in web usability : A multi-criteria approach. Journal of Computer Information Systems, 48(4), 115–127. https://doi.org/10.1080/08874417.2008.11646041
Perçin, S. (2017). Evaluating airline service quality using a combined fuzzy decision-making approach. Journal of Air Transport Management, 68, 48–60. https://doi.org/10.1016/j.jairtraman.2017.07.004
Percin, S., & Aldalou, E. (2018). Financial performance evaluation of Turkish airline companies using integrated fuzzy AHP fuzzy topsis model, 583–598. https://doi.org/10.18092/ulikidince.347925
Presley, A., & Fellows, P. (2013). An analytic hierarchy process model for evaluating and comparing website usability. International Journal of Business Information Systems, 12(2), 123–139. https://doi.org/https://doi.org/10.1504/IJBIS.2013.052047
Qi, S., Law, R., & Buhalis, D. (2017a). Comparative evaluation study of the websites of China-based and international luxury hotels. Journal of China Tourism Research, 00(00), 1–25. https://doi.org/10.1080/19388 160.2017.1289135 Journal of ICT, 19, No. 3 (July) 2020, pp: 399-
Qi, S., Law, R., & Buhalis, D. (2017b). Comparative evaluation study of the websites of China-based and international luxury hotels. Journal of China Tourism Research, 13(1), 1–25. https://doi.org/10.1080/193881 60.2017.1289135
Quiñones, D., & Rusu, C. (2017). How to develop usability heuristics : A systematic literature review. Computer Standards & Interfaces, 53, 89–122. https://doi.org/10.1016/j.csi.2017.03.009
Rekik, R., Kallel, I., & Alimi, A. M. (2016). Ranking criteria based on fuzzy ANP for assessing e-commerce web sites. In IEEE International Conference on Systems, Man, and Cybernetics (pp. 3469–3474). Hungary: IEEE. https://doi.org/10.1109/SMC.2016.7844770
Rekik, R., Kallel, I., Casillas, J., & Alimi, A. M. (2016). Using multiple criteria decision making approaches to assess the quality of web sites. International Journal of Computer Science and Information Security, 14(7), 747–761.
Rekik, R., Kallel, I., Casillas, J., & Alimi, A. M. (2018). Assessing web sites quality: A systematic literature review by text and association rues mining. International Journal of Information Management, 38(1), 201–216. https://doi.org/10.1016/j.ijinfomgt.2017.06.007
Rekik, R., Kallell, I., & Alimi, A. M. (2015). Quality evaluation of web sites : A comparative study of some multiple criteria decision making methods. In 1ntelliegent Systems Design and Applications (ISDA) 15th International Conference (pp. 585–590). Marrakech, Morocco: IEEE. https://doi.org/10.1109/ISDA.2015.7489183
Roy, Jagannath Kajal Chatterjee, K., Bandyopadhyay, A., & Kar, S. (2018). Evaluation and selection of medical tourism sites: A rough Analytic Hierarchy Process based multi–attributive border approximation area comparison approach. Expert Systems, 35(1). https://doi.org/10.1111/ exsy.12232
Roy, S., Kumar, P., & Rajib, P. (2016). Quality assurance of academic websites using usability testing : An experimental study with AHP. International Journal of System Assurance Engineering and Management, 8(1), 1–11. https://doi.org/10.1007/s13198-016-0436-0
Roy, S., & Pattnaik, P. K. (2014). Some popular usability evaluation techniques. In S. C. Satapathy, S. K. Udgata, & B. N. Biswal (Eds.), FICTA 2013 (pp. 535–543). Switzerland: Springer International Publishing. https://doi.org/10.1007/978-3-319-02931-3_61
Roy, S., Pattnaik, P. K., & Mall, R. (2014). A quantitative approach to evaluate usability of academic websites based on human perception. Egyptian Informatics Journal. https://doi.org/10.1016/j.eij.2014.08.002
Saaty, T. L., & Ozdemir, M. S. (2003). Why the magic number seven plus or minus two. Mathematical and Computer Modelling, 38, 233–244. Journal of ICT, 19, No. 3 (July) 2020, pp: 399-
Safari, H., & Aghasi, E. (2014). Select a hypermarket location based on fuzzy multi criteria decision making (F-MCDM) techniques. Arabian Journal of Business and Management Review, 4(1), 76–95. Retrieved from https://search.proquest.com/openview/bf7c290421907d199c67262b7c 55a0de/1?pq-origsite=gscholar&cbl=1606372
Sagar, K., & Saha, A. (2017). Qualitative usability feature selection with ranking: A novel approach for ranking the identified usability problematic attributes for academic websites using data-mining techniques. Human-Centric Computing and Information Sciences, 7(1), 1–24. https://doi.org/10.1186/s13673-017-0111-8
Samanlioglu, F., & Ayag, Z. (2018). A fuzzy AHP-VIKOR approach for evaluation of educational use simulation software packages. Journal of Intelligent & Fuzzy Systems. https://doi.org/10.3233/JIFS-172290
Shahin, A., Pool, J. K., & Poormostafa, M. (2014). Evaluating and ranking hotels offering e-service by integrated approach of Webqual and fuzzy AHP. International Journal of Business Information Systems, 15(1), 84–104. Retrieved from https://www.inderscienceonline.com/doi/ abs/10.1504/IJBIS.2014.057966
Si, S., You, X., Liu, H., & Zhang, P. (2018). DEMATEL Technique : A systematic review of the state-of-the-art literature on methodologies and applications. Mathematical Problems in Engineering, (1). Retrieved from https://doi.org/10.1155/2018/3696457
Simon, N., Carbonera, B. J., & Custodio, B. (2017). A comparative study on the usability of educational platforms used by instructors in the university of the Philippines. In I. K. Jussi, T. Barath, N. Salman, & T. Andre (Eds.), Advances in Human Factors, Business Management, Training and Education (pp. 187–194). Springer Cham. https://doi.org/https://doi.org/10.1007/978-3-319-42070-7_18
Soleymaninejad, M., Shadifar, M., & Karimi, A. (2016). Evaluation of two major online travel agencies of US using TOPSIS method. Digital Technologies, 2(1), 1–8. https://doi.org/10.12691/dt-2-1-1
Speicher, M. (2015). What is Usability? A Characterization based on ISO 9241-11 and ISO/IEC 25010. ArXiv Preprint ArXiv, 1–10. Retrieved from http://arxiv.org/abs/1502.06792
Stanujkic, D., Paunovic, M., & Stankovic, G. (2014). Multi-criteria model for evaluating quality of websites of the regional tourism organizations. In International Conference on Information and Software Technologies (pp. 311–316). Druskininka: Springer. https://doi.org/10.13140/2.1.1498.7846
Stewart, T. (2012). Websites – quality and usability. Behaviour & Information Technology, 31(7), 645–646. https://doi.org/10.1080/0144929X.2012.7 07372 Journal of ICT, 19, No. 3 (July) 2020, pp: 399-
Sun, C., & Lin, G. T. R. (2009). Using fuzzy TOPSIS method for evaluating the competitive advantages of shopping websites. Expert Systems With Applications, 36(9), 11764–11771. https://doi.org/10.1016/j. eswa.2009.04.017
Sun, S., Fong, D. K. C., Law, R., & He, S. (2017). An updated comprehensive review of website evaluation studies in hospitality and tourism. International Journal of Contemporary Hospitality Management, 29(1), 355–373. https://doi.org/10.1108/IJCHM-12-2015-0736
Syke, H. S., Nina, D. N. B., Syke, J. M., & Keune, H. (2013). Multi-criteria decision analysis ( MCDA ) in ecosystem service valuation, 1, 1–6.
Tavana, M., Yazdani, M., & Di-Caprio, D. (2017). An application of an integrated ANP-QFD framework for sustainable supplier selection. International Journal of Logisitics Research and Applications, 20(3), 254–275. https://doi.org/10.1080/13675567.2016.1219702
Tsai, W., Chou, W., & Lai, C. (2010). An effective evaluation model and improvement analysis for national park websites : A case study of Taiwan. Tourism Management, 31(6), 936–952. https://doi.org/10.1016/j. tourman.2010.01.016
Ugras, T., Gulsecen, S., Cubukcu, C., Erdognus, I. lli, Gashi, V., & Bedir, M. (2016). Research trends in web site usability A sytematic review. pdf. In A. Marcus (Ed.), Design, User Experience and Usability. 5th International Conference of HCI International (pp. 517–528). Switzerland: Springer International Publishing.
Vatansever, K., & Akgu, Y. (2014). Applying fuzzy analytic hierarchy process for evaluating service quality of private shopping website quality: A case study in Turkey. Journal of Business, Economics and Finance, 3(3), 283–301. Retrieved from http://dergipark.gov.tr/jbef/ issue/32410/360451
Velasquez, M., & Hester, P. T. (2013). An analysis of multi-criteria decision making methods. International Journal of Operations Research, 10(2), 56–66.
Wanke, P., Azad, A. K., Barros, C. P., & Hadi-vencheh, A. (2015). Predicting performance in ASEAN banks: An integrated fuzzy MCDM–neural network approach. Expert Systems, 00(00). https://doi.org/10.1111/ exsy.12144
Wardoyo, R., & Wahyuningrum, T. (2018). University website quality ranking using logarithmic fuzzy preference programming. International Journal of Electrical and Computer Engineering, 8(5), 3349–3358. https://doi.org/10.11591/ijece.v8i5.pp3349-3358
Wen-Hsien, T., Jun-Der, L., & Wen-Chin, C. (2009). The development of an evaluation model of e-commerce websites for the Taiwanese airline industry. Development, 1, 1–2009. Retrieved from https://pdfs. Journal of ICT, 19, No. 3 (July) 2020, pp: 399-semanticscholar.org/234b/d3b742f7e678985549d56491a1fd48ee2581. pdf
Yi-wen, L., Kwon, Y., & Kang, B. (2007). A fuzzy AHP approach to evaluating e-commerce websites. In Fifth International Conference on Software Engineering Research, Management and Applications (pp. 114–122). IEEE. https://doi.org/10.1109/SERA.2007.29
Younghwa, L., & Kenneth, K. A. (2006). Investigating the effect of website quality on e-business success : An analytic hierarchy process ( AHP) approach. Decision Support Systems, 42, 1383–1401. https://doi.org/10.1016/j.dss.2005.11.005
Yu, X., Guo, S., Guo, J., & Huang, X. (2011). Rank B2C e-commerce websites in e-alliance based on AHP and fuzzy TOPSIS. Expert Systems With Applications, 38(4), 3550–3557. https://doi.org/10.1016/j. eswa.2010.08.143
Zaim, H., Ramdani, M., & Haddi, A. (2016). Multi-criteria analysis approach based on consumer satisfaction to rank B2C e-commerce websites. In 2016 11th International Conference on Intelligent Systems: Theories and Applications (SITA 2016). Mohammedia, Morocco: IEEE. https://doi.org/10.1109/SITA.2016.7772260
Zare, M., Pahl, C., Rahnama, H., Nilashi, M., Mardani, A., Ibrahim, O., & Ahmadi, H. (2016). Multi-criteria decision making approach in e-learning: A systematic review and classification. Applied Soft Computing Journal, 45, 108–128. https://doi.org/10.1016/j.asoc.2016.04.020
Zavadskas, K., Turskis, Z., & Simona, K. (2014). State of art surveys of overviews on MCDM/MADM methods. Technological and Economic Development of Economy, 20(1), 165–179. https://doi.org/10.3846/202 94913.2014.892037
Zhang, Y., Zhao, X., Feng, L., & Li, X. (2015). The usability evaluation of the ecological agriculture park website. International Journal of Multimedia and Ubiquitous Engineering, 10(12), 53–60. https://doi.org/http://dx.doi.org/10.14257/ijmue.2015.10.12.06
Zhou, X., Hu, Y., Deng, Y., Chan, F. T. S., & Ishizaka, A. (2018). A DEMATEL-based completion method for incomplete pairwise comparison matrix in AHP. Annals of Operations Research, 271(2), 1045–1066. https://doi.org/10
Published
Issue
Section
How to Cite
Research impact
Harvested 2026-10-09Counts differ between services because each indexes a different body of literature. None of them is the whole picture.
- OpenCitations 0 View →
- Scopus 0 not indexed View →
- Google Scholar no free count Search →
- Dimensions no free count Search →
2002 - 2020
