MULTI-CRITERIA DECISION-MAKING BASED APPROACHES IN WEBSITE QUALITY AND USABILITY EVALUATION: A SYSTEMATIC REVIEW

Websites are important in every organisation and tremendous effort is made to design websites that not only look and feel good, but are usable and of high quality. Nevertheless, one critical task is how to evaluate these websites to ensure that users are satisfied with its quality and usability. Although a variety of methods and approaches have been proposed, there is currently an increase in research efforts to model website quality and usability evaluation from the point of view of decision-makers which existing methods do not handle. Thus, this has led to the application of multi-criteria decision-making (MCDM) approaches in the evaluation of websites to handle complexity in decision-making. This paper, therefore, provides a review of the various MCDM methods that have been used in the usability and quality evaluation of websites. The search strategy which was adopted identified a total of 63 published articles in peer-Received


INTRODUCTION
Presently, a common platform for information searching by numerous users is through websites and hence, its evaluation has attracted the attention of various practitioners and researchers worldwide.In turn, this has resulted in a tremendous increase in website evaluation research (Blagec, Romagnoli, Boyce, & Samwald, 2016;Roy & Pattnaik, 2014;Simon, Carbonera, & Custodio, 2017;Sun, Fong, Law, & He, 2017).Website evaluation studies may be viewed from the perspective of quality, usability, accessibility, navigability or functionality, among others.Quality evaluation encompasses the totality of the evaluation as a whole, whereas usability evaluation involves the assessment of ease of use of a product in order to identify usability problems (Adepoju, Oyefolahan, Abdullahi, & Mohammed, 2018a;Federici & Borsci, 2017;Speicher, 2015).Functionality evaluation aims at evaluating the extent to which the functions embedded in a product are performing expected tasks (Diniz, Porto, & Adachi, 2005).
One of the most important factors in website quality is website usability, and the evaluation is conducted in order to obtain measures of usability.The aim of the evaluation is targeted at improving usability or determining to what extent usability objectives have been achieved.Both website quality and usability evaluation encompass many criteria and have been evaluated using different quality models (Alptekin, Hall, & Sevim, 2015;Jaroslaw, Paweł, & Jarosław, 2016).Website usability can be carried out using a variety of methods.These are categorized as expert-based method, user and tool-based methods, soft computing based methods, analytical modelling based method and multiple-criteria decision-making (MCDM) based methods (Adepoju et al., 2018a;Fernandez, Insfran, & Abrahão, 2011;Nagpal, Mehrotra, & Bhatia, 2017).Among these methods, MCDM is gaining popularity and attracting researchers' attention of late.
The focus of this paper is on the review of the usage of MCDM approaches in website usability and quality evaluation.The purpose of this scholarly article is to identify trends in the literature on the application of MCDM approaches in the usability and quality evaluation of websites.Besides, this study looks at which genre of websites have been extensively evaluated using MCDM methods, the commonly used criteria, the number of criteria used in the studies including the location by country where the websites were used in previous studies.In addition to this, trends on the most widely used standalone and integrated MCDM methods are also investigated.The review considers papers that appeared in reputable journals and conference proceedings from 2005 to 2017.Conference papers are included because it is also an important avenue to disseminate the latest research results in the field of computer science.
Existing studies focused mainly on website quality using the MCDM approach and did not adopt a comprehensive literature review in the studies (Rekik, Kallel, Casillas, & Alimi, 2016;Rekik, Kallell, & Alimi, 2015).Others focused on a particular type of MCDM methodology (Asemi et al., 2013;Behzadian, Kazemzadeh, Albadvi, & Aghdasi, 2010;Govindan & Jepsen, 2015;Mardani, Zavadskas, & Govindan, 2016).Hence, this paper aims to conduct a comprehensive review of MCDM approaches combining both website usability and quality evaluation based on available information.The remainder of this paper is organised as follows: The following section presents the overview of MCDM methods, followed by a description on the research methodology.Analysis of the review results is discussed after that and subsequently, the conclusion and suggestions for future work.

OVERVIEW OF MULTI-CRITERIA DECISION-MAKING METHODS
There have been efforts in the past to conduct a review on website quality and website usability.Some of these studies focused mainly on website usability evaluation trends (Das & Patil, 2014;Fernandez et al., 2011;Gupta, 2015;Nawaz & Clemmensen, 2013;Paz & Pow-Sang, 2014).Recently, the use of MCDM or otherwise known as multiple-criteria decision analysis (MCDA) approach in measuring usability and quality of web interfaces is gaining prominence.MCDA is largely used when solving complex decision problems which involves multiple criteria by selecting the best alternative (Roy, Pattnaik, & Mall, 2014;Stewart, 2012).According to Nagpal et al. (2017), MCDA entails studying the methods and procedures that incorporate multiple and conflicting criteria in the decision-making process.MCDM can be subdivided into: multi-attribute decision-making (MADM) and multi-objective decisionmaking (MODM).MADM methods involve solving problems with discrete decision spaces based on a predetermined or a restricted number of choices (Kubler, Robert, Derigent, Voisin, & Le, 2016).It involves making selections among some courses of action in the presence of multiple usually conflicting finite attributes.
As described earlier MCDM involves subjective evaluation of a finite number of alternatives under a finite number of criteria by a decision-maker or group expert through the design of computational and mathematical tools (Kubler et al., 2016;Si, You, Liu, & Zhang, 2018).It simply involves making a decision by selecting the best alternative in the presence of multiple criteria.There are different types of MCDM methods.These include: Analytical Hierarchy Processing (AHP), Technique for Order Preference by Similarity to Ideal Solution (TOPSIS), Analytical Network Processing (ANP), Preference Ranking Organization METHod for Enrichment Evaluations (PROMETHEE), Decision-Making Trial and Evaluation Laboratory (DEMATEL), ELimination Et Choix Traduisant la Réalité or Elimination and Choice Translating Reality (ELECTRE), Multi-Attribute Utility Theory (MAUT) and Vlse Kriterijumska Optimizacija I Kompromisno Resenje (VIKOR).Some of these methods are described briefly as follows.
AHP was developed by Saaty and it involves modelling decisionmaking problems in hierarchies of goals, criteria, sub-criteria and alternatives.Pairwise comparisons of the criteria are performed to measure relative importance at each hierarchical level and the alternatives are evaluated from the lowest to the highest level of the hierarchy in making decisions (Roy, Jagannath Kajal Chatterjee, Bandyopadhyay, & Kar, 2018;Samanlioglu & Ayag, 2018).ANP is a generic form of AHP which gives room for more relationship, interdependence, complexity and feedback among the hierarchy elements.It enables functional interdependency among decision-makers to be defined among the deciding factors which belong to different levels and the same level of a hierarchy (Rekik, Kallel, & Alimi, 2016;Tavana, Yazdani, & Di-Caprio, 2017).Fuzzy AHP is another variant of AHP proposed earlier by Saaty.It was developed to handle subjectivity and uncertainty observed in complex decision problems.It is integrated with fuzzy set theory.
TOPSIS is an MCDM technique that is used in identifying solutions from a finite set of alternatives based on their distances from positive ideal solutions (PIS) and negative ideal solutions (NIS).The alternatives to be chosen have the shortest distance from the PIS and the farthest distance from the NIS (Alptekin et al., 2015;Ceballos, Teresa, & David, 2016).PROMETHEE belongs to the family of outranking methods in MCMD and was first introduced by Brans in 1982 for partial and complete ranking of alternatives.As an outranking method, a finite set of alternative actions has to be ranked and selected among many conflicting criteria.With the advancement of this method, the iterations of this method have improved (Çelik & Cansu, 2017;Zavadskas et al., 2014).
ELECTRE was developed in 1965 and is an outranking method that is used for choosing the best action from a given set of actions.It consists of aggregation and exploitation phases.The aggregation phase involves the construction of pairwise comparisons with the use of concordance and non-discordance concepts.The second phase of exploitation is aimed at constructing and presenting expected results for a given problem.In addition to the decision matrix, preference information in the form of weights, thresholds, and other parameters are vital in ELECTRE (Govindan & Jepsen, 2015;Kaya & Kahraman, 2013).DEMATEL is a structural modelling approach used in analysing the cause and effect relationships among system components.It helps managers to know how to assess direct and indirect relations and to construct a map to reflect relative relationships within them.This can be used to investigate further to solve complicated problems (Si et al., 2018;Zhou, Hu, Deng, Chan, & Ishizaka, 2018).
The key aim of MCDM methods is to assist decision-makers to deal with complex problems which may take the form of evaluation, selection and prioritisation through the imposition of a disciplined methodology.Website usability and website quality fall can be solved by using MCDM approach as they fall under the selection or evaluation process (Kubler et al., 2016).Rekik et al. (2016) explored known MCDM techniques to assess website information in specific domains or identify current developments in online literature.In general, various steps are involved to solve an MCDM problem as proposed by Dodgson, Spackman, Pearman and Phillips (2009) which is shown in Figure 1.(Dodgson et al., 2009) Past reviews on applications of some MCDM methods in different domains have been carried out by different researchers.Examples of such studies include Zare et al. (2016) who focused on MCDA approaches in e-learning; Das and Patil (2014) who described general usability evaluation methods and Behzadian et al. (2010) who reviewed Preference Ranking Organization Method for Enrichment Evaluation (PROMETHEE) which is a MCDA method.Besides, reviews on MCDA applications was conducted by Mardani et al. (2015) and Ugras et al. (2016) and studies on ELimination and Choice Expressing Reality (ELECTRE) which is also one of the MCDA approaches was conducted by Govindan and Jepsen (2015).
MCDA reviews with a focus on website quality were performed by Rekik et al. (2015) and Rekik et al. (2016).In addition to this, Nagpal et al. (2017) conducted a review of the current trends in usability evaluation methods.There were also studies on research trends in website usability (Ugras et al.,  (Dodgson et al., 2009) Past reviews on applications of some MCDM methods in different domains have been carried out by different researchers.Examples of such studies include Zare et al. (2016) who focused on MCDA approaches in e-learning; Das and Patil (2014) who described general usability evaluation methods and Behzadian et al. (2010) who reviewed Preference Ranking Organization Method for Enrichment Evaluation (PROMETHEE) which is a MCDA method.Besides, reviews on MCDA applications was conducted by Mardani et al. (2015) and Ugras et al. (2016) and studies on ELimination and Choice Expressing Reality (ELECTRE) which is also one of the MCDA approaches was conducted by Govindan and Jepsen (2015).
MCDA reviews with a focus on website quality were performed by Rekik et al. (2015) and Rekik et al. (2016).In addition to this, Nagpal et al. (2017) conducted a review of the current trends in usability evaluation methods.There were also studies on research trends in website usability (Ugras et al., 2016), systematic evaluation of the web (Fernandez et al., 2011) and a review of website usability in the Asian continent (Nawaz & Clemmensen, 2013).Findings from previous studies showed the popular usage of the MCDM approach in different domains which enhanced the need to explore its trend in website evaluation.To the best of our knowledge, there is no study conducted to review MCDM based approaches with a focus on website usability and quality evaluation, using a systematic Identify the objectives and criteria

Examine the results
Combine the weights and scores for each option to derive an overall value

Scoring
Identify the options to be appraised

Sensitivity analysis
Assess the expected performance of each option against the criteria Assess the values associated with the consequences of each option for each criterion Assign weight to each of the criterion to reflect the relative importance to the decision 2016), systematic evaluation of the web (Fernandez et al., 2011) and a review of website usability in the Asian continent (Nawaz & Clemmensen, 2013).Findings from previous studies showed the popular usage of the MCDM approach in different domains which enhanced the need to explore its trend in website evaluation.To the best of our knowledge, there is no study conducted to review MCDM based approaches with a focus on website usability and quality evaluation, using a systematic review approach.Also, existing reviews failed to give a detailed classification as compared to this paper which intends to fill the gap.

RESEARCH METHODOLOGY
This review was undertaken in order to realize the trend in the usage of MCDM approaches in website quality and usability evaluation using the systematic literature review (SLR) approach proposed by Kitchenham and Charters (2007) which has been used recently in several studies (Quiñones & Rusu, 2017;Rekik, Kallel, Casillas, & Alimi, 2018).The SLR process is made up of three stages namely, planning the review, conducting the review and reporting the review.These phases are shown in Figure 2.

RESEARCH METHODOLOGY
This review was undertaken in order to realize the trend in the usage of MCDM approaches in website quality and usability evaluation using the systematic literature review (SLR) approach proposed by Kitchenham and Charters (2007) which has been used recently in several studies (Quiñones & Rusu, 2017;Rekik, Kallel, Casillas, & Alimi, 2018).The SLR process is made up of three stages namely, planning the review, conducting the review and reporting the review.These phases are shown in Figure 2.

Figure 2.
Phases in systematic literature review (Kitchenham & Charters, 2007) The planning stage involves identifying review needs, starting the review, specifying research The planning stage involves identifying review needs, starting the review, specifying research questions, and developing and evaluating review protocol which involves search strategies and terms and study selection criteria.
Conducting the review stage deals with the selection of primary studies, data extraction, data monitoring and data synthesis.Reporting the review involves dissemination of the review report.The details of these are described in the next section.
To guide in the search and presentation of the results, this article attempts to answer the following research questions: RQ1 : What is the number of studies undertaken in the evaluation of website quality and usability using the MCDM approach over the years?RQ2 : What is the genre of website(s) which most commonly apply MCDM methods?RQ3 : What are the MCDM approaches used in website usability and quality evaluation?RQ4 : What is the number of criteria often used in MCDM methods?RQ5 : What is the location of the websites by country used in the studies?
The search strategies used for the study involved searching and collecting articles through the use of several digital library databases including IEEEXplore, Springer Link, Elsevier, Science Direct, Inderscience, and Taylor and Francis.These include major journals and conferences in operation research, computer science and allied disciplines.Also, Database hosts like Scopus and general online search engines such as Google Scholar was also used to search for relevant works.Cross-references that were relevant in reviewed articles were searched to collect papers from standalone journals and publishers whose library databases could not be accessed.Other sources include literature from peer-reviewed academic journals and reputable conferences.Publications from books, dissertations and other unreliable sources were excluded from the study.Only papers published from 2005 to 2017 were included in the study.This is because the authors could not retrieve any articles on MCDM approach in website usability and quality evaluation before 2005.
The search terms used were "website usability" OR "web site usability" OR "web site quality" OR "website quality" AND "multi-criteria decision analysis" OR "multi-criteria decision-making."Articles were analysed based on various MCDM approaches, the number of article publications by year, the number of criteria used, the genre of websites evaluated and country of website affiliation.The search string was applied to the title, abstract and keywords of each article for all the sources.The same search string was applied to all other sources where the articles were manually sourced from in order to ensure conformity.Initially, a total of 902 articles were retrieved from the search.However, after a thorough review and the elimination of duplicate or repeated articles including using search criteria (based on inclusion and exclusion criteria as stated), finally 63 articles comprising 48 journal articles and 15 conference papers were used for the review.

RELATED WORK ON MCDM APPROACHES IN WEBSITE USABILITY AND QUALITY EVALUATION
As mentioned, MCDM methods are frequently used in solving problems which have multiple, inconsistent, and disproportionate criteria and/or objectives in real-world scenarios (Kubler et al., 2016).This simply involves making a decision by selecting the best alternative in the presence of multiple criteria.The key aim of MCDM methods is assisting decision-makers to deal with complex problems which may take the form of evaluation, selection and prioritisation through the imposition of a disciplined methodology.Website usability and website quality fall can be solved by using MCDM approach as they fall under the selection or evaluation process (Kubler et al., 2016).Rekik et al. (2016) explored known MCDM techniques to assess website information in specific domains or identify current developments in online literature.This paper identifies the MCDM methodology and provides a comparison of existing research.The breakdown of the previous studies is shown in Tables 1 and 2. Table 1 shows the studies which focus on the use of MCDM approaches in website usability evaluation and Table 2 presents the MCDM approaches in website quality evaluation.Six studies have been found to apply MCDM approaches as shown in Table 1 and the types of the websites are basically education, e-government, finance, ecological parks, e-commerce including hotels.Altogether there is a total of 12 studies.In contrast to Table 1, MCDM approaches in Table 2 have been widely applied in almost all types of websites including hospitals, airlines, political parties, e-shopping, e-learning and banking.Thus, more focus has been on applying MCDM approaches to website quality compared to website usability.

MCDM Approach in Educational Website Usability and Quality Evaluation
One of the early users of websites comprised academic institutions (colleges, universities, polytechnics, institutes).In considering the role websites play in academic institutions, some studies have used various MCDM approaches in evaluating both the quality and usability of academic websites.These studies are grouped under university, college or portal website evaluation, e-learning website evaluation and library website evaluation.Some of these studies are discussed as follows.
Nagpal, Mehrotra, Bhatia, and Sharma (2015b) used fuzzy AHP to rank four Indian educational institutes on usability.The study criteria were based on response time (RT), ease of use (EOU), ease of navigation (EON) and informative (INF).Further studies by the same authors combined the use of fuzzy AHP and fuzzy TOPSIS to rank four university websites.An integration of fuzzy AHP and entropy approach was used to determine the usability of six academic websites based on the same four criteria as previously used (Nagpal et al., 2016).It was reported in the results that response time contributed the most to academic website usability by using the entropy approach, while easeof-use contributed the most to academic website usability by using Fuzzy AHP.Büyüközkan et al. (2010) evaluated the e-learning website using fuzzy axiomatic and fuzzy TOPSIS.The method incorporated requirements which enabled a reduction in the size of a problem(s).Dominic and Jati (2010) in their study proposed a quality evaluation model based on fuzzy AHP for five university websites in Malaysia.Furthermore, quality evaluation of five university websites in Malaysia using 11 criteria obtained from automated tools was also conducted by Dominic et al. (2013).The outcome of the research indicated that the selected Malaysian universities were not paying adequate attention to performance and quality criteria.The ranking of Greek universities based on quality by using AHP was also conducted by Kostoglou, Papathanasiou, and Moschidis (2014).
The study was based on five criteria and employed the simos method.Results obtained showed that coverage/content and web services received the highest weighting while objectivity and presentation of research obtained the least weighting.
Garg and Jain ( 2017) evaluated e-learning websites using fuzzy AHP for weight selection in their study.A combination of COPRAS (COmplex Proportional ASssessment), VIKOR and WDBA (Weighted Distance Based Approximation) were used for the ranking.The results showed that the developed model used in the study was effective and efficient in its assessment.Jain, Garg, and Bansal (2015) carried out an assessment of the quality of e-learning websites using TOPSIS.A total of 21 e-learning websites based on seven criteria were used for the study.Usability analysis on a university website was also evaluated by combining AHP and Heuristic evaluation method based on the severity of usability problems detected from the website (Delice & Gungor, 2009).Lin (2010) also evaluated course websites using FAHP based on four criteria grouped into 16 sub-criteria among different independent groups.

MCDM Approach in E-commerce Website Usability and Quality Evaluation
E-commerce websites are also very important considering its impact on the economy of any given country.Masudin and Saputro (2016) applied fuzzy AHP and fuzzy TOPSIS to conduct usability evaluation on two e-commerce B2C websites.Five criteria of trustworthiness, shopping support, information access efficiency, ease of comprehension and hedonic quality were applied.It was further subdivided into 15 sub-criteria.The results of fuzzy AHP showed that security and privacy were the most important criteria, followed by trust, loading time, easy transaction, and e-payment support, respectively.The quality of three e-commerce websites in Turkey was examined by using fuzzy AHP and fuzzy VIKOR as stated in a study by Aydin and Kahraman (2012).Five main criteria and 20 sub-criteria were used in the study and the methodology proposed were reported to provide the advantage of having the ability to use both positive and negative fuzzy numbers to evaluate hierarchy.
Fuzzy AHP was also applied to determine success factors in e-commerce websites in a study conducted by Kong and Liu (2005).It involved five criteria of trust, system quality, content quality, online service and use which were further subdivided into 17 sub-criteria.The most important criteria were found to be trust and online service, with security and tracking order status being the most critical factors under each of these two criteria.Younghwa and Kenneth (2006) in their study was focused on selecting the most preferred website based on website quality factors and their relative importance.The association between website preference and financial performance was well considered.DeLone and McLean's Information System (IS) success model was extended through the application of AHP on some e-business companies in the study.
The objective of the study by Rekik et al. (2016) was to secure the best criteria for the evaluation of the quality of e-commerce websites using fuzzy ANP based on eight criteria.The study concluded based on the results that customer satisfaction and transaction security are the two most important criteria for a successful e-commerce website.Yi-wen et al. (2007) proposed the use of fuzzy AHP approach to evaluate e-commerce websites using four criteria which were further grouped into different sub-criteria.The results obtained when compared with the AHP method were found to be consistent.In another study, a model for web interface evaluation of e-procurement website was developed by Kabak and Burmaoğlu (2013).Combinations of DEMATEL, ANP and fuzzy set theory were used to develop the model.A total of 10 criteria were used and the results obtained from the study showed that standardisation, links, reliability and navigability are the most important criteria when evaluating e-procurement websites.
Similarly, shopping websites were evaluated using MCDM by Sun and Lin (2009).Fuzzy TOPSIS method was used to evaluate the competitive advantage of four shopping websites based on 12 criteria.It was discovered based on the results that security and trust are the most important factors needed to improve the competitive advantage of these websites.Vatansever and Akgu (2014) applied fuzzy AHP to measure the service quality of private e-shopping websites in Turkey.Four criteria and 22 sub-criteria were used.From the analysis of results, vendor-specific quality was the most significant factor which affected the quality of the website.This was followed by service quality, system quality and information quality, respectively.An exploratory study to determine usability factors in e-commerce websites was also conducted by Pearson and Pearson (2008).Five criteria involving: navigation, customization and personalisation, download speed, accessibility and ease of use were used.It was discovered in the findings that, ease-of-use and navigation were two critical components in determining e-commerce website usability.Yu et al. (2011) integrated AHP and fuzzy TOPSIS to rank five e-commerce websites based on nine criteria.The criteria were speed, confidence, appearance, price, abundance, intelligence, security, ease-of-use and trust.Dey et al. (2015) developed a hybrid evaluation model that combined the use of AHP and fuzzy TOPSIS.Six major e-shopping websites of India were considered as alternatives and six important criteria factors which influenced online shopping the most were taken into consideration.Results obtained from the study indicated that price and quality of product, purchase security, account privacy statement and customer support were five top most influential criteria in online purchasing in the Indian market.A fuzzy MCDM approach for evaluating B2C e-commerce websites was also developed by Liang et al. (2017).It was developed using single-valued trapezoidal neutrosophic decision-making trial and evaluation laboratory (SVTN-DEMATEL).Four criteria of efficiency, fulfilment, system availability and privacy were used to evaluate six e-commerce websites in China.The outcome of the research showed that impact factors affecting e-commerce services were affected by different priority levels and interrelationships.

MCDM Approach in E-government Website Usability and Quality Evaluation
MCDM approaches have also been applied in e-government website evaluation.Byun and Finnie (2011) in their study proposed AHP method in order to assess the usability of e-government websites and ranking of Australian state government portals.Six main usability criteria and 59 sub-criteria were used.Markaki et al. (2010) applied fuzzy AHP to evaluate the quality of e-government websites using five criteria subdivided into 17 sub-criteria.The results obtained indicated that the e-service axis was the most important factor which affected the overall quality of the public authority websites.Other essential criteria included website content and technical performance.Fuzzy AHP was also applied in conducting a usability evaluation on e-government websites from a study by Lamichhane and Meesad (2011).It was based on four criteria and five websites were used as alternatives.Relevance and complete information about the services were found to be the most important sub-criteria.
Burmaoglu and Kazancoglu ( 2012) evaluated e-government websites in Turkey using fuzzy AHP and fuzzy VIKOR.It was based on three criteria of e-democracy, e-service and website design.These were subdivided further into different sub-criteria.The study concluded by recommending that e-service applications must be developed by different countries.Dominic et al. ( 2011) compared e-government websites in five Asian countries using a combination of Linear Weightage Model (LWM), fuzzy AHP, AHP, and a new hybrid model (NHM).The results of the study confirmed that quality and performance criteria were neglected by most Asian e-government websites.

MCDM Approach in Travel Website Usability and Quality Evaluation
Another important genre of websites are travel websites.A study by Kabir and Hasin (2012) examined important factors in the website quality of travel agencies based on user perception.The study further explored the use of MADM approaches to evaluate the service quality of travel websites.Five criteria grouped into 17 sub-criteria were used to develop the model.Soleymaninejad et al. (2016) evaluated two travel agent websites based in the USA using TOPSIS and six criteria in the study.Results from the study showed that the most important criteria were visibility, findability, functionality and accessibility.Furthermore, Khan and Dominic (2013) evaluated the website quality of four Malaysian airline websites using AHP based on criteria generated from web diagnostic online.The best airline with the most quality website was obtained at the end of the study.
Wen-Hsien et al. ( 2009) developed a model using DEMATEL ANP and VIKOR to evaluate five airline websites in Taiwan.The study concluded that Taiwanese airlines did not utilize the full potential of the Internet.It was also discovered that all the five websites did not perform well in price negotiation, low price, responsiveness and communication.Studies by Kabir and Sutana (2013) considered users' viewpoint in evaluating major factors of travel agency websites.Fuzzy TOPSIS was used for the evaluation.Hu (2009) developed a genetic-algorithm methodology to determine the degree of importance of respective criteria from fuzzy MCDM.Four representative travel websites in Taiwan were selected to determine the critical criteria of customers' concerns.A study by Lee et al. (2012) proposed a hierarchical MCDM evaluation model based on fuzzy AHP and fuzzy TOPSIS methods to evaluate the quality of four travel websites in Taiwan based on five aspects which was further subdivided into 17 criteria.
Another research proposed a fuzzy MCDM model to formulate the problem of travel website evaluation (Law, 2007).Dominic and Jati (2011) and Khan and Dominic (2013) in their studies developed MCDM models to measure the quality of Asian airline websites via web diagnosis tools by using AHP, FAHP and Linear Weighted Model (LWM).The newly proposed method was found to be effective in measuring the quality of airline websites.

MCDM Approach in other Website Genres
Other website genres where MCDM approach has been applied are discussed as follows.Zhang et al. (2015) conducted usability evaluation on four ecological park websites using a model that was based on an index layer, criterion layer and target layer.Three criteria of topicality, functionality and information elements were selected by using Group AHP.Presley and Fellows (2013) used AHP to evaluate the usability of three financial portals in the USA based on five criteria and 14 sub-criteria.The order of the importance of criteria was the ease of use, content, made-for-the-media, emotion and promotion.From the sub-criteria level, the three highest weighting were structure, goals and feedback.Aytuna et al. (2012) developed a model based on AHP to evaluate political websites in Turkey using five criteria which were functionality, efficiency, usability, reliability and interactivity.Findings from the study indicated that functionality and visibility were the main goals of the websites of Turkish political parties, with interactivity showing the lowest weighting.Alptekin et al. (2015) proposed the use of fuzzy TOPSIS method to evaluate the quality of five Turkish bookstore websites.Four criteria which were further categorised into 15 subcategories were used in the evaluation.Mirbargkar and Zadmehr (2015) investigated the quality of three hospital websites using ANP and fuzzy TOPSIS.Six major criteria and 19 sub-criteria were used.The results indicated that information quality criteria was ranked first.This was followed by assurance and the least was responsiveness.From the sub-criteria, information accuracy and trust were ranked first and second, respectively, with customisation ranked last.Tsai et al. (2010) used a combination of DEMATEL ANP, VIKOR and Weight Variance Analysis (WVA) to evaluate the quality of seven national park websites in Taiwan.Djordj et al. (2013) employed the use of ANP to measure the relative importance of usability factors of a defence ministry portal in Serbia.Three factors: usability, safety and flexibility were used for the analysis in the study.Bijan and Salehi (2013) in their study proposed a model to compare customer satisfaction indices of two e-recruitment websites in order to select the most preferred website in a specific context.The model was developed by merging the ANP approach with the America Customer Satisfaction Index (ACSI) for e-commerce.Kaya and Kahraman (2011) developed an assessment methodology for four e-banking websites using an integrated fuzzy AHP-ELECTRE approach.The criteria used were: information system (IS) quality, reliability, competence, access, customer service quality, security, ease-ofuse and product quality.An evaluation model to measure the performance of hospital websites based on quality by using two MCDM methods: AHP and PROMETHEE were developed by Bilsel et al. (2006).It consisted of seven e-service quality dimensions which were used to test the model.Kaya (2010) proposed a methodology for e-business website quality based on a multi-attribute approach.The model involved the use of a modified fuzzy TOPSIS approach, where a fuzzy AHP procedure was used to determine the weight of the evaluation criteria.Chou and Cheng (2012) developed a hybrid approach combining fuzzy ANP and fuzzy to evaluate the website quality of four certified professional accounting firms in Taiwan.Three criteria were used in the study which were further subdivided into 12 sub-criteria.The results showed that richness, comprehensibility, assurance, relevance, and reliability were the top five priorities.
Another important area where the MCDM approach has been widely used is in hotels and tourism.Akincilar and Dagdeviren (2014) evaluated fivestar hotels in Ankara, Turkey using AHP and PROMETHEE.The quality of e-service of four hotels using Webqual and fuzzy AHP was also conducted by Shahin et al. (2014).Three criteria were used in the study and it was found that the highest priority was information quality followed by usability dimensions and service interaction.In another study, fuzzy TOPSIS was used to evaluate and compare the ability and functionality of hotel websites in China based on usability.Four criteria: navigation, website friendliness (ease of use), language and overall layout were further subdivided into 25 sub-criteria which were used in the study (Qi, Law, & Buhalis, 2017b).Calisir et al. (2009) in their study used ANP to determine the relative importance of usability and functionality factors using two online auction and shopping websites as case studies.The findings revealed that users of these websites gave higher priority to usability than to functionality factors.More so, navigation and interaction were found to be factors of high relative importance.

ANALYSIS AND RESULTS
This section discusses the classification based on the following: year of publication, the genre of websites, MCDM methods employed, number of criteria used in the studies and location of websites by country.

Distribution of Academic Papers Based on Year of Publication
RQ1: What is the number of studies undertaken in the evaluation of website quality and usability using the MCDM approach over the years? Figure 3 provides information about frequency distribution by publication rate based on the number of articles per year.Fewer articles were published in the earlier years (2005)(2006)(2007)(2008)(2009).However, there has been considerable growth in the number of papers on MCDM approaches in terms of website quality and usability evaluation since 2010.The highest number of articles was published in 2017 and 61% of the total number of articles have been published since 2010.There is an expectation that more articles would be published in subsequent years.

Distribution of Academic Papers Based on Website Genre
RQ2 : What is the genre of websites which most commonly apply MCDM methods? Figure 4 shows the distribution according to the genre of websites.Websites from different domains like education, business, government, hospitality, tourism and agriculture have been used in previous studies over the years.The use of MCDM in e-commerce websites in terms of usability and quality takes the lead, as 28% of the reviewed papers focused on them.This is followed by  takes the lead, as 28% of the reviewed papers focused on them.This is followed by e-government (11%), travel (9%), academic websites (9%), airline (6%), hotel (5%), tourism (5%), e-learning (3%), e-banking (3%) and political parties (2%).This implies that the studies have cut across virtually every genre of websites except religious organisation websites.Figure 5 shows the distribution based on the MCDM approaches used in the reviewed papers.It can be seen from the figure that the fuzzy AHP approach is the most popular MCDM approach used in website usability and quality evaluation.It has slightly taken over the AHP approach due to its integration with fuzzy logic.It can also be observed that AHP and FAHP are mostly integrated with other methods in order to provide a better model and better results.Previous studies have also made this observation (Kubler et al., 2016;Mardani, et al., 2015;Zare et al., 2016).Another observation is that there is a proliferation of new approaches being developed in conjunction with soft computing techniques.The common trend, however, is that most of these methods are integrated with each other.
This is necessary in order to handle different operations being performed with different MCDM Airline Academic 9%

Distribution of Academic Papers Based on MCDM Approach
RQ3: What are the MCDM approaches used in website usability and quality evaluation?
Figure 5 shows the distribution based on the MCDM approaches used in the reviewed papers.It can be seen from the figure that the fuzzy AHP approach is the most popular MCDM approach used in website usability and quality evaluation.It has slightly taken over the AHP approach due to its integration with fuzzy logic.It can also be observed that AHP and FAHP are mostly integrated with other methods in order to provide a better model and better results.Previous studies have also made this observation (Kubler et al., 2016;Mardani, et al., 2015;Zare et al., 2016).Another observation is that there is a proliferation of new approaches being developed in conjunction with soft computing techniques.The common trend, however, is that most of these methods are integrated with each other.This is necessary in order to handle different operations being performed with different MCDM methods like selection, ranking and prioritisation.Table 3 further gives a breakdown of the papers showing the associated authors with the MCDM approach used in various studies including TrIFMAGDM, a triangular intuitionistic fuzzy multiple attribute group decision-making.Table 3.
Publications by authors based on MCDM approaches

Distribution of Academic Papers Based on the Number of Criteria Used
RQ4: What is the number of criteria often used in MCDM methods?Another important trend to know is the number of criteria used in the studies.This is very important as MCDM works based on criteria and in some cases, are divided into sub-criteria.Figure 6 shows the number of criteria used in both website quality and usability evaluation studies.The most commonly used number of criteria as observed from the study is five while the maximum number of criteria used is 12. Saaty and Ozdemir (2003) in their study, suggested that the number of criteria should not be more than the magic number seven plus or minus two.This is to reduce the effect of too much cognitive load on human computational ability.It was affirmed that humans could judge criteria using pairwise comparison with a few errors if the set criteria were limited.This assertion is further corroborated by Byun and Finnie (2011).criteria used is 12. Saaty and Ozdemir (2003) in their study, suggested that the number of criteria should not be more than the magic number seven plus or minus two.This is to reduce the effect of too much cognitive load on human computational ability.It was affirmed that humans could judge criteria using pairwise comparison with a few errors if the set criteria were limited.This assertion is further corroborated by Byun and Finnie (2011).In this study, attention was focused on website affiliation rather than author affiliation as has been done in some previous work (Behzadian, Khanmohammadi, Morteza, & Ignatius, 2012;Mardani, Jusoh, Zavadskas, & Nor, 2015).Websites from 18 countries were used in the studies under review.
From the geographic distribution based on the countries in Figure 7, websites from Turkey (25%) takes the lead followed by India (18%), China (12%) and Taiwan (10%), respectively.Further, it was observed that in most cases, the author's country of affiliation was also the same as the website's affiliation.Therefore, most of the websites frequently used in the studies were from the Asian continent.In contrast, there was virtually none from the African continent.Perhaps some studies could have been retrieved from countries like France, Iran and Japan; however this was not possible due to the English language exclusion criteria used in these studies.Figure 7 clearly shows that authors from Asia and especially Turkey, are active in MCDM research.A similar trend on the dominance of the Asian continent in publications on MCDM applications was observed by Behzadian et al. (2012) and Mardani et al. ( 2015) where it was reported that most of the authors from the study were from Taiwan, China and Turkey, respectively.some studies could have been retrieved from countries like France, Iran and Japan; however this was not possible due to the English language exclusion criteria used in these studies.Figure 7 clearly shows that authors from Asia and especially Turkey, are active in MCDM research.A similar trend on the dominance of the Asian continent in publications on MCDM applications was observed by Behzadian et al. (2012) and Mardani et al. ( 2015) where it was reported that most of the authors from the study were from Taiwan, China and Turkey, respectively.

CONCLUSION AND RECOMMENDATIONS
This paper has presented an extensive review of the literature on MCDM approaches in terms of website usability and quality evaluation.A total of 63 scholarly articles retrieved from reputable journals and conference proceedings were carefully analysed.From the review, a variety of individual and integrated methods were proposed for website quality and usability evaluation.On the whole, most studies focused on the aspect of website quality compared to studies on the usability aspect which is relatively small but growing.Thus not much work has been conducted in website usability using MCDM approaches as compared to website quality.The most popular methods are AHP and fuzzy integrated with AHP, and five is mostly the number of criteria used in the studies.Besides, there are more studies on e-commerce websites than any other genre of websites.Despite the popularity of academic websites, not much attention has been devoted to these websites from the MCDM perspective.
In addition, some websites such as political and religious websites are not well represented.The most popular criterion from a quality perspective is usability, although the most important criteria from a usability perspective are ease of use and navigability.More so, most studies are from websites affiliated with Turkey followed by India.On the other hand, there is a dearth of studies from African countries and hence the quality and usability of their websites are not adequately represented and studied.Furthermore, integrating MCDM methods with artificial intelligence techniques like Artificial Neural Network (ANN) and genetic algorithms (GP) although gaining popularity in other areas has yet to be widely applied in website usability and quality evaluation.In an area like MCDM application to supplier selection and evaluation, AHP-GP is the most popular integrated approach (Ho et al., 2010).
One of the limitations of this study is that the results are based on data obtained from scholarly journals and conferences.These do not include masters theses, doctoral dissertations, textbooks, and unpublished working papers in the MCDM literature.Besides, only articles published in English were used, hence publications in other languages were not considered.The focus was on the quality and usability aspects of the websites, thus other aspects like functionality, maintainability, efficiency and security could be considered in future studies.Therefore, further studies that include all these aspects and other databases not covered could be considered as well.

Figure 3 .
Figure 3. Distribution of publications based on year

Figure 4 .
Figure 4. Distribution according to website genres.
18 would be published in subsequent years.

Figure 3 .Figure 4
Figure 3. Distribution of publications based on year

Figure 4 .
Figure 4. Distribution according to website genres.

Figure 5 .
Figure 5. Number of publications based on MCDM approach(s).
methods like selection, ranking and prioritisation.Table 3 further gives a breakdown of the papers showing the associated authors with the MCDM used in various studies including TrIFMAGDM, a triangular intuitionistic fuzzy multiple attribute group decision-making.

Figure 5 .
Figure 5. Number of publications based on MCDM approach(s).

Figure 6 .
Figure 6.Number of criteria used in each website focused on website quality.

Figure 6 .
Figure 6.Number of criteria used in each website focused on website quality.

Figure 7 .
Figure 7. Distribution of publications based on website location by country.

Figure 7 .
Figure 7. Distribution of publications based on website location by country.

Table 1
Studies on website usability evaluation using MCDM approach

Table 2
Studies on website quality evaluation using MCDM approach

Table 3
Publications by authors based on MCDM approaches