EVALUATING THE PERFORMANCE OF COLOMBIAN BANKS BY HYBRID MULTICRITERIA DECISION MAKING METHODS

The aim of the study in this paper is to show how the performance of banks can be evaluated by ranking them based on Balanced Scorecard (BSC) and Multicriteria Decision Making (MCDM) methods. Nowadays, assessing the performance of companies is a vital work for finding their weaknesses and strengths. The banking sector is an important area in the service sector. Many people want to know which bank performs best when entrusting their money to them. For assessing the performance of banks, BSC can be used. This method helps to translate strategic issues to meaningful insights for the respective financial institutions. After that, the banks will be ranked based on performance indicators by the Weighted Aggregated Sum Product Assessment (WASPAS) method. Because this method is based on a decision matrix, weights are required. To find such weights, the Step-wise Weight Assessment Ratio Analysis (SWARA) method is applied. The results show that the International Bank of Colombia has a much better performance than other Colombian banks. Besides, further insights regarding the evaluation process based on BSC, SWARA, and WASPAS are obtained.


Introduction
Nowadays, many factors affect the performance of banks. One of the most important is globalization (Claessens & Horen, 2014), which induced many banks to revise their performances and try to improve them. As a result of globalization, many customers want the same services they would get from outstanding banks, and they prefer to use services based on state-of-the-art technologies (Lerner et al., 2018).
For these reasons, many researchers have published studies on the performances of banks (LaPlante & Paradi, 2015;Shafiee et al., 2016;Sun et al., 2017). Moreover, many banks attempt to increase their performance. Many private banks aim at entering foreign markets and gain market shares in the respective countries. To attract customers, they invest in advanced technologies and provide improved services. This leads to major challenges for domestic banking companies.
The Colombian banking sector is no exception to this rule. Based on a 2017 statistics of the World Bank, the economy of Colombia improves dramatically. The GDP increased from 4.04 billion dollars in 1960 to 282.46 billion dollars in 2016. In addition, the population size increased approximately from 16.5 million persons in 1960 to 48.6 million persons in 2016. The rate of people living in poverty fell sharply from 49.7% in 2002 to 28% in 2016. These developments show that the economic situation in Colombia improved drastically during these years. As a result, the number of interactions between customers and banks has increased as well.
The objective of this research is to assess and measure the performance of banks in order to rank them based on various indicators. This ranking helps customers get insights into the banks' performances and make a decision about entrusting their money to them. Another merit of this paper is that it provides a guide for banks to improve their performances and become excellent banks.
The aim of this paper is to prioritize six selected banks in Colombia, based on Performance Indicators (PIs). These performance indicators are obtained from a Balanced Scorecard (BSC) model. For ranking these PIs, a hybrid approach is used. As the main technique of ranking the Weighted Aggregated Sum Product Assessment (WASPAS) method is applied. As this method is a decision matrix method, weights must be determined for aggregating the different criteria. To obtain these weights, the Step-wise Weight Assessment Ratio Analysis (SWARA) method is used. Finally, banks are ranked based on these PIs, the obtained weights, and the Multicriteria Decision Making (MCDM) method WASPAS.
The contributions of this paper and the gap in the existing literature can be addressed as follows: 1. One of the original contributions is to suggest and use a hybrid model based on a combination of the SWARA and the WASPAS methods. Both metods are comparably easy-to-use, reliable, and are well suited for their combined application. An extensive literature search did not reveal any other paper dealing with the combination of these methods in the banking sector. 2. Another contribution of this paper is the case study on Colombian banks. There was no evidence of similar research on the performance measurement of banks in Colombia. Thus, the paper addresses two research questions: How can the performance of Colombian banks be evaluated? Is it possibly to use both SWARA and WASPAS in a BSC context for performance measurement? The combined usage of both methods appears to be promising since SWARA provides a convenient and reliable way to asses criterion-specific weights, while WASPAS is a suitable tool to utilize such weights taking into account how the different criteria could be aggregated.
The rest of the paper consists of these parts: After the introduction, a literature review on the banking industry and BSC is depicted. Section 2 illustrates MCDM methods. The research methodology is shown in Section 3. Section 4 concerns data analysis and presents the results of the application of the methods mentioned. In Section 5 managerial implications are discussed. In the final section, the conclusions are presented.

Banking industry
The banking industry is one of the most important industries in the field of providing services to people. Everyday, numerous people require diverse banking services. Usually, in a country there are domestic and foreign banks with a strong competition among them. Therefore, the performance of banks is a popular topic among researchers. Approaches to assess the banking industry can be divided into two main categories. First, there are diverse kinds of studies based on Data Envelopment Analysis (DEA). Secondly, there are various studies that use MCDM methods. Let us note, that DEA and MCDM methods found wide areas of applications such as, for instance, for supplier selection (Beheshtinia & Nemati-Abozar, 2017), for prioritizing renewable power plants' construction (Sedady & Beheshtinia, 2019), or for evaluating the quality of hospital services (Torkzad & Beheshtinia, 2019). However, in the following the literature review only considers respective studies in the field of banking.
The approaches based on DEA can be subdivided into four types. First, there is classic DEA which is mentioned in the follwing papers: Paradi and Zhu (2013) applied DEA for an analysis of the effienciency of banks. They found out that there were diverse inputs and outputs for assessing banks and they could mix the results of DEA with other methods. Emrouznejad and Anouze (2010) assessed the banks of the Persian Gulf Cooperation Council countries using DEA and classified them using a regression tree. The result showed which factors affected the efficiency of these banks. Another study is related to network DEA. Paradi et al. (2011) demonstrated a two-stage DEA method for bank branch efficiency analysis. They found out that small and medium branches were more efficient regarding both production and profitability. The final subcategory relates to applications of fuzzy data in DEA. Chen et al. (2013) assessed the performance of business banks by fuzzy DEA. They concluded that when data of inputs and outputs were not deterministic, they had to use fuzzy numbers and, in particular, triangular fuzzy numbers helped them solve the problem. As a fourth variant of DEA, let us mention that it became popular to use DEA in connection with Malmquist indicators which are traditionally used to compare two econmomies regarding their performance using a given production technology. For instance, Emrouznejad and Yang (2018) have found more than 300 publications dealing with DEA in combination with Malmquist indicators. Also in the field of banking such approaches are used occasionally, for instance in Asmild et al. (2004) where Candian banks were evaluated based on data from a 20 years period. In Van der Westhuizen (2008) the efficiency and causes for efficiency change were evaluated for the four largest banks in South Africa for a period of 36 months. Majumdar and Asgari (2017) studied the performance measurement of UAE companies by DEA Malmquist. They used labor, capital, materials, revenue and EPS as the inputs and outputs of the model. The result showed that the performance of these companies will increase because of advanced technology, investment and new production. Guo et al. (2017) evaluated hospitals performance of Hong Kong by DEA Malmquist and Tobit regression. Inputs and outputs of this research were the number of staff, the number of beds, the discharge rate, the length of stay, the total ED attendances, the total outpatient attendance, and the crude mortality rate. The result revealed that rich people like to receive better services in private hospitals. Wu et al. (2017) used DEA Malmquist for measuring energy and environment performance in China. Labor, capital, energy, GDP and waste gas were inputs and outputs of this study. The result demonstrated that most provinces had low energy and environmental efficiency.
The second category of approaches deals with using MCDM methods for both crisp and fuzzy data. Most researchers used MCDM methods with crisp data. In early publications using MCDM for the analysis of banks, mostly financial data such as those based on balance-sheet analysis were used, see, e.g., Hanne (1995), while later the set of criteria was expanded. Dash (2017) showed how the PROMETHEE method helps to create a model for the measurement of bank performance. He used the CAMEL method for extracting factors and then prioritized the banks based on these factors. Fallah Jelodar (2016) showed a model for prioritizing factors for bank efficiency by the combination of DEA and the Analytical Hierarchy Process. The result shows which banks are efficient and which are inefficient and also which factors are affected by the performance measurements. The second category deals with the applications of MCDM with fuzzy data. Akkoç and Vatansever (2013) demonstrate how the fuzzy Analytical Hierarchy Process (AHP) and the fuzzy TOPSIS (Technique for Order of Preference by Similarity to Ideal Solution) helped the Turkish bank sector to defeat the global financial crisis. The results obtained by both approaches were similar and led to the same ranking. Beheshtinia and Omidi (2017) assessed the performance of banks in Iran by a combination of fuzzy MCDM methods. They found out that some factors such as return on investment, debt ratio, and lower energy consumption were very important factors of a bank's performance. Shaverdi et al. (2011) used fuzzy MCDM methods together with BSC for the Iranian private banking sectors. The result showed that the customer's perspective was most important and that customer satisfaction was the most important performance measurement criterion in the study.

Balanced scorecard
There are various methods for evaluating companies. One of those is the ISO standard that is used in more than 70 countries. Total Quality Management (TQM) is another method for assessing companies. This method is based on the philosophy and concept of increasing customer satisfaction. Management by Objective (MBO) is a third method that demonstrates how companies work to achieve objectives set before. Management by Values (MBV) points out all plans, make decision conducts based on value. The Malcom Baldrige National Quality Award is an annual prize for USA organizations that recognizes their efforts to achieve qualitative objectives. The aim of this award is increasing quality awareness as the important element of competitive strength and profit of organizations. Activity Based Costing (ABC) is the another method for evaluating companies. Using this method companies assess perfor-mance based on solely financial parameters while other parameters such as internal processes or HR are not considered. Hoshin Kanri is another method for evaluating companies (and performance programs of companies) based on a plan-do-check-act (PDCA) cycle. Data Envelopment Analysis (DEA) is a method of evaluating companies based on linear programming and related models based on inputs and outputs of a company. This method evaluates DMUs (decision making units) based on a score of inputs and outputs and then ranks them. As a result, it is shown which DMUs are efficient and which of them are inefficient. Table 1 shows the comparison of these methods. Due to some weaknesses in these approaches and because Balanced Scorecard is one of the most frequentsly used approaches for performance assessment, this methodology is considered in the following. Balanced Scorecard (BSC) is a management framework that translates an organization's strategic objectives into a set of performance measures to be applied to customer, product, process and market development. From this model, people understand that using only financial performance measurements is insufficient for measuring the performance of an organization. Therefore, additional perspectives such as internal processes as well as learning and growth should be taken into account (Kaplan & Norton, 1992). One of the merits of BSC is that it depicts cause and effect relationships among performance indicators (Kaplan & Norton, 1996c). This model consists four dimensions: 1. Financial perspective: Before introducing BSC many companies evaluated the performance of their companies by financial indicators which show how successful they are in achieving their goals. Kaplan and Norton demonstrated that although financial indicators had an important role in the evaluation of companies they cannot solely ensure the achieement of goals and fulfilling the company vision. For reaching the financial targets many factors are needed. 2. Customer perspective: One of factors that effect financial indicators are customers indicators. Researchers understand that for increased targets of financial indicators such as profit, revenue, and so on they need to focus on customers and especially increase their loyalty and satisfaction. As a consequence financial indicators increased dramatically.
3. Internal perspective: After recognizing the role of customers, companies attempted to increase the target of customers indicators. Meanwhile they understood that some internal factors have a significant effect on customer loyalty and satisfaction. These factors are reengineering, process management, supply chain management and so on. They can cause that work will be done with low total cost and within less time than before. 4. Learning and growth perspective: Many companies are looking forward to find out how improvement programs of the internal perspective can be successfully implemented. Often failures were observed during respective implementation projects. As a result of various studies, the key role staff during such projects was understood. Staff must be trained and care must be taken to overcome the possible resistance of improvement project implementations (Kaplan & Norton, 1992, 1996a, 1996b. After introducing BSC, many papers were published, most of them focusing on a combination of BSC with MCDM methods. One of the popular MCDM methods used in this field is the Analytical Hierarchy Process (AHP). To address this issue, Sundharam et al. (2013) used BSC and the AHP to analyze the growth of manufacturing industries. They showed that the combination of these two tools helped figure out which are the best factors to evaluate industries. Bentes et al. (2012) studied the assessment of organizational performance with BSC and AHP. They showed that AHP had some drawbacks and to eliminate them, a groupbased version of the AHP was used. Y. H. Kim and M. Kim (2010) created a new model for assessing websites by using BSC and AHP. They set CSFs (Critical Success Factors) for the evaluation of websites by BSC and prioritized them. The result showed that the customer perspective was the most important perspective. Another MCDM tool for ranking performance measurement of BSC is TOPSIS. Asli et al. (2013) used TOPSIS for prioritizing strategy planning and BSC. They discussed how factors for prioritization can be extracted, based on a combination of a strategy map and BSC. The term strategy map refers to a diagram that is used to document the primary strategic goals of an organization or management team and is characteristic for BSC approaches of the second generation. Sadeghi et al. (2013) applied grey TOPSIS and BSC for defining a strategy map. They illustrated how TOPSIS helped managers find out the most important factors of BSC. The Analytic Network Process (ANP) is another MCDM technique. Hu et al. (2015) pointed out how to measure knowledge resources by BSC and ANP. The result showed that this method contributed to clarifying the measurement of the performance of knowledge resources. Some papers were published which use a combination of fuzzy ANP and BSC. Bhattacharya et al. (2014) clarified how Green Supply Chain Management can be analyzed with fuzzy ANP and BSC. This model helped managers prioritize factors of Green BSC in manufacturing. Zolfani and Ghadikolaei (2013) combined DEMATEL, VIKOR and ANP with BSC to evaluate private universities in Iran. They showed that among four BSC perspectives the perspective of the internal processes is the most important one. Dincer and Hacioglu (2013) explained a model for the performance assessment of Turkish banks with fuzzy VIKOR, AHP and BSC. The result elucidated that private banks had higher performance levels than foreign banks. Sofiyabadi et al. (2016) evaluated the service business sectors by fuzzy VIKOR and BSC. From the combination of these two methods and the implementation in the company LG they concluded that this company should focus on the perspectives customer, learning and growth. One of the tools for creating strategy maps is DEMATEL. This method shows cause and effect relationships among factors. Valmohammadi and Sofiyabadi (2015) studied how one can create strategy maps of Iranian automobile companies by fuzzy DEMATEL and BSC. Kala and Bagri (2016) used DEMA-TEL for performance measurement of BSC for creating strategy maps of hotels. Varmazyar, Dehghanbaghi, and Afkhami (2016) explained how to use BSC and a hybrid MCDM model for the evaluation of research and technology organizations. They used DEMATEL, Additive Ratio Assessment (ARAS), Complex Proportional Assessment (COPRAS), Multi-Objective Optimization by Ratio Analysis (MOORA) and TOPSIS.
As mentioned before, the main contribution of this paper is the use of the SWARA and the WASPAS methods for ranking the performance of banks. For the WASPAS method, weights are required as preference-based information. There are many methods which contribute to finding these weights such as AHP, ANP and so on but when the number of items is increased beyond six items, the inconsistency rate rises significantly, which makes the finding of correct weights very difficult. Therefore, another method is required that does not have this drawback. The SWARA method is one of the methods which fulfils this purpose. There is no evidence of the use of these two methods for asessing the performance of banks in previous research. Table 2 indicates performance indicators used in the discussed research.  (2017)

Hybrid approach 2.1. Step-wise Weight Assessment Ratio Analysis (SWARA)
The SWARA method is a popular tool for calculating weights related to performance measurement and the resulting importance levels.
Step-wise Weight Assessment Ratio Analysis (SWARA) (Keršuliene et al., 2010) is one of the MCDM methods which are based on a rational dispute resolution method (Keršulienė & Turskis, 2011). In this model, experts have key roles. First, they express their preferences. Then, based on the average expert's judgements, the comparative importance of each aspect is calculated and the results are sorted in descending order. The weight of each criterion is calculated based upon the relative importance of the next more important criterion.
The steps of the SWARA method are as follows: Step 1. First of all, all criteria are sorted from top to down by estimation of significance.
Step 2. Consider second criterion, experts specify the relative importance of criterion j compared to the previous (j -1) criterion. The ratio of this comparison is called comparative importance average of value or . j S Step 3. The coefficients j K are obtained by following equation: Step 4. The recalculated weights denoted as j q are computed as follows: Step 5. The weights of criteria are identified by where j W is the weight of criterion j.

Weighted Aggregated Sum Product Assessment (WASPAS)
This method was introduced by Zavadskas et al. (2012). In this method the aggregated Weights Sum Model (WSM) and the Weighted Product Model (WPM) are used for better accuracy in decision making. The following steps illustrate the WASPAS method.
Step 1. Decision matrix: In the first step a decision matrix will be created.
In this matrix, m is the number of alternatives and n is the number of criteria. ij x is an indicator of the performance of alternative i regarding criterion j.
Step 2. Normalized data: The data will be normalized based on the following formulas.
For benefit criteria (to be maximized) For cost criteria (to be minimized) It is assumed that the ij x values are positive to avoid interpretation problems or division by zero. The normalized values of ij x are shown as ij x .
Step 3. Calculation of the total relative importance of an alternative: In the WASPA method, a joint optimality criterion based on two subcriteria is used. The first subcriterion for optimality can be interpreted as weighted success and it is similar to the one used in the WSM (Weights Sum Model) method. This method is among the popular MCDM methods for evaluating a finite set of alternatives. Based on the WSM method the total relative importance of alternative i is calculated as follows: In this formula, the j w values are the weights of the jth criterion. For the other subcriterion, the computing formula for the total relative importance of an alternative is as follows: Step 4. Total relative importance of an alternative: Based on the combination of the two above criteria an overall criterion is calculated as follows: The ranking of alternatives is based on the Q values and the best alternative is the one which has the highest Q value (Šaparauskas et al., 2011). In the above formula, if 0 λ = the WASPA method changes to the WPM (Weighted Product Model) method and in case of 1 λ = the method corresponds to the WSM method Step 5. Calculation of optimal values for λ In Zavadskas et al. (2012) it is suggested to calculate values for λ which lead to a maximal accuracy of estimation of the Q values. Assuming that errors in determining the initial criteria values are stochastic, the variances of the two criteria ( ) The variance of the total relative importance of an alternative can then be calculated as Optimal values of λ of an alternative can then be calculated (assuming that the derivative of the above function should be 0) as follows: The estimates of variances of normalized initial criteria values can be calculated as follows Here, it is assumed that these values are normally distributed and that the uncertainty can be expressed by a exponential coefficient k = 0.10 of the criteria values and a further multiplier t = 2.0 depending on the distribution law of errors. The variance of the (standardized) initial criteria values can then be expressed as for a credibility level q = 0.05.

Reasons for choosing WASPAS and SWARA
Based on various research, Zavadskas et al. (2012) believe that the WASPAS technique is a better technique than the Weighted Product Model (WPM) and the Weighted Sum Model (WSM). In addition, among MCDM techniques usually one must select either an additive aggregation of criteria values (e.g. a multi-attribute utility function) or a multiplicative form of aggregation. Instead, the WASPAS method allows to select both types of MCDM aggregation techniques. Of course, there are various more complex methods in the field of MCDM which make smaller assumptions concerning the aggregation possibilities of criteria such as the family of outranking methods. However, WASPAS is still a rather uncomplicated and easy-to-use method which makes it attractive in practical settings. The SWARA method for determining weights is flexible and works economically regarding required information input (such as comparisons) (Mardani et al., 2017). In particular, SWARA, works with a smaller amount of information to be assessed for determining weights (n -1 judgements) compared to some other MCDM approaches, especially those which rely on pairwise comparisons (1/2 (n 2 -n) judgements) which is especially important for a larger number of criteria and when dealing with experts with a small amount of available time (e.g. managers). In addition, the method is not oversimplicstic as it does not require to determine weights directly which is know to be cumbersome when dealing with decision makers.

Research methodology steps
The steps of this research are the following: Step I: In this step, PIs of banks are extracted from strategic planning based on BSC.
Step II: Next, weights for each PI are to be determined based on the SWARA method.
Step III: Define sampling of this research by Cochran's formula (see below).
Step IV: Designing a questionnaire based on PIs, banks and the WASPAS method.
Step V: Data gathering from questionnaires.
Step VI: Data analysis by the WASPAS software.
Step VII: Ranking banks based on PIs and the WASPAS method.
In the subsequent subsection the main steps are illustrated.

Identification of PIs
As mentioned before, BSC is a tool to translate strategy into a common language that everybody can understand. In this paper, the performance measurement of BSC is based on the following aspects: -Financial perspective: total assets, income, total cost, debt, ROI, net profit.
-Customer perspective: customer satisfaction, market share, number of customers, new customer rates, response time to customer requests. -Internal processes perspective: cost of R&D, number of improved processes, introduction of new services. -Learning and growth perspective: staff satisfaction, motivation index, productivity of employees, staff training index. Although it is possible to add further perspectives to BSC models, the standard perspective appeared to be sufficient for the considered purposes. For instance, modern aspects in banking services and performance, such as innovations and IT support can be found among the existing perspectives as they have impact, e.g., on customer satisfaction, the improvement of internal services, or the productivity of employees. In addition, aspepts such as those relating to technology and innovation did not appear to be most relevant as the considered Colombian banks are, in general, lagging behind international top banks and FinTech newcomers. Therefore, additional perspectives were not considered. Table 3 shows the PIs with their definitions and respective references. Debt reveals to money, that is owed to someone else, or the state of owing something Faraglia et al. (2008), Feng and Wang (2000) Total cost It refers to both fixed costs and variable costs Productivity of staff Indicator of the productive staff of a company in percent Carmen (2015a, 2015b)

Weighting the PIs
These PIs are ranked by the SWARA method. In this step, weights are to be determined for the performance indicators. First DMs were asked to specify their preferences for the indicators. These DMs were eleven top managers in the areas of banking and economics in Colom-bia. A group decision-making approach was used for both SWARA and WASPAS for ranking these indicators. Table 4 shows the weights and ranking of PIs by the SWARA method.

Data sampling
For data sampling, Cochran's sample size formula for the infinite population is used. The formula is as follows: where z -confidence level, p -the percentage of people in the population who have the attribute, (1 -p) -the percentage of people in the population who do not have the attribute, and d -the allowed error.
In this study, z corresponds to 95% or 1.96, p and (1-p) are equal to 0.5 and d is equal to 0.1. By putting these numbers into the formula the questionnaire needs to be distributed to 96 persons. As interviewees for this questionnaire Colombian individuals were considered that have an account at one of these banks. It was necessary to send the questionnaire to 107 randomly selected customers (because of nonresponding customers) in order to get the desired 96 responses.

Data collection
For data collection a questionnaire for the DMs was used. In this questionnaire, the DMs responses to questions regarding the performance of banks are based on PIs. In this study, six selected banks in Colombia were considered. These banks included all Colombian banks (six from seven banks) where access their data was possible. For confidentiality reasons, the names of the banks are not specified in the respective data and are indicated by B1, B2, B3, B4, B5, and B6. While five of these financial institutions are domestic banks, one is a branch of an international bank.

Data analysis
After distributing the questionnaires to the DMs, the resulting data are gathered and analyzed. First, the average of evaluations from the responses is calculated. Table 5 shows the decision matrix data of the WASPAS method.
The normalized data are shown in Table 6. The variance matrix as calculated by the WASPAS software for the individual criteria values is shown in Table 7.
The variance of data for the quality indicators ( ) Table 8.

Managerial implications
The banking industry has usually a key role in the economy of a country. Banks provide many services for customers. Banks can be divided into domestic and international banks. Today, these banks compete with each other to attract customers. As a consequence, they provide the customers with new high quality services. This competition helps customers choose the banks with the best services, terms, and conditions. Colombian banks are not exceptional in this regard. After international banks arrived in Colombia, a new rivalry started in the banking industry. As the number of banks in Colombia is small, the competition is rather strong. Therefore, Colombians want to assess these banks and find out which has the best performance. To assess their performance, the BSC method is used. It is a powerful tool for the translation of strategies into action. Based on BSC, first performance indicators are extracted. Then weights are allocated to these PIs based on the SWARA method. The result of the SWARA method revealed that among these PIs the net profit indicator from the financial perspective had the highest priority and weight, whereas the staff training index from the learning and growth perspective had the lowest priority and weight among the PIs. Based on the preferences specified by the DMs, six banks in Colombia, which included five local banks and one international bank, were assessed regarding their performance indicators. For evaluating these banks, 18 performance indicators in financial, customer, internal and learning and growth perspectives are considered. The banks are ranked based on the DMs preferences and, the performance indicators and the WASPAS method. As the WASPAS method is based on a decision matrix, specified weights are required. To obtain these weights, the SWARA method is used. The result shows that of the six banks the international bank has the highest performance from the DMs' viewpoints. This indicates that this bank provides suitable services on time and with a high quality. The next banks in the ranking are domestic banks. There is a significant difference between the score of the international bank and the domestic bank which ranks second. The score of the other banks did not differ much. To increase their performance, domestic banks need to benchmark with the international bank in terms of performance measurement criteria.

Conclusions
Based on a literature review, 18 performance indicators for banks were extracted which are relevant for strategic planning. These performance indicators were categorized into four categories of BSC, which are denoted as financial, customer, internal processes, learning and growth perspectives. Based on these perspectives, six performance indicators from the financial perspective, five performance indicators from the customer perspective, three performance indicators from the internal processes perspective, and finally, four performance indicators from the growth and learning perspectives were considered. This means that financial and customer perspectives and performance measurement had key roles for the assessment of these banks. Then, the performance indicators were ranked by using SWARA. Among these performance indicators the net profit had the highest priority and the staff training index had the lowest priority. As mentioned before, the ranking of banks were 1 4 6 2 3 6 B B B B B B > > > > > where the international bank (B1) had the highest performance and bank B6, which is a domestic bank, had the lowest performance.
During the study it turned out that both SWARA and WASPAS are rather easy to use and can be applied in a scenario involving experts (for some application domain) who are not familiar with these approaches. The approaches are easier to use (with respect to the elicited information and regarding their outcomes) than various other MCDM methods (such as some outranking approaches, for instance). On the oter hand, these approaches are not oversimplistic, e.g. regading the assessment of weights or with respect to the subsequent aggregation of criteria.
Regarding limitations of this study, it is well-known from other applications, that results of multicriteria problems may differ more or less as a result of different information used in different MCDM approaches and the way how the information is processed. For future research, therefore other methods can be used which are based on pairwise comparisons such as AHP, ANP, MACBETH, Rembrandt and so on for finding weights for evaluating a decision matrix for the considered application area. In addition, for ranking banks researchers can use other MCDM methods such as COPRAS, ARAS, TOPSIS, or VIKOR. It would be interesting to find out differences rearding the results obtained by different methods. In addition, the suggested methodology should be used for evaluating companies in other countries or from different industries.